The risks of using GenAI for legal research

As generative artificial intelligence (GenAI) software is increasingly foisted upon both private and public sector workforces, with the now familiar mantra of “boosting productivity”, many lawyers have attempted to harness some of the tools for legal research. In this article we consider the scale of the uptake and identify some of the risks, alongside expert comment from leading industry insiders.

What GenAI tools are being used?

At present, there are essentially two buckets of GenAI software being used for legal research:

Generalist

These include the big three – OpenAI’s ChatGPT, Google’s Gemini, Microsoft’s Copilot – alongside smaller players such as Perplexity, Claude and Deepseek. Although these tools are not specifically designed for the legal sector, they can provide useful answers to basic legal questions and are low cost or free, arguably a boon for access to justice. Many law firms actively encourage fee earners to use Copilot in particular, perhaps in part due to it being increasingly baked into Microsoft Office and Windows. Although these generalist large language models (LLMs) have the lowest barriers to adoption, they are the most prone to providing erroneous information or hallucinations.

Legal specific

The legal AI bonanza, recently predicted to be worth in excess of $10 billion by 2030, has been driven by the promise of more reliable GenAI tools, with Harvey recently raising $300 million funding at an eye watering $5 billion valuation. Incumbent legal publishing behemoths have released their own legal-specific LLMs (Lexis+ AI and CoCounsel from Thomson Reuters) which have been trained exclusively on their own repositories of legal information and therefore claim to be more accurate and less prone to hallucination. However, tests have revealed that even these expensive GenAI tools developed specifically for legal professionals cannot be relied upon.

How many lawyers are using GenAI?

According to a recent report from LexisNexis, 61 per cent of legal professionals now use generative AI for work purposes, with around half of them solely using generalist tools. However, 76 per cent of lawyers are concerned about using AI for legal research due to the possibility of inaccurate information and digital hallucinations, and only 17 per cent believe that AI is embedded in their strategy and operations.

Another recent report from Thomson Reuters reveals that, of those legal professionals currently using AI tools, 74 per cent use them for legal research – the second highest use case, just below document review at 77 per cent. For government lawyers, legal research is the highest use case of AI, at 69 per cent.

However, surveys only touch the surface of real world usage of GenAI by lawyers. Whilst many firms will have their own approved tools and policies for AI use, potentially half of all employees are covertly using AI at work every day, with work often being easily transferred between personal and corporate devices.

Risks of using GenAI for legal research

There have been several well publicised instances of litigants coming a cropper due to digitally hallucinated case citations, with eight incidents of fabricated citations being reported in the first week of September 2025 alone. An Australian lawyer recently faced professional sanctions for using AI tools without due diligence.

Back in the UK, in the case of Ayinde v Haringey [2025] EWHC 1383 (Admin), the Divisional Court, acting on Hamid referrals, had to consider two cases where a number of fake citations had been used by legal professionals during the course of litigation. Although neither case was deemed deserving of contempt proceedings, presiding judge Dame Victoria Sharp, issued a stark warning over the use of GenAI tools for legal research:

“Those who use artificial intelligence to conduct legal research notwithstanding these risks [of digital hallucinations] have a professional duty … to check the accuracy of such research by reference to authoritative sources, before using it in the course of their professional work (to advise clients or before a court, for example)”. In addition, she asked the Bar Council, the Law Society, and the Council of the Inns of Court, to “consider as a matter of urgency what further steps they should now take in the light of this judgment.”

But although hallucinated case citations seem to garner a lot of media attention when it comes to GenAI (mis)use by lawyers, there are obviously many other risks inherent in LLM based legal research, including:

failure to identify pertinent (and real) case law;
references to out of date legislation;
wrong jurisdiction (eg applying English law to a Scottish case);
inability to take into account recent developments (ie post-LLM training); and
inclusion of fake, misleading or inaccurate information.

Furthermore, without adequate prompting, legal research conducted using GenAI can miss the mark entirely, providing irrelevant answers and wasting fee earning time.

Should lawyers be using AI for legal research?

According to Will Richmond-Coggan, a partner at Freeths LLP specialising in data and AI law, the acceptability of lawyers using GenAI for legal research depends on the particular tool:

“There should be no circumstances in which lawyers should be using ChatGPT or other freely available chatbots to do legal research, any more than they should be looking up the answer via a search engine. The tendency of such chatbots to hallucinate, and the difficulty that there is in tracing the origins of general propositions that the chatbot confidently states to be applicable, make it next to useless as a serious research tool. A number of lawyers have received stern warnings over their deployment of citations generated by chatbot hallucinations, without having validated these, and it should be expected that future issues in this area will be dealt with increasingly severely.”

Will gives more of a nod to legal specific tools, but not without reservations:

“The paid for specialist legal AI tools ought to be able to be relied on to a greater extent, but are still not without their flaws. The technical legal language used in judgments is a long way removed from the general language on which many AI models are originally trained, leading to difficulties parsing the meaning of decisions, and a difficulty in distinguishing the ratio (or substantive decision and reasoning) from obiter observations which are not intended to carry the same weight. In our use of a range of tools we have found them to be good for identifying potential lines of investigation, but some way from being able to generate a comprehensive reliable note in the way that a PSL or trainee might be expected to do. Our rule is that ultimately every piece of work that is produced is the responsibility of the lawyer producing it, regardless of the extent to which their work has been supported by AI in the background, and needs to be checked and verified by them accordingly.”

This point about lawyers taking ultimate responsibility is reflected in the ruling of Dame Victoria Sharp on her point of “professional duty”.

Brian Inkster, Founder and CEO of Inksters, is more sceptical on the usefulness of any form of GenAI for legal research:

“It is well known that GenAI hallucinates. This is not confined to generic/free ones such as ChatGPT. It has been found to equally happen within legal specific tools such as Lexis+ AI. Furthermore, there will be legal authorities that you may wish to rely on that have not been digitalised within the data set of the LLM you are using. It simply won’t find them even if it doesn’t hallucinate in the process.”

Although many people say GenAI is useful for creating summaries of case law, Brian warns that there is a “great danger that if you only read a summary of a case you miss an important paragraph, sentence or point that is significant to the case that you are working on.” He says that there “is no substitute for reading case law in full” during legal research, and says that “it could take you twice as long to check and correct GenAI output than it would take to simply do the legal research the tried and tested old fashioned way (‘Inkster’s Law‘).”

Brian is unequivocal in his stance on using GenAI for legal research:

“There is, sadly, an ever increasing number of examples of lawyers using GenAI to cite cases in court without having carried out those checks and corrections. I personally wouldn’t touch it with a barge pole for legal research. We have no specific AI policies at Inksters, but I think my team of solicitors are all very aware of my opinions on the current limits of GenAI and hopefully would not be as daft as to use it for legal research.”

But many law firms (and businesses more generally) whose stance on AI is less clear, could probably benefit from AI policies – both in terms of providing guidance to their own staff and transparency for their clients.

Corinne McKenna of Lawtelligence helps to train law firms on how to effectively use AI within their marketing processes. But she argues that lawyers should not be using GenAI for actual legal research:

“Legal professionals study the law, not to memorise cases and legislation, but to become equipped with the mental tools to understand them and how to interpret the law as it applies to different fact situations. Because LLMs can only predict the next logical word in sequence, they are completely unable to analyse a set of facts and apply the law. And why would they? They are effectively nothing more than a giant autocorrect. This is the fundamental reason why LLMs cannot be used for legal research.”

This fundamental point – that GenAI in its current form does not actually “understand” its own output and is simply creating strings of words based on an algorithm – is worth bearing in mind.

Joanne Brook, Consultant Solicitor at Lionshead Law, warns that AI poses an existential threat to the Rule of Law, and potentially to the legal profession itself:

“We know that AI poses a threat to how law is practiced and to jobs within the legal profession, turning lawyers from the source of knowledge to curators of the AI output. We know it poses an existential threat to the Rule of Law too and many studies have shown us that, despite law being about certainty and accuracy, many cases quoted and laws stated by AI are fiction.”

She notes that the Solicitors Regulation Authority (SRA) identified back in 2023 – in its Risk Outlook report – that “there is evidence that people may place more trust in computers than in humans” and crucially that, “Errors by an AI system could affect many more people than errors by a single person could.” These dangers are compounded by the fact that the “high output of an AI system also makes it harder to supervise effectively”. Furthermore, Joanne alludes to Stanford University’s 2024 research which found that, for legal tasks, hallucination rates ranged from 69 to 88 per cent in response to specific legal queries:

“meaning that more often than not, AI makes it up. This has to be a major concern for anyone using AI for legal research – going into the search with the knowledge that more than half of what they will find will be inaccurate or just untrue and yet, little is being done to add scrutiny and impose accuracy on these systems, and the providers are infrequently called to account contractually.”

It’s worth noting that a follow-up study from Stanford found that even legal specific GenAI tools hallucinated in at least one out of every six queries.

Overall, the general consensus seems to be that lawyers should avoid using generalist GenAI tools entirely for legal research – and that even legal specific products cannot be relied upon without thorough human oversight (which may negate or even reverse any productivity gains).

The future of GenAI for legal research

Despite all the warning signs, the onward march of AI is only gathering momentum, with governments around the world encouraging its adoption and many law firms embracing the potential of boosting their profits by swapping ever increasing wage bills for more manageable licensing fees. Since LLMs are generally improving (although sometimes it looks like it’s a case of taking two steps forward and three steps back), it’s possible that lawyers will be able to rely on them for legal research at some point in the not too distant future – and the profession may even end up facing an existential crisis. On the other hand, firms may become increasingly wary of hallucinations and question the actual usefulness of the tools, turning the legal AI boom to bust. Whilst we await an answer, any lawyers using generative AI for legal research should exercise a traditional professional attribute which has stood the test of time: caution.

Internet for Lawyers Newsletter

Edited by Nick Holmes