Justice will come under threat from AI’S odd ‘hallucinations’
This week: The law and technology
THE euphemism we’ve settled on is a “hallucination.” Human beings experience hallucinations by chowing down on magic mushrooms, experiencing an acute psychotic episode, or by watching BBC coverage of great state occasions involving members of the royal family.
Artificial intelligence, by contrast, starts hearing voices when you ask it a simple question.
Did you know that large language models like ChatGPT are in the habit of embedding random but superficially-plausible falsehoods into the answers they generate? These are your hallucinations. Facts are made up. Counterfeit sources are invented. Real people are conflated with one another. Real world sources are garbled.
Quotations are falsified and attributed to authors who either don’t exist, or didn’t express any of the sentiments attributed to them.
And troublingly, none of these errors is likely to be obvious people relying on the pseudo-information produced, because it all looks so plausible and machine generated.
We aren’t helped in this by uncritical representations of AI as the sovereign remedy to all ills – from Youtube advertisers hawking easy solutions to struggling workers and firms, to governments trying to position themselves as modern and technologically nimble.
Back in January, Keir Starmer announced that “artificial intelligence will deliver a decade of national renewal” promising a plan that would “mainline AI into the veins of this enterprising nation”.
An interesting choice of metaphor, you might think, for a government which generally takes a dim view of the intravenous consumption of stupefying substances.
Conciousness
DESCRIBING these failures as “hallucinations” is not uncontested. Some folk think the language of hallucinations is too anthropomorphic, attributing features of human cognition and human consciousness to a predictive language process which we all need reminding doesn’t actually reason or feel.
The problem here isn’t seeing fairies at the bottom of the garden, but faced with an unknown answer, making up facts to fill the void. One of the definitions of these systems failures I like best is “a tendency to invent facts in moments of uncertainty”.
This is why some argue “bullshitting” much better captures what generative AI is actually doing. A liar knowingly tells you something that isn’t true. A bullshitter, by contrast, preserves the illusion of themselves as knowing and wise person by peddling whatever factoids they feel they need to get them through a potentially awkward encounter – reckless or indifferent to whether or not what they’ve said is true.
Generative AI is a bullsh***er. The knowledge it generates is meretricious. When using it, the mantra should not be “trust but verify” – but “mistrust and verify”. And given this healthy mistrust and timeconsuming need for verification, you might wonder how much of a time-saver this unreliable Chatbot can really be.
Higher education is still reeling from the impact. Up and down the country this month, lecturers have been grading papers, working their way through exam scripts and sitting in assessments boards, tracking our students many achievements, but also contending with the impact of this wave of bullshit, as lazy, lost or desperate students decide to resort to generative AI to try to stumble through their assessments.
Problematic?
IF you think the function of education is achieving extrinsic goals – getting the essay submitted, securing a grade, winning the degree – then I guess AI assisted progress to that end won’t strike you as problematic.
One of the profound pleasures of work in higher education is watching the evolution of your students. When many 18-year-olds arrive in law school for the first time, they almost always take a while to find their feet. The standards are different.
The grading curve is sharper. We unaccountably teach young people almost nothing about law in Scottish schools, and new students’ first encounter with the reality of legal reading, legal argument and legal sources often causes a bit of a shock to the system.
But over four years, the development you see is often remarkable, with final year students producing work which they could never have imagined was in them, just a few teaching terms earlier. And that, for me, is the fundamental point. The work is in the student’s.
Yes, it requires a critical synthesis with the world, engagement with other people’s ideas, a breadth of reading and references – but strong students pull the
I worry about the impact hallucinations will have on people forced to represent themselves in legal actions
project out of their own guts.
They can look at the final text and think, with significant and well-earned satisfaction – I made that. Now I know I’m capable of digesting a debate, marshalling an argument, presenting a mess of facts in a coherent and wellstructured way – by myself, for myself. Education has changed me. It has allowed me to do things I couldn’t imagine doing before.
Folk turning in the AI generated dissertations or essays, undetected, can only enjoy the satisfactions of time saved, getting away with it, and the anxious future knowing that given the opportunity to honestly test themselves and show what they had in them, they decided instead to cheat.
Students rumbled
AT university, being rumbled for reliance on AI normally results in a zero mark and a resit assessment, but the real-world impacts of these hallucinations are now accumulating in ways that should focus the mind, particularly in the legal sector.
In London last week, the Court of Appeal handed down a stinging contempt of court judgment involving two cases of lawyers rumbled after citing bogus case law in separate court actions.
The lawyers in question join hundreds of others from jurisdictions across the world, who’ve found their professional reputations shredded by being caught by the court after relying on hallucinated legal sources.
We aren’t talking about nickel and dime litigation either here. One of the
‘AI could be homo sapiens’ very last invention’
two cases was a £89 million damages claim against the Qatar National Bank.
The court found that the claimants cited 45 cases, 18 of which turned out to be invented, while quotations which had been relied on in their briefs were also phoney.
The second case involved a very junior barrister who presented a judicial review petition, relying on a series of legal authorities which had the misfortune not to exist.
As Dame Victoria Sharp points out, there are “serious implications for the administration of justice and public confidence in the justice system if artificial intelligence is misused” in this way, precisely because of its ability to produce “apparently coherent and plausible responses” which prove “entirely incorrect,” make “confident assertions that are simply untrue,” “cite sources that do not exist,” and “purport to quote passages from a genuine source that do not appear in that source.”
The Court of Appeal concluded that “freely available generative artificial intelligence tools, trained on a large language model such as CHATGPT are not capable of conducting reliable legal research”. I agree.
For legal professionals to be presenting cases in this way is indefensible, with serious implications for professional standards integrity, for courts relying on the legal argument put before them, and for clients who suffer the consequences of their case being presented using duff statements of the law or duff sources.
Legal Aid crisis
I WORRY too about the potentially bigger impact these hallucinations will have on people forced to represent themselves in legal actions. Legal aid remains in crisis in this country.
Many people who want to have the benefit of legal advice and representation find they cannot access it, particularly in civil matters. The saying goes that “a man who represents himself in court has a fool for a client.” In modern Britain, a person who represents themselves in court normally has the only lawyer they can afford, as foolish and unfair as this might be.
Acting as a party litigant is no easy task. Legal procedures are often arcane and unfamiliar. Legal institutions can be intimidating.
If the other side has the benefit of a solicitor or advocate, there’s a real inequality of arms. But even before you step near a Sheriff Court, you need to have some understanding of the legal principles applying to your case to state it clearly. Misunderstand and mispresent the law, and you can easily lose a winnable case.
In Scotland in particular, significant parts of our law isn’t publicly accessible or codified.
This means ordinary people often can’t find reliable and accessible online sources on what the law is – but it also means that LLMS like CHATGPT also haven’t been able to crawl over these sources to inform the automated answers they spit out.
This means that these large language models are much more likely to give questioning Scots answers based on English or sometimes even American law than the actual rules and principles a litigant in person needs to know to persuade the Sheriff that they have a good case. Hallucination rates are high. Justice will suffer.