Courts on AI / The court's own use of AI
United States v. Deleon
U.S. Court of Appeals for the Eleventh Circuit · Federal courts of appeals · · 116 F.4th 1260 · No. 23-10478 · concurrence
On 2024-09-05, the Eleventh Circuit affirms a robbery sentencing enhancement. Judge Newsom concurs to revisit ordinary meaning and explains that he queried ChatGPT, Gemini, and Claude repeatedly about “physically restrained,” using the variation in responses to discuss LLMs and everyday speech patterns.
- Disposition
- The court affirmed the sentencing enhancement; Judge Newsom concurred, reporting a multi-model LLM experiment on the phrase “physically restrained.”
What the court wrote
Verbatim from the court's document (public domain). Ellipses mark omitted text; nothing is paraphrased.
This is a sequel of sorts to my separate opinion in Snell v. United Specialty Insurance Co., 102 F.4th 1208 (11th Cir. 2024) (Newsom, J., concurring). There, I floated the following suggestion, which I suspected (rightly, as it turns out) would be fairly provocative: “Those, like me, who believe that ‘ordinary meaning’ is the foundational rule for the evaluation of legal texts should consider—consider—whether and how AI-powered large language models like OpenAI’s ChatGPT, Google’s Gemini, and Anthropic’s Claude might—might—inform the interpretive analysis.” Id. at 1221. With the benefit of a little perspective, and incorporating by reference here all the caveats that I expressed there, I stand by what I said. The LLM research that I conducted in connection with this case, though, raised a question that I hadn’t squarely confronted in Snell, one that seemed worth flagging and exploring: What should we make of the fact that the models sometimes provide subtly different answers to the exact same question? ... That seems about right to me. But because there’s no ready dictionary definition of the composite phrase, because by their very nature LLMs aim to capture and reflect how real people ordinarily use both individual words and multi-word phrases, and because, well, I couldn’t help myself, I queried ChatGPT.
Authorities cited in the passage: Snell v. United Specialty Insurance Co., 102 F.4th 1208 (11th Cir. 2024)