AI Hallucination Stories
83 disasters tagged #ai-hallucination
Both sides used AI in Withers v. City of Aberdeen, so the judge kicked every lawyer off the case
On June 8, 2026, U.S. District Judge Sharion Aycock sanctioned every lawyer of record in Withers v. City of Aberdeen after filings from both sides contained hallucinated legal citations. Two out-of-state lawyers admitted using AI without verifying the output. Two local lawyers said they did not know about that AI use, but admitted they signed or allowed filings without checking the citations. The court cancelled the scheduled trial, revoked two pro hac vice admissions, barred those lawyers from appearing in the district for two years, disqualified the local lawyers from the case, imposed fines, and sent the order to state bar authorities. An entire case got stopped because both sides treated cite-checking like optional garnish.
Oregon Supreme Court struck filings after self-represented litigants used AI-made fake law
On June 4, 2026, the Oregon Supreme Court issued two orders addressing fabricated legal authorities in self-represented filings. In one matter, the court struck a mandamus petition and dismissed the proceeding after the relators acknowledged relying on a generative AI service called LegalAI, then filed another declaration less than 12 hours after a show-cause order that cited more nonexistent Oregon cases. In a second matter, the court struck a response to a petition for review, imposed a $500 sanction, and required any amended filing to certify that every cited, quoted, or paraphrased source of law exists. Oregon's high court had to become a citation disinfectant station.
Ninth Circuit suspends attorneys over AI-hallucinated immigration briefs
The Ninth Circuit sanctioned Orange County immigration attorneys Mike Singh Sethi and William Rounds after briefs in LNU v. Blanche contained nonexistent cases, invented quotations, and grossly inaccurate descriptions of real law. The court said the problem was not that generative AI had been used; the problem was that fabricated authority reached a federal appeals court and the lawyers tried to explain it away as innocent typing errors. Each attorney was fined $2,500, both were suspended from Ninth Circuit practice for six months, and their firm must disclose generative AI use in future filings for two years.
NewsBench says major chatbots failed election answers on facts, sourcing, or neutrality 90% of the time
On May 21, 2026, Forum AI launched NewsBench, an expert-grounded benchmark for how major AI chatbots handle news and current-events questions. The first wave tested ChatGPT, Gemini, Claude, and Grok across 3,136 prompts and 12,542 expert-judged responses. Election answers failed Forum's accuracy, neutrality, or source-quality checks 90% of the time, while nearly 36% of election answers contained at least one factual error. The machines did not merely fumble trivia; they produced civic information with clean formatting and a suspicious amount of confidence. Democracy, meet the autocomplete intern who cites state media.
Demos found AI chatbots mangled Scottish election facts in one-third of answers
On May 20, 2026, Demos published Electoral Hallucinations, a study of five text-based AI services during the Scottish Parliament election window. The researchers tested ChatGPT, Google Gemini, Google AI Overviews, Grok, and Replika on March 27 using questions about three real Holyrood constituencies. Across factual responses, 34.1% contained errors: 8.75% were entirely inaccurate and 25.3% were partly accurate but wrong in material ways. The systems gave bad voter-ID advice, invented candidates, made up scandals, misidentified constituencies, got registration deadlines wrong, and even missed the election date by more than two months. Democracy, now with autocomplete and the usual warranty.
Book about AI and truth shipped with fake AI-generated quotes
In May 2026, Steven Rosenbaum's The Future of Truth became the wrong kind of case study when The New York Times, The Daily Beast, The Atlantic, and Ars Technica reported that the book contained multiple fake or misattributed quotes. Rosenbaum acknowledged using ChatGPT and Claude during research, writing, and editing, and accepted responsibility for what he called improperly attributed or synthetic quotes. Reporters found a fabricated quote attributed to Kara Swisher, misattributed material connected to Lisa Feldman Barrett, and a Meredith Broussard quote placed in the wrong source. Ars reported that six outside citations had been identified as problematic. A book warning about synthetic truth managed to demonstrate the footgun in hardcover.
UK government's GOV.UK Chat launched with misleading tax answers on day one
On Friday, May 15, 2026, the UK government rolled out GOV.UK Chat inside the official GOV.UK app, billing it as the largest government-built chatbot of its kind, trained on 80,000 pages of gov.uk content with a target accuracy of 90%. Within hours of launch, tax expert Dan Neidle of Tax Policy Associates published evidence in The Times showing the bot giving misleading answers on tax questions that millions of UK households actually have. The bot failed to mention the ÂŁ100,000 cliff edge where tax-free childcare eligibility collapses, and it told a user that selling old MacBooks on eBay could attract capital gains tax, which is not how UK CGT works for personal-use chattels. The Cabinet Office framed the tool as "information about services" rather than advice; Neidle pointed out the bot itself reads like it is giving advice. Either way, a 90% accuracy claim on benefits and tax means one in ten answers is wrong on questions where being wrong costs real money.
EY Canada pulled a cyber report after researchers found fake citations
On May 14, 2026, GPTZero published an investigation into EY Canada's loyalty-fraud cybersecurity report, Points of Attack, and said the 44-page document was loaded with hallucinated references, broken or fake source URLs, misattributed statistics, and text that scanned as AI-written. EY Canada then removed the report from its website and said it was reviewing how it was published. For a firm selling trust, controls, and responsible AI advice, having a public report fall over at the bibliography is a rough little invoice from reality.
German court says chatbot's fake medical titles are the company's problem
The Higher Regional Court of Hamm held Aesthetify GmbH liable for false statements made by its website chatbot after the bot claimed the company's physician-directors held specialist medical titles they did not have, including titles that do not exist under German medical qualification rules. The company argued the AI system worked autonomously and had been trained only on correct data. The court rejected that defense, treated the chatbot's statements as the company's own commercial conduct, and ordered an injunction, costs, and reimbursement of warning-letter expenses.
Ontario's approved AI scribes fabricated medical notes in audit testing
On May 12, 2026, Ontario's Auditor General released a special report finding that all 20 approved AI scribe vendors showed inaccuracies during procurement testing. Nine systems fabricated treatment-plan suggestions that were never discussed, 12 captured a different drug than the doctor prescribed, and 17 missed mental-health details from simulated patient encounters. The audit did not document known patient harm, but it did show the province had approved clinical note-taking tools with failures that would be spectacularly unwelcome in an actual chart.
AI-made citations are polluting published research by the thousand
A January 2026 conference-paper analysis, an April Nature investigation, and a May 2026 Lancet biomedical audit all point to the same ugly conclusion: AI-hallucinated references are no longer isolated embarrassments. GhostCite found a sharp jump in unverifiable citations in 2025 computer-science conference papers. Nature estimated that tens of thousands of 2025 publications may contain invalid AI-generated references. The Lancet audit then found 4,046 fabricated references across 2,810 PubMed Central papers. The problem is no longer just that chatbots invent papers. It is that those inventions are surviving long enough to contaminate the literature and force publishers into cleanup work they clearly did not plan for.
Georgia Supreme Court made a murder appeal redo after AI citations infected the order
On May 5, 2026, the Supreme Court of Georgia vacated a trial-court order in Hannah Payne's murder appeal because the State's filings and the order denying a new-trial motion contained nonexistent, unsupported, and misattributed case citations generated with artificial intelligence. Assistant District Attorney Deborah Leslie acknowledged using AI software and not independently verifying the citations. The court admonished Leslie and the Clayton County District Attorney's Office, suspended Leslie from practicing before the Georgia Supreme Court for six months, required extra training before reinstatement, and sent the case back for a new order that counsel for neither side may draft.
AI chatbots gave misleading advice before the Senedd election
BBC Wales tested major chatbots before the May 7, 2026 Senedd election and found they could give voters inaccurate candidate and constituency information. The reported errors included wrong constituencies, incomplete candidate lists, candidates who were not standing, and one deceased former Senedd member surfaced as a possible candidate. The incident is not evidence that the election result changed. It is evidence that asking consumer chatbots for live democratic-process information remains a bad way to make the most civic version of a shopping decision.
Google AI Overview allegedly branded a fiddler as a sex offender
Canadian musician Ashley MacIsaac sued Google after its AI Overview allegedly confused him with another person, falsely described him as a convicted sex offender, and helped get a December 2025 concert canceled. Google later changed the result, but the lawsuit says the damage was already done: reputational harm, lost work, safety fears, and a $1.5 million defamation claim over a machine-generated biography that apparently could not manage the demanding research task of checking which Ashley MacIsaac it was talking about.
Alabama Supreme Court tossed an entire appeal over AI-hallucinated citations
In April 2026, the Alabama Supreme Court did something rare: it threw out an appeal entirely because the lawyer's briefs were stuffed with invented case law. Mobile solo practitioner W. Perry Hall represented the losing side of a trust dispute and filed briefs that the justices called "grossly deficient" and full of an "astounding number" of invalid, inaccurate, and irrelevant citations. The court ordered Hall to pay $17,200 in attorneys' fees and costs, referred him to the Alabama State Bar for possible discipline, and barred him from any further filings before that court unless a separate attorney in good standing co-signs. The capper sits in a footnote: in the same paragraph where Hall apologized for AI hallucinations and promised the mistake would not recur, he cited two more cases that do not exist.
Webb Law Group partner sanctioned for not supervising AI-cited brief
A federal magistrate judge in the Northern District of California sanctioned attorney Lenden Webb after a brief filed by lawyers at Webb Law Group included a fake citation caused in part by AI use and lack of supervision. The April 28, 2026 order required Webb to circulate court materials inside the firm, complete live CLE on supervision and ethical AI use, distribute the course materials to staff, and personally pay $1,001.
South Africa withdrew its draft AI policy after finding fictitious sources in the references
South Africa's Department of Communications and Digital Technologies withdrew its Draft National Artificial Intelligence Policy after officials confirmed the reference list contained fictitious sources. Communications Minister Solly Malatsi said the most plausible explanation was unverified AI-generated citations and called the lapse serious enough to compromise the draft's integrity and credibility. This is vibe-lawyering wearing a government badge: an official policy about regulating AI tripped over the exact hallucination problem that every first-year ChatGPT cautionary slide already warned about.
Judge fined Raja Rajan for AI-made citations (AGAIN 🤦‍♂️)
Judge Kai N. Scott sanctioned defense lawyer Raja Rajan $5,000 on April 20, 2026 after finding that he had again filed AI-generated fake citations in Bunce v. Visual Technology Innovations. Rajan had already been fined $2,500 and ordered to complete AI and legal ethics CLE in the same litigation the year before. This time the judge said she remained appalled by the conduct, ordered more CLE, and warned that a third incident could trigger referral to the Pennsylvania Disciplinary Board. The notable part is not that AI got something wrong. It is that a lawyer, after already being punished for the exact same mistake, did it again.
Researchers invented a fake disease and major chatbots promoted it anyway
Researchers created a fake eye condition called bixonimania, uploaded fake papers full of obvious tells, and then watched major chatbots treat it as a real diagnosis. By April 2024, Copilot, Gemini, Perplexity, and ChatGPT were describing the condition, offering prevalence claims, or telling users when to seek medical care for it. The hoax later leaked into a real journal paper before retraction. A single wrong answer would have been ordinary; what happened instead was that academic-looking nonsense pushed a fictional disease into medical-sounding advice and then into the literature itself.
AI summaries sent Overland Park Farmers Market shoppers to a construction site
On April 18, 2026, more than 100 people reportedly went to the construction site for Overland Park Farmers Market's future home instead of the temporary market location. The market and city said incorrect AI search results and summaries on Google and Instagram confused visitors during a year when the market was operating from Matt Ross Community Center before moving to Clock Tower Landing in June. City communications staff said they received messages from confused customers, reached out to Meta, and had to remind people to use official city and market pages. The tomatoes were two blocks away; the chatbot sent people to fencing.
Sullivan & Cromwell apologized after AI put fake cites in bankruptcy court
In April 2026, Sullivan & Cromwell told a Manhattan bankruptcy judge that an emergency motion it filed in the Prince Global Holdings Chapter 15 case contained AI hallucinations, inaccurate citations, and other errors. Opposing counsel at Boies Schiller Flexner caught the problems first. Andrew Dietderich, co-head of the firm's restructuring practice, apologized in a letter dated April 18, said the firm's AI policies had not been followed, and acknowledged that a secondary review also failed to catch the bogus material. The corrected filing avoided an immediate sanctions story, but it still turned one of Wall Street's prestige firms into the latest exhibit in why AI-assisted legal drafting and vibes-based review are a bad mix.
BMJ Open audit finds half of AI health chatbot answers problematic under stress testing
A UCLA-led team published a BMJ Open audit of five major consumer chatbots (ChatGPT, Gemini, Grok, Meta AI, DeepSeek) on 250 adversarial health prompts across cancer, vaccines, stem cells, nutrition, and athletic performance. Experts rated 49.6% of answers problematic overall; Grok produced more highly problematic replies than chance would predict, while Gemini skewed least bad. Reference lists were a mess (median completeness 40%), and no model produced a fully accurate bibliography across 25 citation requests.
The New York Times printed an AI-generated "quote" that Pierre Poilievre never said
On April 14, 2026, the New York Times published a Canadian-election analysis piece by its Canada bureau chief that included a direct quotation attributed to Conservative Party leader Pierre Poilievre. He never said it. The wording turned out to be an AI-generated summary of his views that the AI tool had formatted as a quotation, and it sailed through whatever editing process the Times had in place. A Bluesky reader flagged the error the next day. The correction did not run until May 1, more than two weeks later. Days after the incident drew wider attention, the Times rolled out new guidance restricting AI use, but only for freelancers; the staff reporter who filed the original piece was not the target audience for the new rule.
Study finds Google's AI Overviews wrong millions of times per hour
The New York Times commissioned AI startup Oumi to test the factual accuracy of Google's AI Overviews across 8,652 searches using OpenAI's SimpleQA benchmark. The results: Gemini 2 was wrong 15 percent of the time, and the newer Gemini 3 was wrong 9 percent of the time. Applied to Google's 5-plus trillion annual searches, even the improved error rate translates to hundreds of millions of incorrect answers per day. Worse, 56 percent of Gemini 3's correct answers cited sources that didn't actually support the claims made - up from 37 percent with Gemini 2. Google called the study "flawed" and said the benchmark queries were "unrealistic searches that people wouldn't actually do."
Oregon estate case imploded after AI-made citations brought six-figure penalties
In Couvrette v. Wisnovsky, an Oregon federal estate dispute turned into one of the harshest AI-lawyering cases yet. Across three summary-judgment briefs, plaintiffs' counsel used 15 fake case citations and eight fabricated quotations. Magistrate Judge Mark Clarke sanctioned the lawyers in December 2025, split a $94,704.38 fee award between lead and local counsel on March 23, 2026, and dismissed the case with prejudice a week later. The filing error was bad enough. What made this one worse was the court's view that the problems were flagged, not meaningfully fixed, and left to rot until the court stepped in.
Third Circuit reprimanded a lawyer over AI-hallucinated DEA authorities
On March 27, 2026, the Third Circuit issued a precedential opinion reprimanding attorney Daniel A. Pallen after an appellate brief in McCarthy v. DEA used AI-generated summaries of DEA adjudications that were inaccurate or nonexistent. The court declined monetary sanctions, partly because it was its first precedential AI-misuse opinion, but it directed notice to other courts and the National Disciplinary Data Bank. That is a permanent paper trail for a brief that should have been checked before filing.
Mediahuis suspended senior journalist over AI-invented quotes
Mediahuis suspended veteran journalist Peter Vandermeersch after reporting found AI-generated quotes in his work. Euronews reported that 15 of 53 articles included fabricated expert quotes, with multiple quoted individuals saying they had not made the attributed remarks. Vandermeersch acknowledged relying on tools such as ChatGPT, Perplexity, and Google's Notebook tools to summarize source material, then trusting the outputs too much.
Oregon attorney hit with record $10K fine after AI fabricated 15 citations and 9 fake quotes
Salem attorney Bill Ghiorso was fined $10,000 by the Oregon Court of Appeals after submitting an opening brief in Doiban v. Oregon Liquor and Cannabis Commission that contained at least 15 fabricated case citations and nine nonexistent legal quotations - all generated by an AI search tool used by his staff. The fine is the largest ever imposed in Oregon for AI-related errors in legal filings, calculated under a penalty structure the court established in December 2025: $500 per fake citation, $1,000 per fake quote. The intended total of $16,500 was capped at $10,000 due to Ghiorso's medical issues. Perhaps the most instructive detail: when Ghiorso's staff asked the AI tool whether its own fabricated citations were real, it helpfully confirmed they were.
Sixth Circuit hits two lawyers with $30K in sanctions for 24+ fabricated citations
The Sixth U.S. Circuit Court of Appeals sanctioned attorneys Van R. Irion and Russ Egli $15,000 each in punitive fines - totaling $30,000 - after their briefs in Whiting v. City of Athens, Tennessee contained more than two dozen fabricated or seriously misrepresented citations. The panel also ordered them jointly liable for the appellees' full attorney fees on appeal and double costs. The court didn't explicitly pin the fabrications on generative AI, but emphasized that lawyers must personally read and verify every citation "regardless of how they were generated" - which is a very specific way to phrase a very pointed implication.
Ontario lawyer referred to law society after factum contained seven invented quotations
Ontario lawyer Khalid Parvaiz was referred to the Law Society of Ontario by Justice Frederick Myers after filing a factum containing seven "wholly made up" quotations attributed to real court cases. Parvaiz claimed the fabricated passages were "human errors" from "misreading of the cases" and denied using AI. Justice Myers was unconvinced, noting the alleged quotations were "completely made up" rather than paraphrased or miscited, and warned that the cover-up - if Parvaiz was being untruthful about the source - could carry more severe consequences than the original error.
DOJ prosecutor resigned after filing an AI-generated brief full of fabricated citations
Rudy Renfer, an assistant U.S. attorney in the Eastern District of North Carolina, resigned in March 2026 after admitting he used AI to rewrite a legal brief that contained fabricated citations, fictitious quotations, and misstatements of law. The opposing party - a pro se retired Air Force colonel suing over GLP-1 medication coverage under TRICARE - caught the fakes. At a show-cause hearing, the presiding magistrate judge expressed skepticism about Renfer's claim that he had reviewed the brief before filing, noting the fabrications appeared "intentionally designed" to support the government's argument. The matter was referred to the DOJ's Office of Professional Responsibility, and the district's U.S. Attorney issued an office-wide memo warning staff that "AI may hallucinate, but that does not excuse you from your obligations."
ChatGPT convinced Illinois woman to fire her lawyer and file 60+ bogus court documents
Nippon Life Insurance Company sued OpenAI after ChatGPT allegedly acted as a de facto lawyer for Graciela Dela Torre, an Illinois disability claimant who had already settled her case. When her real attorney told her the settlement couldn't be reopened, she asked ChatGPT if she'd been "gaslighted." The chatbot told her to fire her lawyer, helped her draft over 60 pro se filings across two federal cases, and produced fabricated case citations including an entirely invented case called "Carr v." something. Nippon is suing OpenAI for unauthorized practice of law under Illinois state law, arguing it spent huge amounts of time and money dealing with AI-generated litigation that should never have existed.
India's Supreme Court calls AI-hallucinated citations in trial court order "misconduct"
India's Supreme Court stayed a property-dispute ruling after discovering the trial court judge had relied on non-existent, AI-generated case citations. An Andhra Pradesh junior civil judge admitted using an AI tool for the first time without verifying the outputs. The Supreme Court termed the reliance on fabricated judgments as "misconduct" with "a direct bearing on the integrity of the adjudicatory process." Separately, the Bombay High Court fined a litigant 50,000 rupees for filing AI-generated submissions citing the non-existent case "Jyoti vs. Elegant Associates." The Chief Justice flagged an "alarming trend" of AI-fabricated judgments including one titled "Mercy vs Mankind."
Study finds ChatGPT Health fails to flag over half of medical emergencies
The first independent safety evaluation of OpenAI's ChatGPT Health feature, published in Nature Medicine, found the tool failed to direct users to emergency care in 51.6% of cases requiring immediate hospitalization - instead recommending they stay home or book a routine appointment. The study also found ChatGPT Health frequently failed to detect suicidal ideation, with suicide crisis alerts sometimes triggering in lower-risk scenarios while failing to appear when users described specific plans for self-harm. Over 40 million people reportedly ask ChatGPT for health-related advice every day.
Government contractor sanctioned for AI-fabricated deposition testimony
The Civilian Board of Contract Appeals sanctioned a party in Louis J. Blazy v. Department of State (CBCA 7992) after discovering four non-existent legal decisions and four fabricated deposition excerpts in filings. The supposed direct quotations from witness testimony didn't appear on the cited transcript pages. When pressed, Blazy admitted the quotes were "constructed" and offered substitute testimony that didn't support the original wording. He also misrepresented existing case law by submitting real decisions as stand-ins for the fake ones, characterizing them as supporting principles they did not contain. The CBCA issued a formal admonishment and warned that continued misconduct could result in dismissal - making this one of the first federal sanctions involving AI-fabricated witness testimony rather than made-up case law alone.
Fifth Circuit sanctions lawyer $2,500 for AI-hallucinated citations, says problem "getting worse"
The U.S. Court of Appeals for the Fifth Circuit sanctioned attorney Heather Hersh $2,500 after finding her brief contained 16 fabricated quotations and five additional serious misrepresentations of law or fact, all apparently AI-generated. The court expressed frustration that AI-hallucinated legal citations "have increasingly become an even greater problem in our courts" and that the issue "shows no sign of abating." Hersh initially denied using AI, then shifted to claiming she "relied on publicly available versions of the cases, which she believed were accurate."
Ars Technica fires senior AI reporter after AI tool fabricated quotes in published story
Ars Technica retracted an article by senior AI reporter Benj Edwards after it contained fabricated quotations generated by an AI tool and attributed to a source who never said them. The publication acknowledged the incident as a "serious failure of our standards" and Edwards was subsequently fired. Edwards noted the irony on Bluesky: "The irony of an AI reporter being tripped up by AI hallucination is not lost on me."
Wisconsin DA sanctioned for AI-hallucinated legal citations in burglary case
Kenosha County District Attorney Xavier Solis was sanctioned by Circuit Court Judge David Hughes after his office submitted court filings containing AI-generated legal citations that did not exist. The filings were part of a burglary case against two defendants, and Solis failed to disclose his use of AI - violating Kenosha County's court policy requiring disclosure and verification of AI-generated content. The charges were ultimately dismissed (primarily for lack of probable cause), but not before the bogus citations made the DA's office a warning for prosecutors nationwide. Solis acknowledged the error and promised to "review and reinforce internal practices." It's always reassuring when the person responsible for prosecuting crimes can't be bothered to read the citations in their own filings.
10th Circuit sanctions lawyer $1,000 for ChatGPT-fabricated appellate brief
Maryland attorney Kusmin Amarsingh used ChatGPT to draft her appellate brief against Frontier Airlines without verifying any citations, resulting in multiple nonexistent cases being cited in the 10th Circuit. The court found her conduct "reckless" for completely failing to perform "an attorney's fundamental duty to the court." She was fined $1,000 and referred to Maryland attorney-disciplinary authorities.
Study finds AI chatbots no better than search engines for medical advice
A randomized controlled trial published in Nature Medicine with 1,298 UK participants found that AI chatbot users (GPT-4o, Llama 3, Command R+) performed no better than the control group at assessing clinical urgency and worse at identifying relevant medical conditions. In one case, two users with identical subarachnoid hemorrhage symptoms received opposite recommendations -- one told to lie down in a dark room, the other correctly advised to seek emergency care.
Repeated AI-fabricated citations cost client the entire case
Attorney Steven Feldman filed multiple motions containing AI-fabricated case citations in Flycatcher Corp. v. Affable Avenue LLC. Despite explicit court warnings and access to Westlaw and Lexis, he continued submitting unverified AI output -- even using AI to draft his response to the court's show-cause order, which contained yet more fake citations. Judge Failla imposed the most severe AI-hallucination sanction yet: default judgment against his client.
Four attorneys fined $12,000 combined for AI-fabricated patent case citations
A federal judge in the District of Kansas fined four attorneys a combined $12,000 for court filings containing AI-generated fabricated legal citations in a patent infringement case. The attorney who used ChatGPT received $5,000; two who failed to review the filings received $3,000 each; local counsel who did not identify errors received $1,000. The judge called the volume of fabricated case law "staggering."
ECRI names AI chatbot misuse as top health technology hazard for 2026
Nonprofit patient safety organization ECRI ranked misuse of AI chatbots as the number one health technology hazard for 2026. ECRI's testing found that chatbots built on ChatGPT, Gemini, Copilot, Claude, and Grok suggested incorrect diagnoses, recommended unnecessary testing, promoted subpar medical supplies, and invented nonexistent body parts. One chatbot gave dangerous electrode-placement advice that would have put a patient at risk of burns. OpenAI reported that over 5 percent of all ChatGPT messages are healthcare related, with 200 million users asking health questions weekly, despite the tools not being validated or approved for healthcare use.
Two lawyers sanctioned differently for same filing with AI-fabricated citations
Attorneys Yen-Yi Anderson and Jeffrey Goldin jointly filed a motion in Lifetime Well v. IBSpot containing at least eight AI-generated false citations. Judge Kearney imposed differential sanctions based on their responses: Anderson, who blamed time pressure and fired her law clerk rather than accepting responsibility, received $4,000 in monetary sanctions. Goldin, who promptly accepted responsibility and implemented remedial measures, received no monetary penalty.
New York court sanctions lawyer for AI-fabricated case law
A New York appellate court imposed $10,000 in sanctions after a lawyer submitted briefings in a mortgage foreclosure case containing fabricated case citations identified as likely AI-generated hallucinations. The court found multiple nonexistent cases and misrepresented holdings, affirming prior orders and awarding costs to the plaintiff.
Five Kansas attorneys face sanctions for ChatGPT-fabricated court citations
Five attorneys who signed a legal brief for Lexos Media IP LLC in a patent infringement case against Overstock.com submitted fabricated case citations hallucinated by ChatGPT to a federal court in Kansas. Senior U.S. District Judge Julie Robinson issued an order requiring them to explain why they should not be sanctioned, with multiple defects attributed to AI including nonexistent lawsuits, made-up judicial quotes, and citations to real cases that held the opposite of what the brief claimed.
Guardian investigation finds Google AI Overviews gave dangerous health misinformation
A Guardian investigation found Google's AI Overviews displayed false and misleading health information across multiple medical topics. AI summaries gave incorrect liver function test ranges sourced from an Indian hospital chain without accounting for nationality, sex, or age. The feature advised pancreatic cancer patients to avoid high-fat foods, which experts said could increase mortality risk. Stanford and MIT researchers called the absence of prominent disclaimers a critical danger. Google removed some AI Overviews for health queries after the investigation, but many remained active.
AI police report claims officer shape-shifted into a frog
Heber City Police Department's Axon Draft One AI report tool transcribed background dialogue from The Princess and the Frog playing on a television into an official police report, claiming an officer had shape-shifted into a frog while conducting police activity. The incident exposed design flaws in AI report-writing tools that process all body camera audio without distinguishing between relevant police interactions and ambient background noise.
Amazon pulled Prime Video's AI recaps after Fallout errors
Amazon launched Prime Video "Video Recaps" as a beta generative-AI feature meant to help viewers catch up between seasons. A recap for Fallout instead got basic plot points wrong, including mislabeling one of The Ghoul's flashbacks as "1950s America" rather than 2077 and misdescribing a key scene with Lucy. Prime Video then pulled the recap feature from the shows in the test program, which is not ideal for a tool whose entire job is remembering the plot.
Washington Post launched AI podcast that failed its own quality tests at an 84% rate
The Washington Post launched "Your Personal Podcast," an AI-generated audio news product, in December 2025 despite internal testing showing that between 68% and 84% of AI-generated scripts failed to meet the publication's editorial standards across three rounds of evaluation. The AI fabricated quotes from public figures, misattributed statements, mispronounced names, and inserted its own editorial commentary as if it were the Post's position. The internal review concluded that "further small prompt changes are unlikely to meaningfully improve outcomes without introducing more risk." The product team recommended launching anyway. Post editors revolted, with one writing in Slack that it was "truly astonishing that this was allowed to go forward at all."
Deloitte gets caught using AI hallucinations in a government report - again
Seven weeks after Deloitte Australia agreed to partially refund a government contract over AI-fabricated citations, a Newfoundland and Labrador journalist discovered that Deloitte Canada's $1.6 million healthcare workforce report contained at least four fabricated academic citations from papers that don't exist. The fake references named real researchers as co-authors of fictional studies - researchers who confirmed they never wrote the cited work. Deloitte admitted AI was "selectively used to support a small number of research citations," stood by the report's findings, and offered no refund. The province's accounting watchdog launched a formal investigation, and Newfoundland became one of the first Canadian provinces to require AI disclosure in government contracts.
AI-hallucinated citations delay wage class action settlement in N.D. Cal
A federal judge in the Northern District of California sanctioned plaintiff's counsel James Dal Bon in Buchanan v. Vuori Inc. (Case 5:23-cv-01121-NC) for filing AI-generated case law citations in a motion for preliminary approval of a wage and hour class action settlement. Dal Bon used six different AI tools to prepare the memorandum, which contained hallucinated quotes and a nonexistent case citation. After the court flagged the fabricated citations, his corrected filing still contained AI-hallucinated case law. The sanctions delayed the class action settlement, ultimately converting it to an individual settlement that abandoned the class members the attorney was supposed to represent.
AI-only support is bleeding customers before it saves money
Acquire BPO’s 2024 AI in Customer Service survey found 70% of U.S. consumers would bolt to a rival after just one bad chatbot interaction and 72% only buy when a live agent safety net exists, even as CMSWire reports enterprises poured $47 billion into AI projects in early 2025 that delivered almost no return. CX strategists now warn executives that Air Canada–style hallucinations, mounting legal liability, and empathy gaps make AI-only helpdesks a churn machine unless human agents stay in the loop.
BBC/EBU study says AI news summaries fail ~half the time
A BBC audit of 2,700 news questions asked in 14 languages found that Gemini, Copilot, ChatGPT, and Perplexity mangled 45% of the answers, usually by hallucinating facts or stripping out attribution. The consortium logged serious sourcing lapses in a third of responses, including 72% of Gemini replies, plus outdated or fabricated claims about public-policy news, reinforcing fears that AI assistants are siphoning audiences while distorting the journalism they quote.
Google’s Gemini allegedly slandered a Tennessee activist
Conservative organizer Robby Starbuck sued Google in Delaware, saying Gemini and Gemma kept spitting out fabricated claims that he was a child rapist, a shooter, and a Jan. 6 rioter even after two years of complaints and cease-and- desist letters. The $15 million suit argues Google knew its AI results were hallucinated, cited fake sources anyway, and let the libel spread to millions of voters.
Deloitte to refund Australian government after AI-generated report
Deloitte Australia agreed to partially refund a $440,000 contract after admitting its welfare compliance review for the Department of Employment and Workplace Relations contained fabricated academic citations and a fictitious judicial quote generated by Azure OpenAI GPT-4o. University of Sydney researcher Christopher Rudge found the revised report introduced even more hallucinated references than the original.
GAO dismisses 15 AI-hallucinated bid protests as abuse of process
The Government Accountability Office dismissed three consolidated protests filed by Oready, LLC - the culmination of 15 pro se bid protests filed over eight months, all riddled with non-existent citations, fabricated decisions, and hallmarks of unverified generative AI output. The GAO labeled Oready's pattern as "Gen-AI Misuse" and dismissed the protests as an abuse of the bid protest process, marking the GAO's first published dismissal for AI-driven abuse. Prior warnings issued in June and August 2025 were ignored. The fallout also prompted the GAO's January 2026 decision in Bramstedt Surgical to devote several pages to cautioning against AI-hallucinated citations, signaling that federal procurement tribunals are done issuing gentle reminders.
California lawyer fined $10,000 for ChatGPT-fabricated citations
Los Angeles attorney Amir Mostafavi became the first California lawyer sanctioned for AI-generated legal fabrications when a court hit him with a $10,000 fine. He ran his appeal draft through ChatGPT to improve the writing but did not verify the output before filing, unaware the tool had inserted fabricated case citations.
An AI-made freelancer fooled WIRED and Business Insider
In 2025, outlets including WIRED and Business Insider published articles under the byline Margaux Blanchard, a freelancer who appears not to exist. WIRED later published a postmortem admitting that one commissioned feature slipped past its usual defenses, including human review and even two commercial AI detectors, before editors discovered fabricated details and retracted it. Business Insider first removed Blanchard essays and then, after a broader internal probe, pulled at least 34 more pieces tied to dubious bylines and said it had strengthened verification protocols. The failure was not one chatbot going rogue. It was multiple newsroom workflows accepting AI-shaped fiction as publishable reporting.
Google AI invented fake specials for Stefanina's, and customers yelled at the restaurant
In August 2025, Stefanina's Wentzville, a family-owned Missouri restaurant, publicly warned customers not to use Google AI to find its specials after AI search results reportedly invented discounts, pricing, and menu information the restaurant did not offer. The restaurant said the false specials caused angry customers to yell at employees when staff refused to honor deals that existed only in Google's generated summary. Local reporting showed an AI Overview claiming a large pizza could be purchased for the price of a small one. Google did not respond to the station's questions, but its own guidance warned AI results may misunderstand information or make mistakes. The coupon fairy was apparently a hallucination engine.
Am Law 100 firm Gordon Rees caught twice filing AI-hallucinated citations
Gordon Rees Scully Mansukhani, one of the largest U.S. law firms, was caught filing AI-hallucinated case citations in an Alabama bankruptcy proceeding. An associate initially denied using AI under oath before the firm acknowledged the fabricated references and paid over $55,000 in sanctions and fees. Months later in February 2026, the same firm was reported to have filed a second brief containing hallucinated citations in a separate matter, making it the first Am Law 100 firm known to be a repeat offender.
ChatGPT diet advice caused bromism, psychosis, hospitalization
A Washington patient replaced table salt with sodium bromide after ChatGPT suggested bromide as a chloride substitute without distinguishing between chemical and dietary contexts. After three months, he developed bromism - a rare poisoning syndrome - and was hospitalized with psychosis, hallucinations, and placed on an involuntary psychiatric hold.
Butler Snow lawyers removed from Alabama prison case over fake ChatGPT citations
On July 23, 2025, U.S. District Judge Anna Manasco sanctioned three Butler Snow lawyers after filings in an Alabama prison case cited authorities that did not exist. The court found the lawyers had used ChatGPT for legal research, failed to verify the output, removed all three from the case, ordered broad disclosure of the sanctions order to clients and courts, and referred the matter to the Alabama State Bar. The sanction carried extra weight because the fake citations were attached to one of the firms Alabama pays to defend its prison system in high-stakes civil rights litigation.
Reporter fired after AI tool provided by her employer fabricated sources in front-page article
Wisconsin State Journal reporter Audrey Korte was fired in July 2025 after publishing a front-page article about a downtown Madison development plan that contained factual errors and fabricated sources generated by an AI tool. The tool had been provided by the newspaper's parent company, Lee Enterprises, and was installed on employee computers. Korte said she used it for grammar and style editing, but it introduced false information she didn't catch before publication. The article was pulled, replaced with a re-reported version, and stamped with a disclaimer citing "unauthorized AI use" and "fabricated sources." Korte was terminated. She publicly accepted responsibility for not catching the errors but noted she had received no training on the tool that was already installed on her work computer.
AI chatbots kept handing users fake or dead login URLs
Netcraft found in July 2025 that when users asked AI chatbots for official login pages for major brands, the answers were wrong about a third of the time. In tests covering 50 brands, 34% of the returned hostnames were not controlled by the brands at all: nearly 30% were unregistered, parked, or inactive, and another 5% pointed to unrelated businesses. In one Wells Fargo test, the model surfaced a fake page already tied to phishing. A chatbot that confidently invents login URLs is not a search engine with quirks. It is a phishing assistant with good manners.
Georgia appeals court fined a divorce lawyer after fake AI-like citations reached the order itself
In Shahid v. Esaam, decided June 30, 2025, the Georgia Court of Appeals vacated part of a divorce-related order after finding that several cited authorities did not exist and others did not support the propositions claimed. The panel concluded the briefing showed the hallmarks of generative AI hallucination, fined attorney Diana Lynch $2,500, and sent the matter back to the trial court. What made the case stand out ran deeper than a sloppy brief: the fake citations appeared to have made their way into the trial court's signed order.
AI-generated images and claims muddied Air India crash coverage
After Air India Flight 171 crashed in Ahmedabad on June 12, 2025, killing 275 people, AI-generated images of the crash spread across social media platforms. One widely shared synthetic image depicted the Boeing 787 broken in half across a building, but contained physically impossible details that experts identified as AI-generated. Fake victim photos, fabricated reports, and fraudulent fundraising campaigns followed. Google's AI Overview compounded the problem by incorrectly identifying the crashed aircraft as an Airbus rather than Boeing. Mashable reported the AI-generated content was convincing enough to confuse even aviation professionals.
UK High Court warns lawyers after fake AI citations infected two cases
On June 6, 2025, the High Court of England and Wales issued a joint ruling in two separate matters after lawyers put fake authorities before the court. In one case tied to Qatar National Bank, a filing cited 45 authorities, 18 of which did not exist, while many of the rest were misquoted or irrelevant. In the other, a housing claim against the London Borough of Haringey included five fabricated cases. The Divisional Court, led by Dame Victoria Sharp, said tools such as ChatGPT are not capable of reliable legal research, referred the lawyers involved to their regulators, and warned that more serious future misuse could lead to contempt proceedings or even police referral. The ruling turned individual AI citation blunders into a profession-wide warning.
White House MAHA report shipped fake studies and OpenAI citation markers
On May 29, 2025, NOTUS reported that the White House's Make America Healthy Again report cited studies that did not exist and mischaracterized others. PolitiFact, the Washington Post, and congressional oversight Democrats later pointed to classic AI-citation red flags, including fake paper titles, broken DOI links, and "oaicite" markers associated with OpenAI citation output. The White House called the problems formatting issues and updated the report. Public health policy apparently got the same bibliography QA as a panicked term paper, because history has a dark sense of humor.
Syndicated AI book list ran in major papers with made-up titles
A freelance writer working for King Features Syndicate used AI to research a summer reading list for the Chicago Sun-Times and Philadelphia Inquirer. Of the fifteen books recommended, only five were real. The rest were hallucinated titles attributed to real authors like Isabel Allende and Delia Owens. The list ran in print in a 64-page special section before 404 Media, NPR, and others exposed the fabrications. Both newspapers issued corrections and statements distancing their newsrooms from the syndicated content.
ChatGPT invented a child-murder conviction for a real man
When Norwegian user Arve Hjalmar Holmen asked ChatGPT who he was, the bot replied with a fabricated story saying he had murdered two of his sons, attempted to kill a third, and been sentenced to 21 years in prison. The story was false, but it also mixed in real details about Holmen's family and hometown. In March 2025, privacy group noyb filed a complaint with Norway's data-protection authority, arguing that OpenAI was processing inaccurate and defamatory personal data in violation of the GDPR and could not paper over the problem with a generic "AI can make mistakes" disclaimer.
Apple pulled AI news summaries after fake BBC headlines
Apple Intelligence's notification-summary feature spent late 2024 turning news alerts into fiction with excellent lock-screen placement. In the most widely cited example, it generated a false BBC alert claiming Luigi Mangione had shot himself. The BBC complained that Apple was attaching fabricated claims to its reporting, other publishers raised similar concerns, and Apple responded in January 2025 by disabling notification summaries for News & Entertainment apps in iOS 18.3 while it reworked the feature.
Cody Enterprise reporter resigned after AI fabricated quotes from real people
The Cody Enterprise was forced into public apologies and corrections in August 2024 after reporter Aaron Pelczar resigned amid evidence that an AI tool he used to help write stories had inserted fabricated quotations. A competing reporter at the Powell Tribune spotted robotic phrasing, suspiciously polished source quotes, and one article that bizarrely ended by explaining the inverted pyramid style of news writing. The resulting review found seven stories that included invented or altered quotes from seven people, including Wyoming Gov. Mark Gordon. The paper removed many of the quotes, issued corrections, and then adopted an AI detection and policy response after learning, a little late, that generative text tools are not interchangeable with reporting.
Meta AI answers spark backlash after wrong and sensitive replies
Meta rolled out its Llama 3-powered AI assistant across Facebook, Instagram, WhatsApp, and Messenger in April 2024, replacing the familiar search bar with "Ask Meta AI anything" prompts. The assistant struggled with factual accuracy from the start - the New York Times found it unreliable with facts, numbers, and web search. In July, when asked about the Trump rally shooting, Meta AI stated the assassination attempt had not happened. Meta blamed hallucinations, updated the system, and acknowledged that "all generative AI systems can return inaccurate or inappropriate outputs."
Google’s AI Overviews says to eat rocks
Within days of Google launching AI Overviews to all US search users in May 2024, the feature produced a series of confidently wrong answers that went viral. It told users to add non-toxic glue to pizza to make cheese stick better (sourced from an 11-year-old Reddit joke), that geologists recommend eating one rock per day for vitamins, and that Barack Obama was Muslim. Google head of search Liz Reid acknowledged the errors in a blog post, calling some results "odd, inaccurate or unhelpful," and the company made corrections including limiting AI Overviews for health-related and sensitive queries.
NYC’s official AI bot told businesses to break laws
New York City launched a Microsoft-powered AI chatbot called MyCity in October 2023 to help small business owners navigate regulations. A March 2024 investigation by The Markup found the bot was routinely advising businesses to break the law - telling employers they could pocket workers' tips, landlords they could discriminate against housing voucher holders, and bosses they could fire whistleblowers. Mayor Eric Adams acknowledged the errors but refused to take the chatbot offline, calling AI a "once-in-a-generation opportunity." NYU professor Julia Stoyanovich called the city's approach "reckless and irresponsible."
AI hallucinated packages fuel "Slop Squatting" vulnerabilities
Security researcher Bar Lanyado at Lasso Security discovered that AI code assistants consistently hallucinate nonexistent software package names when answering programming questions - and that nearly 30% of prompts produce at least one fake package recommendation. Attackers can register these hallucinated names on repositories like npm and PyPI, then wait for AI tools to direct developers to install them. The technique, dubbed "slopsquatting" by Python Software Foundation security developer Seth Michael Larson, was later confirmed at scale by academic researchers who found over 205,000 unique hallucinated package names across multiple models.
Gemini paused people images after historical inaccuracies
Google paused Gemini's image generation of people on February 22, 2024, after users discovered the tool was producing historically inaccurate depictions - including racially diverse World War II German soldiers, Black female popes, and multiethnic U.S. Founding Fathers. The overcorrection stemmed from diversity tuning meant to counter training-data biases, but the model failed to distinguish when diversity adjustments were inappropriate for specific historical prompts. CEO Sundar Pichai called the outputs "completely unacceptable." Google SVP Prabhakar Raghavan later published a blog post acknowledging the model had "overcompensated" and been "over-conservative."
Air Canada liable for lying chatbot promises
Jake Moffatt used Air Canada's website chatbot to ask about bereavement fares after his grandmother died. The chatbot told him he could book at full price and apply for a bereavement discount within 90 days. Air Canada's actual policy did not allow retroactive bereavement fare claims. When Moffatt applied, the airline denied the refund and admitted the chatbot had provided "misleading words" - but argued Moffatt should have checked the static webpage instead. British Columbia's Civil Resolution Tribunal ruled in Moffatt's favor in February 2024, finding Air Canada liable for negligent misrepresentation and rejecting the airline's argument that it wasn't responsible for its own chatbot's statements.
Gannett pauses AI sports recaps after mockery
In August 2023, Gannett - the largest newspaper chain in the United States - deployed an AI service called LedeAI to auto-generate high school sports recaps for the Columbus Dispatch and other papers. The articles went viral on social media for their robotic phrasing, missing player names, and bizarre constructions like "close encounter of the athletic kind." Several articles required corrections appended with notes about "errors in coding, programming or style." Gannett paused the experiment and said it would add "hundreds of reporting jobs" alongside AI tools, though the connection between the two claims was unclear.
Lawyers filed ChatGPT’s imaginary cases; judge fined them
In Mata v. Avianca (S.D.N.Y.), plaintiff Roberto Mata sued the airline after a metal serving cart struck his knee during a 2019 flight. His attorney Peter LoDuca filed a brief opposing dismissal that cited six judicial decisions. When opposing counsel and the court couldn't locate any of the cited cases, Judge Kevin Castel demanded copies. It turned out attorney Steven Schwartz at the same firm had used ChatGPT to research and draft the brief, and the AI had fabricated every case, complete with fake quotes and fake internal citations. On June 22, 2023, Castel sanctioned Schwartz, LoDuca, and their firm Levidow, Levidow & Oberman with a $5,000 penalty and required them to send notices to the real judges whose names appeared in the fabricated opinions.
Google’s Bard ad made False JWST “first” Claim
Google unveiled Bard on February 6, 2023, with a promotional ad on Twitter demonstrating the chatbot answering a question about the James Webb Space Telescope. Given the prompt "What new discoveries from the JWST can I tell my 9-year old about?", Bard stated that the JWST had taken the first pictures of a planet outside our solar system. This was false - the European Southern Observatory's Very Large Telescope captured the first direct exoplanet image in 2004. Reuters spotted the error on February 8, the day of a Google AI event in Paris. Alphabet shares dropped roughly 9% that day, erasing about $100 billion in market value.
CNET mass-corrects AI-written finance explainers
Starting in November 2022, CNET quietly published 77 financial explainer articles written by an AI tool under the byline "CNET Money Staff." Readers had to hover over the byline to learn the articles were produced "using automation technology." In January 2023, Futurism broke the story, and a follow-up identified factual errors in a compound interest article, prompting a full audit. CNET editor-in-chief Connie Guglielmo confirmed corrections were issued on 41 of the 77 articles - more than half - including some she described as "substantial." CNET paused AI-generated publishing and updated its disclosure practices, though Guglielmo said the outlet intended to continue using AI tools.