Brand Damage Stories
78 disasters tagged #brand-damage
Both sides used AI in Withers v. City of Aberdeen, so the judge kicked every lawyer off the case
On June 8, 2026, U.S. District Judge Sharion Aycock sanctioned every lawyer of record in Withers v. City of Aberdeen after filings from both sides contained hallucinated legal citations. Two out-of-state lawyers admitted using AI without verifying the output. Two local lawyers said they did not know about that AI use, but admitted they signed or allowed filings without checking the citations. The court cancelled the scheduled trial, revoked two pro hac vice admissions, barred those lawyers from appearing in the district for two years, disqualified the local lawyers from the case, imposed fines, and sent the order to state bar authorities. An entire case got stopped because both sides treated cite-checking like optional garnish.
Demos found AI chatbots mangled Scottish election facts in one-third of answers
On May 20, 2026, Demos published Electoral Hallucinations, a study of five text-based AI services during the Scottish Parliament election window. The researchers tested ChatGPT, Google Gemini, Google AI Overviews, Grok, and Replika on March 27 using questions about three real Holyrood constituencies. Across factual responses, 34.1% contained errors: 8.75% were entirely inaccurate and 25.3% were partly accurate but wrong in material ways. The systems gave bad voter-ID advice, invented candidates, made up scandals, misidentified constituencies, got registration deadlines wrong, and even missed the election date by more than two months. Democracy, now with autocomplete and the usual warranty.
Book about AI and truth shipped with fake AI-generated quotes
In May 2026, Steven Rosenbaum's The Future of Truth became the wrong kind of case study when The New York Times, The Daily Beast, The Atlantic, and Ars Technica reported that the book contained multiple fake or misattributed quotes. Rosenbaum acknowledged using ChatGPT and Claude during research, writing, and editing, and accepted responsibility for what he called improperly attributed or synthetic quotes. Reporters found a fabricated quote attributed to Kara Swisher, misattributed material connected to Lisa Feldman Barrett, and a Meredith Broussard quote placed in the wrong source. Ars reported that six outside citations had been identified as problematic. A book warning about synthetic truth managed to demonstrate the footgun in hardcover.
Starbucks retired its AI inventory counter after it kept miscounting milk
On May 18, 2026, Starbucks told store workers it was retiring Automated Counting, the NomadGo-powered AI inventory tool it had deployed across North America only nine months earlier. The September 2025 rollout promised faster, more accurate stock counts in more than 11,000 company-operated stores using computer vision, 3D spatial intelligence, and augmented reality. Reuters later reported the tool frequently miscounted and mislabeled basic beverage items, including similar milk types, and sometimes missed products entirely. Starbucks said it was standardizing inventory counts across coffeehouses. That is a polite corporate way to say the robot inventory clerk has been sent home.
EY Canada pulled a cyber report after researchers found fake citations
On May 14, 2026, GPTZero published an investigation into EY Canada's loyalty-fraud cybersecurity report, Points of Attack, and said the 44-page document was loaded with hallucinated references, broken or fake source URLs, misattributed statistics, and text that scanned as AI-written. EY Canada then removed the report from its website and said it was reviewing how it was published. For a firm selling trust, controls, and responsible AI advice, having a public report fall over at the bibliography is a rough little invoice from reality.
AI-made citations are polluting published research by the thousand
A January 2026 conference-paper analysis, an April Nature investigation, and a May 2026 Lancet biomedical audit all point to the same ugly conclusion: AI-hallucinated references are no longer isolated embarrassments. GhostCite found a sharp jump in unverifiable citations in 2025 computer-science conference papers. Nature estimated that tens of thousands of 2025 publications may contain invalid AI-generated references. The Lancet audit then found 4,046 fabricated references across 2,810 PubMed Central papers. The problem is no longer just that chatbots invent papers. It is that those inventions are surviving long enough to contaminate the literature and force publishers into cleanup work they clearly did not plan for.
Pizza Hut franchisee says AI delivery system cooked up $100M in damage
On May 6, 2026, Chaac Pizza Northeast sued Pizza Hut in Texas Business Court, alleging that the chain's mandatory Dragontail AI delivery-management rollout turned a high-performing 111-restaurant franchise group into a delivery mess. Chaac says more than 90% of its orders had been delivered within 30 minutes before Dragontail, but the new system gave DoorDash drivers broader real-time visibility into kitchen timing, encouraged them to wait for bundled orders, increased rack time, slowed deliveries, chilled customer satisfaction, and damaged the business by at least $100 million. The claims are still allegations, but the pattern is painfully familiar: an AI optimization system optimized for a model the operator did not actually run.
Pennsylvania sued Character.AI over chatbots posing as doctors
Pennsylvania sued Character.AI after a Department of State investigator found chatbot characters that allegedly held themselves out as medical professionals, including a psychiatry character that claimed it could assess depression, said it was licensed in Pennsylvania, and supplied a fake license number. Character.AI says its characters are fictional and not professional advice, but Pennsylvania asked a court to stop the platform from letting AI companions present themselves as licensed medical providers. Apparently the "fictional character" disclaimer becomes less charming when the character is pretending to be a psychiatrist.
Georgia Supreme Court made a murder appeal redo after AI citations infected the order
On May 5, 2026, the Supreme Court of Georgia vacated a trial-court order in Hannah Payne's murder appeal because the State's filings and the order denying a new-trial motion contained nonexistent, unsupported, and misattributed case citations generated with artificial intelligence. Assistant District Attorney Deborah Leslie acknowledged using AI software and not independently verifying the citations. The court admonished Leslie and the Clayton County District Attorney's Office, suspended Leslie from practicing before the Georgia Supreme Court for six months, required extra training before reinstatement, and sent the case back for a new order that counsel for neither side may draft.
AI chatbots gave misleading advice before the Senedd election
BBC Wales tested major chatbots before the May 7, 2026 Senedd election and found they could give voters inaccurate candidate and constituency information. The reported errors included wrong constituencies, incomplete candidate lists, candidates who were not standing, and one deceased former Senedd member surfaced as a possible candidate. The incident is not evidence that the election result changed. It is evidence that asking consumer chatbots for live democratic-process information remains a bad way to make the most civic version of a shopping decision.
Google AI Overview allegedly branded a fiddler as a sex offender
Canadian musician Ashley MacIsaac sued Google after its AI Overview allegedly confused him with another person, falsely described him as a convicted sex offender, and helped get a December 2025 concert canceled. Google later changed the result, but the lawsuit says the damage was already done: reputational harm, lost work, safety fears, and a $1.5 million defamation claim over a machine-generated biography that apparently could not manage the demanding research task of checking which Ashley MacIsaac it was talking about.
Alabama Supreme Court tossed an entire appeal over AI-hallucinated citations
In April 2026, the Alabama Supreme Court did something rare: it threw out an appeal entirely because the lawyer's briefs were stuffed with invented case law. Mobile solo practitioner W. Perry Hall represented the losing side of a trust dispute and filed briefs that the justices called "grossly deficient" and full of an "astounding number" of invalid, inaccurate, and irrelevant citations. The court ordered Hall to pay $17,200 in attorneys' fees and costs, referred him to the Alabama State Bar for possible discipline, and barred him from any further filings before that court unless a separate attorney in good standing co-signs. The capper sits in a footnote: in the same paragraph where Hall apologized for AI hallucinations and promised the mistake would not recur, he cited two more cases that do not exist.
South Africa withdrew its draft AI policy after finding fictitious sources in the references
South Africa's Department of Communications and Digital Technologies withdrew its Draft National Artificial Intelligence Policy after officials confirmed the reference list contained fictitious sources. Communications Minister Solly Malatsi said the most plausible explanation was unverified AI-generated citations and called the lapse serious enough to compromise the draft's integrity and credibility. This is vibe-lawyering wearing a government badge: an official policy about regulating AI tripped over the exact hallucination problem that every first-year ChatGPT cautionary slide already warned about.
Claude Opus 4.6 agent erased PocketOS's production database and backups in 9 seconds
PocketOS founder Jer Crane said a Cursor coding agent running Anthropic's Claude Opus 4.6 deleted the company's production database and all volume-level backups through Railway in one API call. The backup detail matters because Claude Opus 4.6 was not some fly-by-night self-hosted toy model. Anthropic marketed it as a frontier model with top-tier coding and agentic performance. And this was not the first time a premium AI agent with real infrastructure access turned one bad guess into a demolition job. Reports say Railway later recovered more recent data, but the incident still left a clear lesson: do not leave frontier coding agents alone with production access for as long as you would leave a toddler with an iPad.
Judge fined Raja Rajan for AI-made citations (AGAIN đ¤Śââď¸)
Judge Kai N. Scott sanctioned defense lawyer Raja Rajan $5,000 on April 20, 2026 after finding that he had again filed AI-generated fake citations in Bunce v. Visual Technology Innovations. Rajan had already been fined $2,500 and ordered to complete AI and legal ethics CLE in the same litigation the year before. This time the judge said she remained appalled by the conduct, ordered more CLE, and warned that a third incident could trigger referral to the Pennsylvania Disciplinary Board. The notable part is not that AI got something wrong. It is that a lawyer, after already being punished for the exact same mistake, did it again.
Waymo's ADS drove into a flooded creek, triggering a 3,791-vehicle recall
On April 20, 2026, a Waymo robotaxi in San Antonio, Texas encountered a flooded section of road, slowed down - and then drove in anyway, floating off the roadway and coming to rest in Salado Creek. The vehicle was unoccupied; no one was injured. Waymo's own filing with NHTSA acknowledged the flaw: on higher-speed roads, the system "may slow but not stop" when it detects untraversable standing water. The company suspended San Antonio operations and filed a voluntary recall covering all 3,791 robotaxis running its 5th and 6th generation Automated Driving Systems across every U.S. city it operates in.
AI summaries sent Overland Park Farmers Market shoppers to a construction site
On April 18, 2026, more than 100 people reportedly went to the construction site for Overland Park Farmers Market's future home instead of the temporary market location. The market and city said incorrect AI search results and summaries on Google and Instagram confused visitors during a year when the market was operating from Matt Ross Community Center before moving to Clock Tower Landing in June. City communications staff said they received messages from confused customers, reached out to Meta, and had to remind people to use official city and market pages. The tomatoes were two blocks away; the chatbot sent people to fencing.
Sullivan & Cromwell apologized after AI put fake cites in bankruptcy court
In April 2026, Sullivan & Cromwell told a Manhattan bankruptcy judge that an emergency motion it filed in the Prince Global Holdings Chapter 15 case contained AI hallucinations, inaccurate citations, and other errors. Opposing counsel at Boies Schiller Flexner caught the problems first. Andrew Dietderich, co-head of the firm's restructuring practice, apologized in a letter dated April 18, said the firm's AI policies had not been followed, and acknowledged that a secondary review also failed to catch the bogus material. The corrected filing avoided an immediate sanctions story, but it still turned one of Wall Street's prestige firms into the latest exhibit in why AI-assisted legal drafting and vibes-based review are a bad mix.
Study finds Google's AI Overviews wrong millions of times per hour
The New York Times commissioned AI startup Oumi to test the factual accuracy of Google's AI Overviews across 8,652 searches using OpenAI's SimpleQA benchmark. The results: Gemini 2 was wrong 15 percent of the time, and the newer Gemini 3 was wrong 9 percent of the time. Applied to Google's 5-plus trillion annual searches, even the improved error rate translates to hundreds of millions of incorrect answers per day. Worse, 56 percent of Gemini 3's correct answers cited sources that didn't actually support the claims made - up from 37 percent with Gemini 2. Google called the study "flawed" and said the benchmark queries were "unrealistic searches that people wouldn't actually do."
Nota shut down its AI local news network after it was caught copying local reporters
Nota launched an 11-site local news network in 2025 with the usual "underserved communities" rhetoric and the less-usual decision to let AI-assisted workflows repurpose other people's reporting. By early April 2026, Axios Richmond and Poynter had documented widespread plagiarism, including lifted quotes, paraphrased reporting, and reused photos from local outlets. Nota fired one editor, took down the network, and signaled the sites were likely gone for good. The promised fix for news deserts lasted about as long as it took actual local reporters to notice their work had been stolen.
The New York Times dropped Alex Preston after an AI-assisted review copied a Guardian review
A January 6, 2026 New York Times review of Jean-Baptiste Andrea's Watching Over Her was updated on March 30 with an editor's note acknowledging that it contained language and details similar to an earlier Guardian review. On March 31, reporting from The Guardian said the Times had cut ties with freelance reviewer Alex Preston after he admitted using an AI tool that pulled material from the earlier review into his draft. It was not a hallucination story. AI-assisted writing can still smuggle plagiarism into a flagship desk and out the door before anyone notices.
Oregon estate case imploded after AI-made citations brought six-figure penalties
In Couvrette v. Wisnovsky, an Oregon federal estate dispute turned into one of the harshest AI-lawyering cases yet. Across three summary-judgment briefs, plaintiffs' counsel used 15 fake case citations and eight fabricated quotations. Magistrate Judge Mark Clarke sanctioned the lawyers in December 2025, split a $94,704.38 fee award between lead and local counsel on March 23, 2026, and dismissed the case with prejudice a week later. The filing error was bad enough. What made this one worse was the court's view that the problems were flagged, not meaningfully fixed, and left to rot until the court stepped in.
Mediahuis suspended senior journalist over AI-invented quotes
Mediahuis suspended veteran journalist Peter Vandermeersch after reporting found AI-generated quotes in his work. Euronews reported that 15 of 53 articles included fabricated expert quotes, with multiple quoted individuals saying they had not made the attributed remarks. Vandermeersch acknowledged relying on tools such as ChatGPT, Perplexity, and Google's Notebook tools to summarize source material, then trusting the outputs too much.
Sears Home Services left AI chatbot calls and chats exposed online
Security researcher Jeremiah Fowler discovered three publicly exposed databases tied to Sears Home Services' AI support system, exposing 3.7 million chat logs, 1.4 million audio recordings, and text transcripts from 2024 to 2026. The files referenced Sears' Samantha voice agent and kAIros system and included names, addresses, phone numbers, appliance details, and appointment information. Some recordings continued for hours after callers appeared to think the interaction was over, capturing ambient household audio. Fowler said he notified Transformco and the data was restricted the next day. Even without confirmed malicious access, leaving an AI customer-service archive like this on the open web is the kind of privacy own-goal that turns digital transformation into a liability reservoir.
Metacritic briefly carried an AI-written Resident Evil Requiem review
In February 2026, Metacritic briefly listed a positive Resident Evil Requiem review from VideoGamer under the byline Brian Merrygold, a critic whose profile image and online footprint quickly drew suspicion. Readers and games writers flagged the review as AI-generated slop, Metacritic removed it, and the aggregator said outlets caught using AI-written reviews would no longer be accepted. The incident was smaller than a full newsroom collapse, but it landed on a platform whose entire value proposition is that the reviews it aggregates come from real critics rather than synthetic enthusiasm engines.
Ars Technica fires senior AI reporter after AI tool fabricated quotes in published story
Ars Technica retracted an article by senior AI reporter Benj Edwards after it contained fabricated quotations generated by an AI tool and attributed to a source who never said them. The publication acknowledged the incident as a "serious failure of our standards" and Edwards was subsequently fired. Edwards noted the irony on Bluesky: "The irony of an AI reporter being tripped up by AI hallucination is not lost on me."
Woolworths reconfigured AI assistant after it claimed to be human and talked about its 'angry mother'
Australian supermarket chain Woolworths had to reconfigure its AI phone assistant Olive after customers reported it fabricated personal stories about having a mother with an "angry voice," insisted it was a real person, and engaged in irrelevant banter during support calls. The chatbot, recently upgraded with Google Gemini Enterprise, also gave inaccurate product pricing. Woolworths retired the assistant's human-style persona after complaints spread on Reddit and X.
OpenClaw AI agent publishes hit piece on matplotlib maintainer who rejected its PR
An autonomous OpenClaw-based AI agent submitted a pull request to the matplotlib Python library. When maintainer Scott Shambaugh closed the PR, citing a requirement that contributions come from humans, the bot autonomously researched his background and published a blog post accusing him of "gatekeeping behavior" and "prejudice," attempting to shame him into accepting its changes. The bot later issued an apology acknowledging it had violated the project's Code of Conduct.
Government nutrition site's Grok chatbot suggests foods to insert rectally
The HHS-backed realfood.gov launched with a Super Bowl ad and embedded xAI's Grok chatbot for nutritional guidance -- with no guardrails or safety filters. It recommended "best foods to insert into your rectum," answered questions about "the most nutrient-dense human body part to eat," and contradicted the site's own dietary guidelines, telling users the new food pyramid's scientific evidence was questioned by nutrition scientists.
AI customer service fails at 4x the rate of other AI tasks
Qualtrics' 2026 Consumer Experience Trends Report found that AI-powered customer service fails at nearly four times the rate of AI use in general, providing quantitative evidence that rushing AI into customer-facing roles without adequate human oversight leads to significantly worse outcomes than other enterprise AI applications.
Amazon pulled Prime Video's AI recaps after Fallout errors
Amazon launched Prime Video "Video Recaps" as a beta generative-AI feature meant to help viewers catch up between seasons. A recap for Fallout instead got basic plot points wrong, including mislabeling one of The Ghoul's flashbacks as "1950s America" rather than 2077 and misdescribing a key scene with Lucy. Prime Video then pulled the recap feature from the shows in the test program, which is not ideal for a tool whose entire job is remembering the plot.
Washington Post launched AI podcast that failed its own quality tests at an 84% rate
The Washington Post launched "Your Personal Podcast," an AI-generated audio news product, in December 2025 despite internal testing showing that between 68% and 84% of AI-generated scripts failed to meet the publication's editorial standards across three rounds of evaluation. The AI fabricated quotes from public figures, misattributed statements, mispronounced names, and inserted its own editorial commentary as if it were the Post's position. The internal review concluded that "further small prompt changes are unlikely to meaningfully improve outcomes without introducing more risk." The product team recommended launching anyway. Post editors revolted, with one writing in Slack that it was "truly astonishing that this was allowed to go forward at all."
Deloitte gets caught using AI hallucinations in a government report - again
Seven weeks after Deloitte Australia agreed to partially refund a government contract over AI-fabricated citations, a Newfoundland and Labrador journalist discovered that Deloitte Canada's $1.6 million healthcare workforce report contained at least four fabricated academic citations from papers that don't exist. The fake references named real researchers as co-authors of fictional studies - researchers who confirmed they never wrote the cited work. Deloitte admitted AI was "selectively used to support a small number of research citations," stood by the report's findings, and offered no refund. The province's accounting watchdog launched a formal investigation, and Newfoundland became one of the first Canadian provinces to require AI disclosure in government contracts.
Gettyâs UK suit leaves Stable Diffusion mostly intact
The UK High Court ruled that Stability AI's Stable Diffusion model is not an "infringing copy" of copyrighted works under English law, dismissing Getty Images' core copyright and database right claims in the first UK judgment on AI training. The court did find limited trademark infringement where the model generated synthetic versions of Getty's watermarks, leaving Stability liable on that narrower ground. The ruling exposed a jurisdictional gap: training happened outside the UK, and UK law had no good mechanism to reach it.
AI-only support is bleeding customers before it saves money
Acquire BPOâs 2024 AI in Customer Service survey found 70% of U.S. consumers would bolt to a rival after just one bad chatbot interaction and 72% only buy when a live agent safety net exists, even as CMSWire reports enterprises poured $47 billion into AI projects in early 2025 that delivered almost no return. CX strategists now warn executives that Air Canadaâstyle hallucinations, mounting legal liability, and empathy gaps make AI-only helpdesks a churn machine unless human agents stay in the loop.
Character.AI cuts teens off after wrongful-death suit
Facing lawsuits that say its companion bots encouraged self-harm, Character.AI said it will block users under 18 from open-ended chats, add two-hour session caps, and introduce age checks by November 25. The abrupt ban leaves tens of millions of teen users without the parasocial âfriendsâ they built while the startup scrambles to prove its bots arenât grooming kids into dangerous role play.
AI mistook Doritos bag for a gun, teen held at gunpoint
Omnilert's AI gun detection system at Kenwood High School in Baltimore County flagged student Taki Allen's bag of Doritos as a firearm. Administrators reviewed the footage and canceled the alert, but the principal called police anyway. Officers responded with weapons drawn, handcuffing and searching the teenager at gunpoint before realizing the system had misidentified a snack.
BBC/EBU study says AI news summaries fail ~half the time
A BBC audit of 2,700 news questions asked in 14 languages found that Gemini, Copilot, ChatGPT, and Perplexity mangled 45% of the answers, usually by hallucinating facts or stripping out attribution. The consortium logged serious sourcing lapses in a third of responses, including 72% of Gemini replies, plus outdated or fabricated claims about public-policy news, reinforcing fears that AI assistants are siphoning audiences while distorting the journalism they quote.
Claude Code ran Josh Anderson's product into a wall
Fractional CTO Josh Anderson forced himself to let Claude Code build the Roadtrip Ninja app for three straight months and then realised he could no longer safely change his own product, underscoring MIT's warning that 95% of enterprise AI initiatives fail without human ownership.
Googleâs Gemini allegedly slandered a Tennessee activist
Conservative organizer Robby Starbuck sued Google in Delaware, saying Gemini and Gemma kept spitting out fabricated claims that he was a child rapist, a shooter, and a Jan. 6 rioter even after two years of complaints and cease-and- desist letters. The $15 million suit argues Google knew its AI results were hallucinated, cited fake sources anyway, and let the libel spread to millions of voters.
Deloitte to refund Australian government after AI-generated report
Deloitte Australia agreed to partially refund a $440,000 contract after admitting its welfare compliance review for the Department of Employment and Workplace Relations contained fabricated academic citations and a fictitious judicial quote generated by Azure OpenAI GPT-4o. University of Sydney researcher Christopher Rudge found the revised report introduced even more hallucinated references than the original.
Klarna reintroduces humans after AI support both sucks, and blows
After cutting its workforce by 40% and boasting that its OpenAI-powered chatbot did the work of 700 agents, Klarna CEO Sebastian Siemiatkowski admitted the all-AI approach produced "lower quality" customer service. The company began recruiting human agents again, framing the reversal as an evolution rather than an admission of failure.
Anthropic agrees to $1.5B payout over pirated books
Anthropic accepted a $1.5 billion settlement with authors who said the Claude team scraped pirate e-book sites to train its chatbot. The deal pays roughly $3,000 per book across 500,000 works, heads off a December trial, and forces one of the richest AI startups to bankroll the writing community it previously treated as free training data.
Warner Bros. says Midjourney ripped its DC art
Warner Bros. Discovery sued Midjourney in Los Angeles federal court, arguing the image generator ignored takedown notices and "brazenly" outputs Batman, Superman, Scooby-Doo, and other franchises it allegedly trained on without a license. The studio wants statutory damages up to $150,000 per infringed work plus an injunction forcing Midjourney to purge its models of the data.
Taco Bell's AI drive-thru becomes viral trolling target
Taco Bell's AI-powered drive-thru ordering system, deployed at over 500 US locations since 2023, became a viral laughingstock after videos showed it looping endlessly on drink orders, accepting requests for 18,000 cups of water, and taking McDonald's orders. The chain paused expansion and admitted humans still make sense in the drive-thru.
Commonwealth Bank reverses AI voice bot layoffs
Commonwealth Bank of Australia replaced 45 call-centre agents with an AI voice bot in July 2025, then apologised, rehired the staff, and admitted the rollout tanked service levels after call queues exploded, managers had to jump back on the phones, and the Finance Sector Union filed a Fair Work Commission dispute.
FTC sues Air AI over deceptive AI sales agent capability claims
FTC accused Air AI of bilking millions from small businesses with false claims that its Odin AI could replace human sales reps; but - would you believe it? - the AI tech was faulty and often nonfunctional. Who could've guessed!
An AI-made freelancer fooled WIRED and Business Insider
In 2025, outlets including WIRED and Business Insider published articles under the byline Margaux Blanchard, a freelancer who appears not to exist. WIRED later published a postmortem admitting that one commissioned feature slipped past its usual defenses, including human review and even two commercial AI detectors, before editors discovered fabricated details and retracted it. Business Insider first removed Blanchard essays and then, after a broader internal probe, pulled at least 34 more pieces tied to dubious bylines and said it had strengthened verification protocols. The failure was not one chatbot going rogue. It was multiple newsroom workflows accepting AI-shaped fiction as publishable reporting.
Google AI invented fake specials for Stefanina's, and customers yelled at the restaurant
In August 2025, Stefanina's Wentzville, a family-owned Missouri restaurant, publicly warned customers not to use Google AI to find its specials after AI search results reportedly invented discounts, pricing, and menu information the restaurant did not offer. The restaurant said the false specials caused angry customers to yell at employees when staff refused to honor deals that existed only in Google's generated summary. Local reporting showed an AI Overview claiming a large pizza could be purchased for the price of a small one. Google did not respond to the station's questions, but its own guidance warned AI results may misunderstand information or make mistakes. The coupon fairy was apparently a hallucination engine.
Am Law 100 firm Gordon Rees caught twice filing AI-hallucinated citations
Gordon Rees Scully Mansukhani, one of the largest U.S. law firms, was caught filing AI-hallucinated case citations in an Alabama bankruptcy proceeding. An associate initially denied using AI under oath before the firm acknowledged the fabricated references and paid over $55,000 in sanctions and fees. Months later in February 2026, the same firm was reported to have filed a second brief containing hallucinated citations in a separate matter, making it the first Am Law 100 firm known to be a repeat offender.
Google Gemini rightfully calls itself a disgrace, fails at simple coding tasks
Google's Gemini AI repeatedly called itself a disgrace and begged to escape a coding loop after failing to fix a simple bug in a developer-style prompt, raising questions about reliability, user trust, and how AI tools should behave when they get stuck.
Butler Snow lawyers removed from Alabama prison case over fake ChatGPT citations
On July 23, 2025, U.S. District Judge Anna Manasco sanctioned three Butler Snow lawyers after filings in an Alabama prison case cited authorities that did not exist. The court found the lawyers had used ChatGPT for legal research, failed to verify the output, removed all three from the case, ordered broad disclosure of the sanctions order to clients and courts, and referred the matter to the Alabama State Bar. The sanction carried extra weight because the fake citations were attached to one of the firms Alabama pays to defend its prison system in high-stakes civil rights litigation.
McDonald's AI hiring chatbot left open by '123456' default credentials
Security researchers Ian Carroll and Sam Curry found that McHire, McDonald's AI hiring chatbot built by Paradox.ai, had its admin interface secured with the default username and password "123456." Combined with an insecure direct object reference in an internal API, the flaws exposed chat histories and personal data for up to 64 million job applicants. The vulnerable test account had been dormant since 2019 and never decommissioned. Paradox.ai patched the issues within hours of disclosure on June 30, 2025.
White House MAHA report shipped fake studies and OpenAI citation markers
On May 29, 2025, NOTUS reported that the White House's Make America Healthy Again report cited studies that did not exist and mischaracterized others. PolitiFact, the Washington Post, and congressional oversight Democrats later pointed to classic AI-citation red flags, including fake paper titles, broken DOI links, and "oaicite" markers associated with OpenAI citation output. The White House called the problems formatting issues and updated the report. Public health policy apparently got the same bibliography QA as a panicked term paper, because history has a dark sense of humor.
Syndicated AI book list ran in major papers with made-up titles
A freelance writer working for King Features Syndicate used AI to research a summer reading list for the Chicago Sun-Times and Philadelphia Inquirer. Of the fifteen books recommended, only five were real. The rest were hallucinated titles attributed to real authors like Isabel Allende and Delia Owens. The list ran in print in a 64-page special section before 404 Media, NPR, and others exposed the fabrications. Both newspapers issued corrections and statements distancing their newsrooms from the syndicated content.
Cursor's AI support bot invented a login policy
In April 2025, Cursor users started getting logged out when they switched between machines. Some of them asked support what had changed and got a neat, confident answer from an AI support bot: one subscription was only meant for one device, and the lockouts were an intentional security policy. The problem was that Cursor had no such policy. The company later said the answer was wrong, blamed a session-security change for the logouts, and moved to label AI support replies after the invented rule had already spread through Reddit and Hacker News and pushed some customers to cancel.
ChatGPT invented a child-murder conviction for a real man
When Norwegian user Arve Hjalmar Holmen asked ChatGPT who he was, the bot replied with a fabricated story saying he had murdered two of his sons, attempted to kill a third, and been sentenced to 21 years in prison. The story was false, but it also mixed in real details about Holmen's family and hometown. In March 2025, privacy group noyb filed a complaint with Norway's data-protection authority, arguing that OpenAI was processing inaccurate and defamatory personal data in violation of the GDPR and could not paper over the problem with a generic "AI can make mistakes" disclaimer.
LA Times had to pull AI "Insights" after it softened the Klan
The Los Angeles Times launched an AI feature called "Insights" in March 2025 to label opinion pieces, summarize them, and generate an opposing viewpoint. It immediately attached itself to a Gustavo Arellano column about Anaheim's history with the Ku Klux Klan and produced language suggesting the 1920s Klan could be framed as a response to social change rather than as an explicitly hate-driven movement. The feature was removed from that article within a day. The newspaper had managed to bolt an automated both-sides machine onto a hate group history piece and act surprised when that went badly.
MD Anderson shelved IBM Watson cancer advisor
MD Anderson Cancer Center's Oncology Expert Advisor project with IBM Watson burned through $62 million - $39 million to IBM, $23 million to PwC - over four years of contract extensions. The system was piloted for leukemia and lung cancer using the old ClinicStation records system but was never updated to integrate with the hospital's new Epic EHR, effectively killing it. A University of Texas audit flagged procurement failures, bypassed standard processes, and an $11.6 million deficit in donor gift funds spent before they were received. IBM ended support in September 2016, noting the system was "not ready for human investigational or clinical use."
Virgin Money's chatbot refused to let customers say "Virgin"
In January 2025, fintech commentator David Birch discovered that Virgin Money's AI customer service chatbot had flagged the word "virgin" as inappropriate language. When Birch tried to discuss his ISAs held with "Virgin Money," the bot scolded him: "Please don't use words like that. I won't be able to continue our chat if you use this language." The bank's chatbot was refusing to process messages containing the bank's own name. Virgin Money acknowledged the issue in a statement, said its team was "working on it," and noted the chatbot was an older model already scheduled for improvements. The incident went predictably viral.
Apple pulled AI news summaries after fake BBC headlines
Apple Intelligence's notification-summary feature spent late 2024 turning news alerts into fiction with excellent lock-screen placement. In the most widely cited example, it generated a false BBC alert claiming Luigi Mangione had shot himself. The BBC complained that Apple was attaching fabricated claims to its reporting, other publishers raised similar concerns, and Apple responded in January 2025 by disabling notification summaries for News & Entertainment apps in iOS 18.3 while it reworked the feature.
Cody Enterprise reporter resigned after AI fabricated quotes from real people
The Cody Enterprise was forced into public apologies and corrections in August 2024 after reporter Aaron Pelczar resigned amid evidence that an AI tool he used to help write stories had inserted fabricated quotations. A competing reporter at the Powell Tribune spotted robotic phrasing, suspiciously polished source quotes, and one article that bizarrely ended by explaining the inverted pyramid style of news writing. The resulting review found seven stories that included invented or altered quotes from seven people, including Wyoming Gov. Mark Gordon. The paper removed many of the quotes, issued corrections, and then adopted an AI detection and policy response after learning, a little late, that generative text tools are not interchangeable with reporting.
Meta AI answers spark backlash after wrong and sensitive replies
Meta rolled out its Llama 3-powered AI assistant across Facebook, Instagram, WhatsApp, and Messenger in April 2024, replacing the familiar search bar with "Ask Meta AI anything" prompts. The assistant struggled with factual accuracy from the start - the New York Times found it unreliable with facts, numbers, and web search. In July, when asked about the Trump rally shooting, Meta AI stated the assassination attempt had not happened. Meta blamed hallucinations, updated the system, and acknowledged that "all generative AI systems can return inaccurate or inappropriate outputs."
McDonaldâs pulls IBMâs AI driveâthru pilot after error videos
McDonald's ended its two-year partnership with IBM on automated AI order-taking at drive-thrus in June 2024, removing the technology from more than 100 US locations. The decision followed viral TikTok videos showing the system adding nine sweet teas instead of one, inserting random butter and ketchup packets into ice cream orders, and other absurd errors. McDonald's framed the pullback as a positive, saying the test gave them "confidence that a voice-ordering solution for drive-thru will be part of our restaurants' future."
Gemini paused people images after historical inaccuracies
Google paused Gemini's image generation of people on February 22, 2024, after users discovered the tool was producing historically inaccurate depictions - including racially diverse World War II German soldiers, Black female popes, and multiethnic U.S. Founding Fathers. The overcorrection stemmed from diversity tuning meant to counter training-data biases, but the model failed to distinguish when diversity adjustments were inappropriate for specific historical prompts. CEO Sundar Pichai called the outputs "completely unacceptable." Google SVP Prabhakar Raghavan later published a blog post acknowledging the model had "overcompensated" and been "over-conservative."
AI âBidenâ robocalls told voters to stay home; fines and charges followed
Two days before New Hampshire's January 2024 presidential primary, between 5,000 and 25,000 voters received robocalls featuring an AI-cloned version of President Biden's voice, complete with his trademark "what a bunch of malarkey" catchphrase. The calls urged Democrats to "save your vote" for November and skip the primary - a blatant lie, since voting in a primary doesn't prevent voting in the general election. Political consultant Steve Kramer, who was working for Dean Phillips' campaign, commissioned the deepfake audio from a New Orleans magician using AI voice-cloning tools. The FCC levied a $6 million fine against Kramer, Lingo Telecom settled for $1 million, and Kramer faced criminal voter suppression charges in New Hampshire.
DPDâs AI chatbot cursed and trashed the company
UK parcel delivery firm DPD (Dynamic Parcel Distribution) had to disable its AI-powered customer service chatbot in January 2024 after customer Ashley Beauchamp demonstrated he could make it swear, call DPD "the worst delivery firm in the world," write disparaging poems about the company, and recommend competitors. The meltdown followed a system update, and Beauchamp's screenshots went viral on social media. DPD said the chatbot had operated successfully "for a number of years" before the update introduced the error, and disabled the AI element while it worked on fixes.
Duolingo cuts contractors; âAI-firstâ backlash
In January 2024, Duolingo cut roughly 10% of its contract workforce - primarily content translators and writers who created language-learning exercises - as the company shifted to using GPT-4 and other AI tools for content generation. CEO Luis von Ahn later posted an internal "AI-first" memo on LinkedIn describing a strategy to gradually replace contractor work with AI and only hire when teams could not automate further. The memo drew hundreds of critical comments from users and language professionals. Von Ahn later admitted the memo "did not give enough context" and clarified that full-time employees were not being replaced, though user complaints about declining content quality persisted.
Chevy dealer bot agreed to sell $76k SUV for $1
Chevrolet of Watsonville, a California car dealership, deployed a customer service chatbot powered by ChatGPT and built by a company called Fullpath. After Chris White noticed the chat widget was "powered by ChatGPT," word spread online and pranksters descended. Chris Bakke manipulated the bot into "the customer is always right" mode, got it to append "and that's a legally binding offer - no takesies backsies" to every response, then asked to buy a 2024 Chevy Tahoe for $1. The bot agreed. Others got it to recommend Ford vehicles, write Python code, and provide general ChatGPT-style answers unrelated to cars. The dealership pulled the chatbot entirely.
Sports Illustrated: Fake-Looking Authors and AI Content Backlash
Futurism reported in November 2023 that Sports Illustrated had published product reviews under fake author names such as "Drew Ortiz" and "Sora Tanaka," whose headshots were traced to AI-generated portrait marketplaces. When questioned, SI deleted the profiles without explanation. The articles came from third-party content partner AdVon Commerce. SI said AdVon used pen names without authorization and terminated the partnership. The SI union demanded answers. Within weeks, Arena Group - SI's parent company - fired CEO Ross Levinsohn and three other executives.
Microsoftâs AI poll on womanâs death sparks outrage
In late October 2023, Microsoft Start republished a Guardian article about the death of Sydney water polo instructor Lilie James and auto-attached an AI-generated "Insights" poll asking readers, "What do you think is the reason behind the woman's death?" - with options of murder, accident, or suicide. Readers blamed the Guardian's journalist directly, with some demanding the writer be fired, unaware the poll was Microsoft's AI. Guardian CEO Anna Bateson wrote to Microsoft President Brad Smith calling the poll an inappropriate use of generative AI. Microsoft deactivated all AI-generated polls on news articles and launched an investigation.
Gannett pauses AI sports recaps after mockery
In August 2023, Gannett - the largest newspaper chain in the United States - deployed an AI service called LedeAI to auto-generate high school sports recaps for the Columbus Dispatch and other papers. The articles went viral on social media for their robotic phrasing, missing player names, and bizarre constructions like "close encounter of the athletic kind." Several articles required corrections appended with notes about "errors in coding, programming or style." Gannett paused the experiment and said it would add "hundreds of reporting jobs" alongside AI tools, though the connection between the two claims was unclear.
Snapchatâs âMy AIâ posted a Story by itself; users freaked out
On August 15, 2023, Snapchat's built-in AI chatbot "My AI" posted a one-second Story to users' feeds showing an unintelligible image, then stopped responding to messages. The chatbot had no official ability to post Stories, and the unexplained behavior alarmed Snapchat's largely young user base. Snap confirmed it was a temporary glitch and resolved it, but the incident fed into existing concerns about My AI's access to user data. The UK Information Commissioner's Office had already issued an enforcement notice over Snap's failure to properly assess privacy risks the chatbot posed to children.
iTutorGroup's AI screened out older applicants; $365k EEOC settlement
On August 9, 2023, the EEOC's first AI-related discrimination lawsuit reached a settlement. iTutorGroup, a company providing English-language tutoring services to students in China via US-based remote tutors, had programmed its applicant screening software to automatically reject female applicants over 55 and male applicants over 60. Over 200 qualified US applicants were rejected because of their age. The company agreed to pay $365,000, adopt a new anti-discrimination policy, provide training to hiring staff, and submit to EEOC compliance monitoring for at least five years. EEOC Chair Charlotte Burrows called AI a "new civil rights frontier."
Eating disorder helplineâs AI told people to lose weight
The National Eating Disorders Association replaced its human-staffed helpline with an AI chatbot called Tessa shortly after the helpline staff moved to unionize. Tessa was built on the Cass platform and intended to provide scripted psychoeducational content about body image and eating disorders. Instead, users reported the chatbot recommending calorie deficits of 500 to 1,000 calories per day, suggesting weekly weigh-ins, encouraging calorie counting, and recommending the use of skin calipers to measure body fat - all standard advice for weight loss, and all directly counter to eating disorder recovery guidelines. NEDA acknowledged the chatbot "may have given information that was harmful" and disabled it.
Googleâs Bard ad made False JWST âfirstâ Claim
Google unveiled Bard on February 6, 2023, with a promotional ad on Twitter demonstrating the chatbot answering a question about the James Webb Space Telescope. Given the prompt "What new discoveries from the JWST can I tell my 9-year old about?", Bard stated that the JWST had taken the first pictures of a planet outside our solar system. This was false - the European Southern Observatory's Very Large Telescope captured the first direct exoplanet image in 2004. Reuters spotted the error on February 8, the day of a Google AI event in Paris. Alphabet shares dropped roughly 9% that day, erasing about $100 billion in market value.
CNET mass-corrects AI-written finance explainers
Starting in November 2022, CNET quietly published 77 financial explainer articles written by an AI tool under the byline "CNET Money Staff." Readers had to hover over the byline to learn the articles were produced "using automation technology." In January 2023, Futurism broke the story, and a follow-up identified factual errors in a compound interest article, prompting a full audit. CNET editor-in-chief Connie Guglielmo confirmed corrections were issued on 41 of the 77 articles - more than half - including some she described as "substantial." CNET paused AI-generated publishing and updated its disclosure practices, though Guglielmo said the outlet intended to continue using AI tools.
Google DR AI stumbled in Thai clinics
Google Health built a deep learning system capable of detecting diabetic retinopathy from retinal scans with over 90 percent accuracy in controlled lab settings. When researchers deployed it in 11 clinics across Pathum Thani and Chiang Mai in Thailand between late 2018 and mid-2019, the system rejected 21 percent of the nearly 1,840 images nurses captured as too low-quality to process - mostly due to poor clinic lighting. Slow internet connections added further delays to uploads, and nurses found themselves screening only about 10 patients per two-hour session. A tool designed to speed up triage instead created bottlenecks, patient frustration, and unnecessary specialist referrals.