References cleaned past recognition

Phil Holland wrote a 71-page report about the ethics and policy of controlling wild deer in New Zealand. The work was produced for the Kellogg Rural Leadership Programme, which asks participants to research an industry topic and present their findings to colleagues. Holland's report combined policy analysis, interviews, personal experience, and a literature review.

It also included an unusually direct disclosure about artificial intelligence. Holland wrote that he used ChatGPT to improve structure and grammar, test alternative wording, draft comparative models and policy summaries, and assist with APA references by checking formatting and suggesting citations. The disclosure said he had reviewed and critically assessed all generated material and accepted responsibility for the report's integrity.

The reference list did not survive contact with people whose names appeared in it.

Newsroom reported in July 2026 that several citations had the wrong authors, dates, or journals, while other listed papers did not appear to exist at all. Holland told the outlet that he wrote the report himself and ran the references through ChatGPT near the deadline. He accepted responsibility and described the skipped verification as the critical step that had not been done properly.

Real researchers, imaginary work

One reference credited Richard Allen, David Forsyth, and Elaine Wright with a 2021 paper titled "Impacts of introduced deer on indigenous vegetation: A global synthesis with implications for Aotearoa." The report supplied a DOI, the scholarly equivalent of a permanent address for a publication. Newsroom found that the address led to an unrelated paper about short-tailed bats.

Another citation attributed "Decolonising conservation: Māori environmental values and DOC partnerships" to University of Auckland researcher Marama Muru-Lanning. She told Newsroom she had never published on that subject.

Adrian Monks of the Bioeconomy Science Institute encountered a similar problem when a colleague asked him for a cited paper he did not recognize. The report relied on the supposed work while discussing whether low-intensity livestock grazing on public conservation land might help control deer and weeds. Monks said he had published research about a narrow case in which sheep or rabbit grazing could control certain exotic plants, but he would not generalize it to public conservation land.

This is a nastier failure than a broken URL. A fabricated citation borrows a real researcher's reputation and attaches it to a proposition the person did not make. Readers may repeat the claim, opponents may challenge the scientist over it, and policy discussions can treat an invented paper as evidence. The reference looks respectable because the names, journal, year, volume, and DOI all have the shape of scholarship. The shape was doing most of the work.

A programme report, not a journal article

The context deserves precision. Holland prepared the report as a participant in a leadership programme, not as an academic journal submission and not as part of his employment duties at Federated Farmers. The report was supported by a Federated Farmers scholarship, and Holland worked as a senior policy adviser there, but Newsroom reported that the project was completed in his personal capacity.

The New Zealand Rural Leadership Trust publishes Kellogg programme reports in their submitted form. Its disclaimer describes them as products of a participant's learning journey and tells readers to assess relevance and accuracy themselves. Chief executive James Ryan told Newsroom that the trust remained satisfied that Holland's overall findings were evidence-based and useful. He also said the citations failed to meet that standard.

That distinction keeps the blast radius in proportion. The report did not enact government policy, pass peer review, or order anyone to change deer-control practices. It was designed to reach industry representatives and colleagues, though, and it discussed current questions about conservation land, animal welfare, governance, and national deer strategy. Sources still matter in a document meant to contribute to those conversations. A disclaimer cannot convert fictional research into acceptable supporting material.

Deadline pressure met autocomplete

Holland told Newsroom that he put the references through ChatGPT during the "madness of a deadline." That sequence is painfully familiar to anybody who has assembled a bibliography late at night. Citation formatting is tedious, the document already feels finished, and handing the cleanup to a tool looks less consequential than handing over the argument.

A language model can standardize punctuation and capitalization when it receives complete, accurate source records. Asking it to suggest or repair missing citation details creates a different job. The model can produce a plausible title, combine the names of researchers working in the same field, assign a realistic journal, and generate a DOI-shaped string. None of those operations checks a library catalog or publisher database unless the workflow explicitly performs that lookup.

Holland's written disclosure said all AI output had been reviewed. His later account showed why "reviewed" needs a more demanding definition. Reading a polished reference and deciding that it looks right is copy editing. Verification means opening the DOI, locating the paper in the named journal, confirming the authors and publication details, and checking that the source supports the claim beside it.

The final reference list contained enough realistic detail to pass a visual inspection. It did not receive the duller, more useful inspection where somebody clicks the links.

Responsibility without theatrics

Holland did several things that often go missing after an AI-related publication failure. He disclosed his use of ChatGPT in the original report, spoke to the reporter, accepted responsibility, and did not blame the model or his employer. He compared the tool to a chainsaw: if he dropped a tree on a house, responsibility would remain with the operator.

The named researchers also kept their criticism measured. Monks told Newsroom he was more concerned about how science and policy systems would handle increasing volumes of synthetic material than about punishing Holland. AI researcher Andrew Lensen said the disclosure and acceptance of responsibility made the case less egregious than concealed AI use.

Accountability does not repair the citations, but it helps identify the actual process failure. The report moved from drafting to publication without anyone independently validating the bibliography. The author was both the person under deadline pressure and the final quality-control layer. ChatGPT supplied finishing work in the part of the document where an invented detail can masquerade most easily as authority.

Ryan said the trust would follow up on AI use in the citations and consider how AI should be handled in future programme reports. A workable review does not need an AI detector. It needs a source ledger: every reference opened, matched to a real publication, and tied to the claim it supports before the document goes online.

Holland's report may still contain worthwhile analysis of deer control. That makes the citation failure more wasteful. Real work now carries an avoidable question mark because the bibliography was given a cosmetic check rather than an evidentiary one. ChatGPT was asked to clean the references and returned several cleaner-looking references to research that never existed. Very tidy. Completely useless.