Aktiver — semantic AI platform Deterministic · traceable · behind your firewall
Insights / Enterprise AI

A $1.6M government health report cited studies that do not exist

On November 22, the Newfoundland and Labrador outlet The Independent reported that a Health Human Resources Plan written by Deloitte for the provincial government contains at least four citations that do not, or appear not to, exist. The 526-page report, published by the province’s Department of Health and Community Services in May 2025, cost nearly $1.6 million according to documents obtained through an access to information request. It was commissioned to address shortages of nurses and doctors.

The story follows a similar episode in Australia, where Deloitte’s Australian member firm partially refunded the government over a report found to contain apparent AI-generated errors. Fortune picked up the Canadian case on November 25.

/ 01What happened

The Independent detailed three of the problem citations. One, attributed in part to Martha MacLeod, professor emerita at the University of Northern British Columbia, described a cost-effectiveness study of a rural nurse retention programme. MacLeod called the citation “false” and “potentially AI-generated,” and said her team never carried out a cost-effectiveness analysis. A second paper, linked to Dalhousie University’s Gail Tomblin Murphy, “does not exist,” she said; she had worked with only three of the six other named authors. A third reference, to an article in the Canadian Journal of Respiratory Therapy, linked to an unrelated paper, and the reporters could not confirm the cited article existed.

According to Fortune, the false citations were used to support conclusions in the report’s cost-effectiveness analyses. Tomblin Murphy told The Independent: “It sounds like if you’re coming up with things like this, they may be pretty heavily using AI to generate work.”

Deloitte Canada’s statement, quoted by Fortune, said the firm “firmly stands behind the recommendations put forward in our report,” that it was “revising the report to make a small number of citation corrections, which do not impact the report findings,” and that “AI was not used to write the report; it was selectively used to support a small number of research citations.”

/ 02Why it matters

The firm may well be right that its recommendations stand. That is not the point. A citation is a promise that a claim can be checked. When the promise is false, the reader can no longer tell which conclusions rest on evidence and which rest on text that merely looks like evidence. A health system deciding how to spend money on recruitment and retention needs to know the difference.

The incident also shows where AI failures in professional services actually bite. Nobody was misled by a chatbot in public. The error sat inside a long, authoritative document, in the part most readers skim: the references. It surfaced only because journalists and the cited researchers went and checked.

/ 03The traceability angle

Language models generate references the same way they generate everything else: as plausible sequences of words. A real author, a realistic title and a credible journal can be stitched together without any underlying paper. Asking the model to be more careful does not change that mechanism.

The structural fix is to separate finding evidence from writing prose. If every claim in a report is linked to a node in a knowledge graph, and every node is linked to a specific document that was actually ingested, then a citation is not generated; it is looked up. A reference that does not resolve to a stored source cannot appear in the output at all. Reviewers can then spend their time judging the argument instead of checking whether the bibliography is real.

The same discipline applies inside organisations, not only in consulting reports. Internal briefings, policy analyses and research summaries increasingly pass through AI tools. Each one is a place where a plausible but unfounded reference can slip in and then be repeated by people who assume someone else checked it.

A citation should be a pointer to a document you hold, not a sentence a model wrote.

/ 04What to do

  • For buyers of analysis: require disclosure of AI use in contracted reports, and spot-check references as part of acceptance.
  • For firms producing it: keep generated text and source retrieval in separate, auditable steps, and fail loudly when a claim has no source.
  • For everyone: treat bibliographies as data, not decoration, and validate them automatically.
How it unfolded
MAY 2025Province publishes Deloitte report[1]NOV 22, 2025The Independent flags fake citations[1]NOV 25, 2025This postMAY 2025Province publishes Deloitte report [1]NOV 22, 2025The Independent flags fake citations [1]NOV 25, 2025This post
Aktiver’s take — Healthcare & pharma

The recommendations may well stand. But a cost-effectiveness case for nurse retention is only as good as the studies under it, and here a reader could not tell which studies were real without contacting their supposed authors.

Treat the bibliography as data instead. Ingest the papers a report relies on, extract each finding as a claim with its population, intervention and outcome, and link every sentence of analysis to the claims it uses. A citation then becomes a lookup against documents actually held; a reference with no stored source has nothing to resolve to and cannot be emitted. The cardiovascular build on our site follows the same discipline: 145 concepts and 284 recommended actions, each tied to the page it came from. For a health department reviewing a workforce plan, the useful question becomes answerable quickly: which conclusions rest on which studies, and which rest on nothing at all.

How we’d model it

Report conclusionStudy findingIngested paperResearcherCitationsupportedByextractedFromauthoredByresolvesToReport conclusionStudy findingIngested paperResearcherCitationsupportedByextractedFromauthoredByresolvesTo
  • Report conclusion —supportedBy→ Study finding
  • Study finding —extractedFrom→ Ingested paper
  • Ingested paper —authoredBy→ Researcher
  • Citation —resolvesTo→ Ingested paper

See where this applies →

Go deeper

See how Aktiver turns raw data into knowledge graphs your AI can reason over — deterministic, traceable, behind your firewall.

← All insights Subscribe via RSS → Book a demo →
Keep reading