In HBO’s Chernobyl, there’s a nine-second moment where someone in the control room asks for a radiation reading and is told it is 3.6 roentgen. The deputy chief engineer’s reply, “not great, not terrible,” went on to become a meme, usually quoted as a line about denial.
The more useful thing about that scene is that the number was correct.
The dosimeter wasn’t broken and nobody was lying. 3.6 was the highest figure that model could display. The real level, once instruments with a usable range arrived, was somewhere around 15,000. The device had been asked a question outside its range and answered the only way it could, which was confidently and precisely. Nothing in its output could say this is my ceiling, not your reading.
The instrument’s limit is a matter of record. The line itself was written for the series rather than recorded at the time, so treat it as television, not history.
That scene keeps coming back to me when I watch how we check AI output, because we have all learned to ask one version of the question and not the other.
The version we ask is about invention. Did it make that up? Is it hallucinating? Can it show me where that came from? Should I have given it a document to work from? Those questions all attack a single failure, a model producing something out of nothing, and they mostly work. The standard fix is to point it at a real source and make it cite the thing.
The version we skip is whether the source it cited still describes the world. That failure looks nothing like hallucination. It arrives sourced, linked and correct-looking, exactly the way 3.6 roentgen arrived correct-looking.
Think about whatever you pointed an AI at this week. An internal wiki, a policy page, a schema doc, a design file, a runbook someone wrote in a hurry. When did anyone last confirm it still matches the thing it names? For most of them the honest answer is that nobody has checked since it was written, and nothing about the file is going to tell you that.
Simon Willison, co-creator of the Django web framework, talking to Lenny Rachitsky on Lenny’s Podcast earlier this year, described the general shape of it: a model can produce a beautifully formatted vulnerability report, and “it’s not actually verified as being a real problem.” He was talking about open-source maintainers being sent AI-generated security reports by people who never checked them. The reports look like the real thing. Someone still has to read every one closely enough to find out.
This article is about that second failure: why a citation proves less than it looks like it proves, and the one extra step that turns a sourced answer into a checked one. I’ll work through it using a single case, because that one case contains every part of the problem.
A source can be cited and still be wrong
In February 2024 the British Columbia Civil Resolution Tribunal decided a small claim that belongs in every AI rollout deck: Moffatt v. Air Canada, 2024 BCCRT 149.
Jake Moffatt was flying to his grandmother’s funeral. He asked Air Canada’s website chatbot about bereavement fares and was told he could book straight away and apply for the reduced rate within 90 days. He booked. The airline then refused the refund, because its published policy says bereavement rates cannot be claimed once travel is complete.
Here is the part that turns it from a bad-chatbot story into something worth studying. The bot’s answer linked to Air Canada’s own bereavement policy page, and that page said the opposite of what the bot had just told him.
It named a source and then contradicted it.
Moffatt had no realistic way of catching that, because an answer that cites a policy page looks exactly like an answer that agrees with one. The dosimeter at least had a physical excuse, a hard ceiling in the hardware. The chatbot had none. The correct information was one click away, on the very page it had linked to.
Air Canada’s defence was that the chatbot was “a separate legal entity responsible for its own actions.” The tribunal rejected that, holding the airline responsible for information on its own website no matter which component produced it.
Keep Moffatt in mind, because every idea below is easier to see through his booking.
How often documents fall out of date
If Air Canada were a one-off, this would be a story about one badly built bot.
A study accepted at ICSE 2025 tested seven models across roughly 28,000 prompts, on Python libraries whose APIs had changed. Depending on the model, between 25 and 38 percent of the code they produced called deprecated APIs. The models were not inventing functions. They were faithfully reporting a world that had moved.
The second study is harder to shrug off, because it looked at the files we write by hand specifically to stop this from happening. Across 356 repositories containing files like CLAUDE.md and AGENTS.md, references to code that no longer existed turned up in 23 percent of them. Those files have exactly one job, which is grounding the agent in reality, and roughly one in four had drifted from the code they describe.
It happens to organisations with every incentive to prevent it. New York City’s official MyCity business chatbot, grounded in the city’s own published material, told business owners they could take a cut of their workers’ tips. The city left it running for weeks after reporters showed what it was doing.
Line those four up and the useful pattern is not that AI gets things wrong. It’s that every one of these systems was already doing the thing we tell people to do. New York grounded its bot in official material. Air Canada’s bot linked the exact page that contradicted it. The models in the ICSE study called functions that genuinely existed at the time their training data was assembled. Nobody skipped the sourcing step. The sourcing step is what produced the confidence.
The two studies that put a number on it landed between a quarter and a third. At that rate, a stale reference is not bad luck you can plan around. It’s the ordinary condition of any document nobody is actively maintaining, which is most documents at most companies.
And notice who caught each one. A research team. A newsroom. A man who had just buried his grandmother. In all four cases the check came from outside the system, because none of these systems had a check inside it. That’s the gap worth closing, and closing it starts with being precise about what “checked” actually means.
The three levels of trust
Most of us describe all of this as “checking our work,” a phrase quietly doing three separate jobs. There are three levels an answer can sit at, and treating them as one is what lets a bad answer through.
Unsourced. No reference at all. The model filled the gap with whatever pattern fit best, and the result is plausible, internally consistent, and built on nothing.
Sourced. The answer came from a specific place you named: a file, a table, a policy page. Naming one genuinely fixes the first problem and fixes nothing else.
Verified. Someone confirmed the named source still matches the thing it claims to describe. A source earns this only at the moment it is checked, and starts decaying again immediately afterwards.
Moffatt’s answer was sitting on level two, and it had every visible marker of level three. It cited a real page. The page existed. The link worked. What nobody had done was open that page and compare it to what the bot was saying.
Most careful people operate at level two and believe they are at level three. The skeptical user is the one who knows those are different levels and refuses to skip one. That isn’t cynicism, and it isn’t distrust of AI. It’s just knowing that a carefully written document can still be wrong today, because the thing it describes kept changing and the document didn’t.
Pair every document with the thing it describes
The fix fits in one sentence. Don’t just give the AI the document. Give it the document and the real thing that document describes.
There are almost always two things you could point at. A design doc, and the interface it describes. A schema doc, and the table itself. A policy page, and the policy as it actually works today. Most of us point at the description, because that’s the one that is easy to find and pleasant to read.
Point at both, then add one rule: when they disagree, the live thing wins. That rule is doing the real work here. A running system cannot be out of date, because it is what your customers are using right now. A document is only somebody’s account of that system, written at some point in the past, and correct for exactly as long as nobody changed anything.
Air Canada shows how little effort this would have taken. The bot had already linked the bereavement page, so the correct policy was sitting one click away from the answer it gave. Anything that compared those two would have found a flat contradiction in seconds. Nobody was comparing.
In practice:
A design document. Open the file and the running interface in the same pass, not one or the other.
A schema doc. Query the real table alongside reading the doc.
An API contract. Hit the actual endpoint next to reading the spec.
A policy page or process doc. Check it against how the process ran last time, rather than how it was written to run.
One instruction matters more than the rest. Tell the agent a mismatch should be reported, not resolved. An agent that quietly picks a winner has buried the most valuable thing it found. In Moffatt’s case the disagreement between the bot and the policy page was the answer, and it reached nobody.
Why a checked answer looks the same as a lucky guess
Moffatt’s case has one limit as a lesson: the mistake came to light. He was refused his refund, he pushed back, a tribunal looked at it, and everyone ended up knowing the bot had been wrong.
Most of the time, nothing comes to light. You get an answer, you use it, and nobody ever tells you whether it was right. That version is worth looking at too, because it is far more common and much harder to notice.
I was making a seven-slide carousel and asked my coding agent to write the image prompts. It came back with a full colour scheme I had never asked for: a background colour, a text colour, four accent colours, and a line in every prompt telling itself to avoid one particular colour.
It had not looked anything up to get there. So I told it to stop guessing, read the design document, and then go and check what colours the live product was actually using.
It did both, and the two did not match. The design document named a main colour the product had stopped using months earlier.
If it had stopped at the design document, which is where I sent it, I would have shipped seven prompts using a colour the brand had dropped. And I would have felt careful doing it, because I asked it to check a source and it checked one. That is the same spot Moffatt was in, without the funeral or the tribunal.
Here is the part I still think about. The checked colours came back almost identical to the ones the agent had made up at the start, when it had nothing to go on at all. The guess and the verified answer were nearly the same, by luck.
You cannot tell, by reading an answer, whether anyone checked it. A lucky guess and a verified answer look the same from outside. If I had shipped that first version, nothing about it would have looked wrong to me, and I would have carried on trusting a way of working that had simply got lucky once.
Pointing at a document stops the model from making things up. It does nothing about whether that document is still true. Those are two different jobs, and most of us only ever do the first one.
The second job cannot be done at the end, by reading the answer more carefully. By then the evidence you would need is already gone. It has to happen in the asking:
Here is the document. Here is the real thing it describes. Tell me where they disagree.
One extra line in a request you were going to send anyway. That line is the whole difference between an answer that cites a source and an answer somebody actually checked.
Air Canada had the page. The control room had the reading. I had the design file. In all three, the right information was already sitting within reach of the person who needed it, and nobody asked the question that would have surfaced it.