You ask for something. A summary, a draft, a plan for the week.
It comes back complete. Nicely formatted, confident, done.
And somewhere inside it is a detail you never gave it.
Maybe a date. Maybe a number. Maybe it picked one of two systems to treat as the real one, and picked wrong. You do not notice, because nothing in the output looks different from the parts that were built on what you actually said.
This happens constantly, and most people read it as the AI making things up.
It is doing something more specific than that. You left a blank. It filled the blank. Then it carried on as if the blank had never been there.
The interesting part is that nobody told it what else to do.
Think about the last real task you handed to an AI. A report from three documents. A spreadsheet to reconcile. A set of notes to turn into something someone else can act on.
Somewhere in that request was a fact you did not supply. You know the one.
You did not write a rule for that moment. Something still decided what it meant.
This post is about writing that rule. It takes one sentence, and it changes what comes back.
Your AI is built to answer, not to ask
There is a measurement for this, and it comes from OpenAI rather than from a critic.
They took questions about an image, then removed the image. Now the model has a question it cannot possibly answer.
In that setup, their o3 model gave a confident answer 87 percent of the time. It described a picture that was not there. It did not say the picture was missing.
OpenAI’s own words for the category: in tasks where the model is missing an important input, earlier models “would hallucinate answers or claim to have solved the underspecified task.”
That is one number and it is the only one in this post. I am not going to bury you in studies. But it is worth holding, because it is much higher than most people assume.
Anthropic documents the same behaviour in its own instructions for developers. Ask about the weather with no location, and the model may simply supply one, filling in New York as though you had said it.
None of this is a defect somebody forgot to fix. It is what these systems are shaped to do. They are scored on answering, and an answer that admits a gap scores the same as no answer at all.
So the model that guesses wins. That is the model you are using.
Here is the part that surprises people. Giving it more material often makes this worse, not better.
You would expect that handing it a folder of documents would make it more willing to say your answer is not in them. The opposite happens. Any context at all raises its confidence, so it reaches for something plausible instead of telling you the folder came up short.
There are two kinds of missing, and they need different answers
Before the fix, one distinction. It sounds like hair-splitting and it is not, because each kind has a different repair.
Ambiguous means your request could be read two ways. You said “last quarter” and there are two calendars in play. There is a small set of possible meanings, and one question settles it.
Incomplete means the thing simply is not there. You never said which number to use. No multiple-choice question will fix that, because there is nothing to choose between.
The test is one line. If more information would fix it, asking works. If nothing anyone tells it would fix it, asking is theatre and you need a decision rule instead.
A simple way to think about it: ambiguous is a fork in the road, incomplete is a missing road.
Write the rule before you write the request
The usual advice is to tell it that it can say it does not know.
That advice is fine and it is not enough. “Say so if you are unsure” leaves the machine to decide what unsure means, and it decides generously.
Two things make it work.
First, put a number in it. Not an adjective.
Answer only if you are more than 80 percent confident. If you are below that, stop and tell me what you need.
A threshold is checkable. “Be careful” is not. And the number is yours to move: a first draft can run at 50, anything that sends an email or deletes a file should run far lower.
Second, say what happens when the bar is not met. There are four answers, and picking the right one is most of the work.
Stop and wait. The job halts until you supply the value. Use this when the mistake is expensive to undo.
Carry on, but flag it. It proceeds on a stated assumption and tells you which one. Use this when you would rather have a draft with a question attached than no draft.
Use a default you agreed in advance, and mark it. The value is pre-decided. The output shows visibly where it was applied.
Refuse. It does not produce anything. Use this when a partial answer would mislead rather than merely disappoint.
Most people only ever use the first one, and only by accident.
Anthropic’s documentation shows a nice version of the third. It tells the model to strip any claim it cannot support and mark the hole with empty brackets. The gap ships, visibly, instead of being smoothed over.
Worth knowing that the defaults are actively working against you here. OpenAI’s own guide for developers building agents recommends this instruction: “Never stop or hand back to the user when you encounter uncertainty”, and “do not ask the human to confirm or clarify assumptions.”
That is not a mistake either. Asking is slow, and slow demos badly. But it means if you want a system that stops, you have to say so, because the thing was built to keep going.
How this works on a job you already do
Take something ordinary. You want a weekly summary pulled from your notes.
Say your notes live in Notion or Obsidian, and you ask Claude to go through the week and tell you what moved, what stalled, and what needs a decision.
Now look for the blanks before you send it.
Which week? If you wrote this on a Monday, does the week mean the last seven days or the one that just ended? That is ambiguous. There are two readings and a question settles it.
What counts as stalled? You have a feel for this. You have never written it down. That is incomplete. No question will fill it, because you have not decided yourself.
Which notes are the real ones? If you keep a scratch file and a proper one, you know which wins. It does not.
Three blanks, in the most boring task on your list. Without a rule, all three get filled silently and the summary looks fine.
So write the rule once and keep it with the request:
Use the last full calendar week. If anything is unclear, ask before you start rather than guessing. If a note looks like a draft rather than a decision, leave it out and list what you skipped at the end.
Read what that does. The first sentence removes one ambiguity outright. The second sets the behaviour for anything else ambiguous. The third gives a documented default for the incomplete case, plus a visible marker so you can see what it dropped.
None of that is clever. It took three sentences and no tooling.
What it does not do is make the summary correct. It makes the gaps visible while you can still act on them, which is a smaller claim and a more useful one.
Try it once on a task you repeat. Then look at what it asked you. The questions are usually the interesting bit, because they are the decisions you had been making by accident.
When telling it to ask makes things worse
I would be selling you something if I stopped here.
“Mayank, so I should just tell it to be cautious about everything?”
No. That backfires, and it backfires in a way you can measure.
Tell a model broadly that it may decline, and it starts declining work it could have done perfectly well. On one benchmark, adding a general permission to say “I don’t know” dropped accuracy from 73.7 percent to 64.7 percent. It got more careful and less useful at the same time.
Add a general “be careful” instruction and the same thing happens with refusals. Systems start refusing ordinary requests because the request pattern-matches something risky.
So the goal is not maximum caution. The goal is a threshold that matches what the task costs when it is wrong.
A weekly summary being slightly off costs you two minutes. Something that sends a message on your behalf costs more. Those two should not carry the same rule, and if you write one rule for everything you will either get a system that never stops or a system that never finishes.
One more limit worth knowing. When you ask a model to flag its assumptions, what you get back is a self-report, and self-reports are not reliable. Anthropic tested whether models mention the thing that actually changed their answer, and found they often do not.
So treat a stated-assumptions list as a courtesy, not an audit. Prefer rules you can check from the outside, like “list what you skipped,” over narration you have to trust.
Try this on your next request
Pick the task you hand over most often. The repeated one, the boring one.
Before you send it, read your own request and find the blanks. There will be two or three. There always are.
For each one, decide which of the four it gets. Stop and wait. Carry on and flag. Documented default with a marker. Or refuse.
Then write it as one or two plain sentences and keep them with the request. Not a document. Not a system. A couple of lines you paste.
Run it and see what comes back as a question instead of an answer.
That output you got this morning, the complete one, the one that looked finished. Somewhere in it there is a detail you never supplied, and right now nothing is set up to tell you which one.
What is the first blank you are going to name?