Do AI Agents Make Things Up? How a Business Owner Catches It
Yes. AI agents still invent facts, numbers, and sources sometimes, and confident wrong answers look exactly like right ones. You catch them without any coding through three habits: demand a source in every report, check the agent's numbers against your own metrics file, and read the git diff before you approve a change.
The unsettling part is not that an agent guesses. It is how good the guess sounds. A made-up statistic arrives in the same calm, tidy sentence as a real one, so you cannot spot the difference by reading harder or trusting your gut. What actually works is not smarter reading, it is a small routine that forces every claim to point at something you can open and see for yourself.
A hallucination is a confident, well-formed statement an AI agent presents as fact when it has no real source for it - an invented citation, a plausible number, a quote nobody said. The fix is not to make the agent smarter. It is to make every claim it hands you traceable back to something real.
Where do agents most often invent things?
In the gaps, and always in a few predictable places. An agent makes things up most when it is asked for something specific it does not actually have: an exact figure, a citation, a quote, a fact from a document it never opened. The pattern is consistent enough that you can watch those four spots directly instead of policing everything.
- Numbers and statistics. Ask for a market size, a conversion rate, or "how many users typically," and an agent will often produce a clean, specific-sounding figure with no real basis. Round, confident numbers with no source attached are the single most common invention.
- Citations and links. Agents will name a study, a report, or a URL that looks completely real and does not exist, or exists and says something else. A citation you cannot click and verify is not evidence, it is decoration.
- Quotes and attributions. "As the CEO said..." or "according to their documentation..." can be entirely fabricated, worded to fit the point the agent is making rather than anything that was actually said.
- Its own past work. Because agents forget between sessions, an agent asked what it did last week may reconstruct a plausible story rather than report the truth. Written records beat asking the agent to remember.
Notice the common thread: invention happens where the agent lacks a source but still wants to be helpful. That is good news, because it means you do not have to distrust everything. You have to distrust the four specific things above, and demand a source exactly there.
What is the 3-check verification habit?
Three checks, and none of them require you to read code or understand the model. We run them on every piece of work an agent hands us, and together they take a couple of minutes. Each one attacks a different kind of invention.
Check 1: a source in every report. Make it a standing rule, written into the agent's instructions, that any claim of fact carries a link or a filename you can open. "Competitors charge around $30" is not acceptable; "Competitor X lists $29 on their pricing page (link)" is. When the report has no source next to a number, that number is the first thing you verify, because a missing source is the tell. This one rule converts most hallucinations into something you can catch by clicking.
Check 2: numbers against your metrics file. Never let the agent be the only witness to its own results. Keep the real numbers - sales, traffic, sign-ups - in a plain file the agent updates from the actual source, and when the agent claims "traffic is up," open that file and look. If the agent says fourteen sales and the file says nine, you found an invention in ten seconds. The agent reports; the file is the record; you compare the two.
Check 3: read the git diff before you approve. When the agent changes files, it works in git, and a git diff is a plain before-and-after list of exactly what changed - readable in English, no programming needed. Before you accept the work, you skim that list. It cannot be talked around: the agent can describe its changes however it likes in chat, but the diff shows what it actually did. If the summary and the diff disagree, the diff wins. This is the check that catches an agent that reports one thing and did another.
These three share a design: each replaces trust with a receipt. You are never grading how honest the agent sounds. You are opening the thing it pointed at - the source, the file, the diff - and seeing whether the claim survives contact with reality.
Do you need to be technical to catch this?
No. Every one of the three checks is a reading task, not a coding task. A source is a link you click. A metrics file is a short list of numbers you compare. A git diff, despite the name, is a plain list of what changed that you skim like a receipt. The agent does the technical work of keeping the file and running git; you do the judging, which is the split that keeps a non-technical owner in control. Your job is to refuse to take any number, source, or claim on the agent's word alone.
FAQ
Have newer AI models stopped making things up?
They invent less, not zero. Newer models are noticeably more reliable and hedge more honestly when unsure, but they still produce confident wrong answers, especially on exact numbers, citations, and anything niche or recent. Treating "the model is better now" as a reason to skip verification is how a plausible invention slips into a real decision. Assume less hallucination, never none, and keep the checks regardless of which model you run.
How do I make an agent cite its sources?
Write it into the agent's standing instructions as a rule, not a one-time request: every factual claim must carry a source you can open - a link or a filename - and anything it cannot source must be labeled as an estimate or guess. Because agents forget between sessions, a rule typed into chat lasts one conversation; the same rule in the contract file is re-read at every startup and actually sticks. Then enforce it by treating any unsourced number as unverified until you check it.
What about the numbers in an agent's reports?
Never take a metric on the agent's word. Keep your real numbers - sales, traffic, sign-ups - in a plain file updated from the actual source, and compare the agent's claim against that file rather than against your memory. If the report says a figure the file does not, the file wins and the agent gets corrected. The agent is a reporter of numbers, not the source of record, and keeping those two roles separate is what stops a made-up figure from steering a decision.