The failure mode nobody notices in time
Ask an assistant to summarise a CI pipeline and you get a tidy paragraph: the lint stage took 42 seconds, the build 3.5 minutes, the whole pipeline 16.4 minutes, 18% faster than last week. It reads like a report. Two hours later someone notices the build job never took 3.5 minutes, and the failure rate was copied from a different week.
Nothing about the paragraph looked suspicious. That is the entire problem: a fabricated number and a real number are typographically identical.
Why models do this
An assistant writing a summary is not looking up rows in your export. It is producing the most plausible next tokens, and plausible is not the same as sourced. Two failure patterns dominate:
- Fabrication — a number invented because a number belongs there. “Failure rate 2.4%” appears because failure rates belong in pipeline summaries, not because 2.4% exists anywhere.
- Drift — a real number that changed on the way: rounded twice, converted from seconds to minutes badly, or lifted from a neighbouring row, week or project.
Both survive review, because review checks the prose, not the arithmetic.
A five-minute workflow that catches most of it
1. Keep the raw export, not the summary. Give the assistant the CSV, the log, the SQL result. If the source itself is already a summary, there is nothing left to verify against.
2. Treat every number as a claim with a required citation. The rule that makes this practical: no number in a report until you can point at the row it came from. This is the same discipline as a source citation in journalism, applied to arithmetic.
3. Check units before you check values. 1500 ms, 1.5 s and 0.025 min are the same duration. Most “wrong” numbers in AI summaries are unit or rounding artefacts, and most genuine fabrications hide behind that noise.
4. Keep a tolerance. A figure rounded to three significant digits in the summary will not match the raw 4183000 exactly. Allow a small relative tolerance (1% is a reasonable default) and treat anything outside it as unverified rather than wrong.
5. Separate “unverified” from “wrong”. An unverified number has no source in what you pasted. It might be right and simply come from data you did not include — but until you find that source, it cannot go into a report.
The AI Number Checker automates steps 3 to 5: paste the AI answer and the source, and it extracts every number, normalises units and multipliers, matches each one against the source, and flags what it cannot trace — with the source line as evidence.
What to do with a flagged number
Work through the flagged list in order of impact, and classify each one:
| Finding | Typical cause | Action |
|---|---|---|
| No match at all, plausible magnitude | Fabrication | Delete it or ask for the source |
| Off by a few percent | Rounding, different week, different scope | Confirm the reference period before publishing |
| Matches only after unit conversion | Summarisation artefact | Fine — keep the converted value, state the unit |
| Matches a nearby but different row | Row drift | Correct it and re-check the neighbours |
Where this approach still fails
Honesty matters more than the pitch: a checker verifies traceability, not truth.
- Correct numbers can support a wrong conclusion — the classifier still needs a human.
- If the source you pasted is stale, wrong or itself generated, every check passes.
- Screenshots and chat summaries work badly as sources, because the numbers you want may have been rounded away before you copied them.
- Numbers that exist only in a system you did not paste stay unverified, which is correct behaviour but not a verdict.
Use the checker as a fast first pass that removes the expensive failure mode — a fabricated figure that reaches a stakeholder — and keep judgement for the argument.
Make it a habit, not a rescue
Two practices make this cheap enough to sustain:
- Ask for the working, not just the answer. “Show the pipeline commands and raw numbers you used” produces a checkable answer most of the time.
- Verify before the meeting, not after. Verification is minutes; the correction email after a fabricated metric has shipped is hours.
The checker runs entirely in your browser — internal reports, CI logs and financial figures never leave your machine — and the JSON formatter is a useful stop on the way in when the source arrives as an API dump.