How to Check Whether an AI Assistant Made Up the Numbers in Its Answer
The failure mode nobody notices in time Ask an assistant to summarise a CI pipeline and you get a tidy paragraph: the lint stage took 42 seconds, the build 3.5 minutes, the whole pipeline 16.4 minutes, 18% faster than last week. It reads like a report. Two hours later someone notices the build job never took 3.5 minutes, and the failure rate was copied from a different week. Nothing about the paragraph looked suspicious. That is the entire problem: a fabricated number and a real number are typographically identical. ...