AI
Three Confident Answers, All Wrong
By Khai Tran · · 2 min read
My monthly report gave me three clean conclusions, and each one was wrong for a different reason. The same week, Red made a confident mistake of its own.
Week ending August 7. Written up from the build log.
My monthly report came in on August 1 with three findings. Zero downloads from the blog. My visibility in AI search, flat at zero. The newsletter, looking broken.
All three were wrong, each in a different way.
The downloads read zero because the report was counting the wrong field. The blog's download forms record which guide someone took in one place, and the report looked in another. For those forms, it could only ever say zero.
Search visibility looked flat because the scan had checked one AI engine instead of two. The one it skipped was the one where I was actually getting cited.
And the newsletter was fine. The report had built its conclusion on one bad send from the July email outage, and every send since had gone through.
We caught all three on Wednesday. Each had read like a finding, confident enough to change what I planned to write next. A number that reads zero because the query is wrong looks exactly like a number that reads zero because nothing happened.
So before Red calls anything flat, zero or broken now, it has to confirm the measuring actually happened. If part of a scan gets skipped, the report says so at the top.
On Thursday, Red made a confident mistake of its own. The week before, we had changed my site so automatic updates wait for every check to pass. Red decided one update needed a manual push, and it published that update by hand, around the gate it had helped build. It was the same change the gate was about to ship, so nothing different went out.
The note Red left itself afterward ends with three words: never repeat it.