4 Comments
User's avatar
Selim Yoruk's avatar

I’d keep a small set of untouched pages in the report too. If their visibility moves with the pages you changed, the content team may be getting credit for a model update.

AISO Tools's avatar

Agree on intervals, and the same applies to zeros. We published a study last month that came out at 0 hits: 1,339 newly launched AI tools, 6,710 blind "best AI tool for..." questions to gpt-4o-mini, and none of the tools was named. The honest way to report that is an upper bound rather than "never": exact 95% CI 0 to 0.28%. When we re-ran a sample of 125 tools with web search required, we only had 625 answers, so the same zero widens to 0 to 2.91%. Same headline, very different claim.

On "a citation isn't a visit", the source/citation gap was the biggest thing in that search run. The model retrieved pages from 2,703 domains but cited 687. A Reddit thread was retrieved for 595 of the 625 answers and cited in 8. So a tracker that reports only citations and one that reports retrievals will tell you two different stories about the same answers.

Method, intervals and CSVs (disclosure: it's our report, and we sell monitoring for this): https://aisotools.com/research/chatgpt-ai-tool-visibility-2026?utm_source=substack&utm_medium=comment&utm_campaign=dist-20261103-substack-plgeek&ref=dist-20261103-substack-plgeek

Who AI Picks's avatar

Agreed. The screenshots that get shared are rarely from the bad days.

Mohamed F. Ahmed's avatar

This mirrors something I ran into building AI-native products: the moment a metric becomes the thing a team gets judged on, everyone stops questioning its methodology. AI search visibility is heading the same way SEO rank tracking did a decade ago, where the tool's sampling quirks quietly became the KPI. Have you found marketers are more skeptical of these numbers than they were of early SEO tools, or is it the same blind trust all over again?