You measure AI visibility by asking each AI engine the same question many times, recording the rate at which your brand appears in the answers, and putting a confidence interval around that rate before you compare it to anything else.
I’m Robert Galasso, and I lead Innovation and AI at Tandem Interactive, a Fort Lauderdale agency, where I built AI visibility tracking that runs in production every day. This is the full methodology: why screenshots lie, how much sampling you need, and how to report numbers you’d stake a budget on.
What is AI visibility?
AI visibility is the rate at which AI assistants such as ChatGPT, Gemini, Perplexity, and Claude mention, recommend, or cite your brand when people ask questions your brand should win, measured across repeated samples of the same prompts on each engine. The number is a rate, because a chat answer has no position one to hold. What you can measure is how often you show up when the question gets asked.
Rank trackers could check a position once and move on, because the results page was mostly stable. An LLM’s answer is generated fresh every time, with randomness built in on purpose, so measuring it takes the tools you’d use for any noisy process, starting with repeated samples and error bars.
Why single screenshots lie
Ask ChatGPT for the best CRM for a small law firm, then ask again in a fresh session, and fairly often you’ll get a different list. Temperature adds randomness by design, retrieval pulls different sources from one run to the next, models get updated quietly, providers run tests, and session context bleeds into answers when you’re not careful.
That variance is the core problem with most brand mention tracking in ChatGPT and every other engine. A screenshot of one answer is an anecdote, and a weekly screenshot is an anecdote on a schedule. The failure I worry about is a client cutting spend because their brand “dropped out of ChatGPT,” when the truth is they landed on the wrong side of sampling variance twice in a row and the report did the damage.
Classic rank tracking had variance too (data centers, localization, personalization), but the industry got twenty years to build intuition for it. AI answer tracking is new enough that one confident screenshot still passes for evidence in a deck. The engines are random by design, so the measurement has to be statistical.
I train generative models on my own time, and nothing cures faith in a single output faster than watching one prompt produce a hundred different results. LLM answers work the same way, so treat every prompt as a distribution.
How many samples do you need?
Run Wilson score intervals on an observed 60% mention rate and watch what sample size buys you:
- 3 hits in 5 runs: the 95% interval spans roughly 23% to 88%. You know almost nothing.
- 12 hits in 20 runs: roughly 39% to 78%. Now you can catch big moves.
- 30 hits in 50 runs: roughly 46% to 72%. Tight enough for competitive comparisons.
Five samples per prompt per engine will catch a large move but will not measure a small one, and it is still worth running, since a wide interval at least shows you how little you know.
Every query costs tokens. Moderate samples across many prompts beat huge samples on three vanity prompts, because you can pool related prompts whenever you need more sensitivity, and if one prompt carries a real business decision, buy extra samples for that prompt specifically.
How to measure AI visibility, step by step
This is the process I run in production, and none of it requires custom software; a spreadsheet and some discipline get you 80% of the way.
- Build the prompt set from real buyer questions. Pull from sales calls, support tickets, People Also Ask, and your GA4 site-search queries. Skip prompts nobody types.
- Freeze the definition of a hit before you run anything. Mentioned by name? Actively recommended? Cited as a source? Write the rule down. Changing it mid-stream poisons the trend.
- Pick your engines. ChatGPT, Gemini, Perplexity, Claude, and Google AI Overviews if they matter in your vertical. Track them separately, because they don’t move together.
- Sample every prompt multiple times per engine, per run. Fresh session every time. No memory, no logged-in personalization, no follow-up questions.
- Record hits over trials, and keep the raw answers. You’ll want receipts the first time someone challenges a number.
- Compute the mention rate and a Wilson confidence interval for every rate you plan to show another human being.
- Compare intervals, never points. Across time, against competitors, and between engines.
- Alert only when intervals separate. Overlap means “not a story yet,” and the report should say exactly that in plain words.
Most tracking falls apart at step 8. An alert fires only when two intervals genuinely separate, and everything short of that gets logged as a watch item.
A worked example with the math shown
These numbers are illustrative and I made them up to show the mechanics, so run your own before you trust anything here.
Say you track one prompt on one engine, 20 fresh runs per month.
- Month one: your brand appears in 12 of 20 runs. That’s 60%, with a Wilson 95% interval of roughly 39% to 78%.
- Month two: 8 of 20. That’s 40%, interval roughly 22% to 61%.
The point estimate dropped 20 points and everybody wants to panic. But the intervals overlap between 39% and 61%, and a brand with a stable 50% true rate produces both months without breaking a sweat, so this one gets logged as a watch item.
- Month three: 3 of 20. That’s 15%, interval roughly 5% to 36%.
Now the whole interval sits below month one’s 39% floor. This time the intervals separated, so the alert fires, and you can defend the call to anyone who asks. Month two was “possible slide, watching” and month three was “confirmed, here’s the evidence.”
When a brand slips across 30 prompts at once, pool them. Aggregated trials tighten the interval and catch broad moves far earlier than any single prompt can.
What a real AI visibility report looks like
The format is plain: every headline number carries its range. A rate of 54% on 100 trials reads as 54% (44% to 63%), which tells the reader what we know and how firmly we know it. Alerts appear only when intervals separate, so when one shows up, people act instead of debating whether it’s real.
The uncomfortable feature is a status that says “we can’t tell yet.” Clients don’t love it at first, and it pays for itself the first time a scare turns out to be sampling noise.
I shipped this in Nexus, the platform I built for my team at Tandem Interactive. Every prompt gets sampled multiple times per engine, every rate carries a Wilson interval, and drop alerts stay quiet until the intervals separate. If you’re evaluating AI search visibility tracking tools instead of building, demand two things from every vendor: the sample count behind each number, and the interval around it. No answer to those questions means the number is decoration.
The strictness comes from years of owning revenue numbers. Budgets move on the strength of a slide, so when the slide is noise, real money follows the noise. The same rules run through everything on my capabilities page.
FAQ
How is AI visibility different from SEO rank tracking?
Rank tracking measures a position on a mostly stable results page, so a single check was defensible. LLM brand tracking measures a probability: how often a model chooses to include you in generated text. Positions get spot-checked, while probabilities have to be sampled.
How many times should you sample each prompt?
Enough that the interval can answer your question. Five runs per engine flags only huge moves, twenty catches meaningful shifts, and pooling related prompts buys sensitivity without more spend. When the interval is too wide to support the decision, the fix is more samples.
What counts as a brand mention in ChatGPT?
Whatever you defined before the run: named in the answer, actively recommended, or cited as a linked source. I track those as separate rates because they mean different things commercially. The exact definition matters less than freezing it, since a moving definition makes every trend unreadable.
Why use Wilson confidence intervals instead of the normal approximation?
The normal approximation misbehaves at small samples and extreme rates. At 0% or 100% it produces a zero-width interval, which claims perfect certainty from a handful of runs. Wilson behaves at the sample sizes token budgets allow, and any stats library or spreadsheet can compute it.
How often should you run AI visibility tracking?
Weekly or monthly both work, as long as each run carries enough samples to mean something. A thin daily run just reports noise on a faster schedule. Match the cadence to how fast you’d genuinely act on a confirmed change.