You measure AI visibility by asking each AI engine the same question many times, recording the rate at which your brand appears in the answers, and putting a confidence interval around that rate before you compare it to anything else. One answer is an anecdote - the mention rate, with its interval, is the measurement.
I’m Robert Galasso, and I lead Innovation and AI at Tandem Interactive, a Fort Lauderdale agency, where I built AI visibility tracking that runs in production every day. This is the full methodology: why screenshots lie, how much sampling you actually need, and how to report numbers you’d stake a budget on.
What is AI visibility?
AI visibility is the rate at which AI assistants such as ChatGPT, Gemini, Perplexity, and Claude mention, recommend, or cite your brand when people ask questions your brand should win, measured across repeated samples of the same prompts on each engine. What you are measuring is a rate rather than a rank, because a chat answer has no position one to hold and the only meaningful number is how often you show up when the question gets asked.
That reframe changes the whole shape of the job. Rank trackers could check a position once and move on, because the results page was mostly stable. An LLM’s answer is generated fresh every time, with randomness built in on purpose. Measuring it takes the tools you’d use for any noisy process, starting with repeated samples and honest error bars.
Why single screenshots lie
Ask ChatGPT for the best CRM for a small law firm, then ask again in a fresh session, and fairly often you’ll get a different list. Temperature adds randomness by design, retrieval pulls different sources from one run to the next, models get updated quietly, providers run tests, and session context bleeds into answers when you’re not careful.
That variance is the core problem with most brand mention tracking in ChatGPT and every other engine. A screenshot of one answer is an anecdote, and a weekly screenshot is an anecdote on a schedule. The failure mode I actually worry about is a client cutting spend because their brand “dropped out of ChatGPT,” when the truth is they landed on the wrong side of sampling variance twice in a row. The loss came from the report, not from the model.
Classic rank tracking had variance too (data centers, localization, personalization), but the industry got twenty years to build intuition for it. AI answer tracking is new enough that one confident screenshot still passes for evidence in a deck. The engines are random by design, so the measurement has to be statistical.
I train generative models on my own time, and nothing cures faith in a single output faster than watching one prompt produce a hundred different results. LLM answers work the same way, so treat every prompt as a distribution, because that is exactly what it is.
How many samples do you actually need?
More than one, fewer than infinity, and the math tells you exactly where you sit. Run Wilson score intervals on an observed 60% mention rate and watch what sample size buys you:
- 3 hits in 5 runs: the 95% interval spans roughly 23% to 88%. You know almost nothing.
- 12 hits in 20 runs: roughly 39% to 78%. Now you can catch big moves.
- 30 hits in 50 runs: roughly 46% to 72%. Tight enough for competitive comparisons.
Five samples per prompt per engine will catch a large move but will not measure a small one, and it is still worth running, because a wide interval is at least honest about how little you know.
Every query costs tokens, so budget like a portfolio manager. Moderate samples across many prompts beat huge samples on three vanity prompts, because you can pool related prompts whenever you need more sensitivity. If one prompt carries a real business decision, buy extra samples for that prompt specifically.
How to measure AI visibility, step by step
This is the process I run in production, and none of it requires custom software; a spreadsheet and some discipline get you 80% of the way.
- Build the prompt set from real buyer questions. Pull from sales calls, support tickets, People Also Ask, and your GA4 site-search queries. Skip prompts nobody actually types.
- Freeze the definition of a hit before you run anything. Mentioned by name? Actively recommended? Cited as a source? Write the rule down. Changing it mid-stream poisons the trend.
- Pick your engines. ChatGPT, Gemini, Perplexity, Claude, and Google AI Overviews if they matter in your vertical. Track them separately, because they don’t move together.
- Sample every prompt multiple times per engine, per run. Fresh session every time. No memory, no logged-in personalization, no follow-up questions.
- Record hits over trials, and keep the raw answers. You’ll want receipts the first time someone challenges a number.
- Compute the mention rate and a Wilson confidence interval for every rate you plan to show another human being.
- Compare intervals, never points. Across time, against competitors, and between engines.
- Alert only when intervals separate. Overlap means “not a story yet,” and the report should say exactly that in plain words.
Step 8 is where most tracking falls apart, so it gets its own rule: an alert fires only when two intervals genuinely separate, and everything short of that gets logged as a watch item instead.
A worked example with the math shown
These numbers are illustrative and I made them up to show the mechanics, so run your own before you trust anything here.
Say you track one prompt on one engine, 20 fresh runs per month.
- Month one: your brand appears in 12 of 20 runs. That’s 60%, with a Wilson 95% interval of roughly 39% to 78%.
- Month two: 8 of 20. That’s 40%, interval roughly 22% to 61%.
The point estimate dropped 20 points and everybody wants to panic. But the intervals overlap between 39% and 61%, and a brand with a stable 50% true rate produces both months without breaking a sweat, so this one gets logged as a watch item rather than an alert.
- Month three: 3 of 20. That’s 15%, interval roughly 5% to 36%.
Now the whole interval sits below month one’s 39% floor. The intervals separated, the alert fires, and you can defend the call to anyone who asks - month two was “possible slide, watching” and month three was “confirmed, here’s the evidence.”
One more trick: when a brand slips across 30 prompts at once, pool them. Aggregated trials tighten the interval and catch broad moves far earlier than any single prompt can.
What a real AI visibility report looks like
The format is plain: every headline number carries its range. A rate of 54% on 100 trials reads as 54% (44% to 63%), which tells the reader what we know and how firmly we know it. Alerts appear only when intervals separate, so when one shows up, people act instead of debating whether it’s real.
The uncomfortable feature is a status that says “we can’t tell yet.” Clients don’t love it at first. They learn to, because a measurement system willing to say we can’t tell yet is the one you can trust when it says something moved.
I shipped this in Nexus, the platform I built for my team at Tandem Interactive. Every prompt gets sampled multiple times per engine, every rate carries a Wilson interval, and drop alerts hold their tongue until the intervals separate. If you’re evaluating AI search visibility tracking tools instead of building, demand two things from every vendor: the sample count behind each number, and the interval around it. No answer to those questions means the number is decoration.
The strictness comes from years of owning revenue numbers. Budgets move on the strength of a slide, so when the slide is noise, real money follows the noise. The same discipline runs through everything on my capabilities page, because a channel you cannot measure honestly is spend you cannot defend.
FAQ
How is AI visibility different from SEO rank tracking?
Rank tracking measures a position on a mostly stable results page, so a single check was defensible. LLM brand tracking measures a probability: how often a model chooses to include you in generated text. Positions get spot-checked, while probabilities have to be sampled.
How many times should you sample each prompt?
Enough samples that the interval can actually answer your question. Five runs per engine flags only huge moves, twenty catches meaningful shifts, and pooling related prompts buys sensitivity without more spend. When the interval is too wide to support the decision, the fix is more samples.
What counts as a brand mention in ChatGPT?
Whatever you defined before the run: named in the answer, actively recommended, or cited as a linked source. I track those as separate rates because they mean different things commercially. The exact definition matters less than freezing it, since a moving definition makes every trend unreadable.
Why use Wilson confidence intervals instead of the normal approximation?
The normal approximation misbehaves at small samples and extreme rates. At 0% or 100% it produces a zero-width interval, which claims perfect certainty from a handful of runs. Wilson stays honest at the sample sizes token budgets actually allow, and any stats library or spreadsheet can compute it.
How often should you run AI visibility tracking?
Weekly or monthly both work, as long as each run carries enough samples to mean something. A thin daily run just reports noise on a faster schedule. Match the cadence to how fast you’d genuinely act on a confirmed change.