You measure AI visibility by asking each AI engine the same question repeatedly, recording the rate at which your brand appears in the answers, and putting a confidence interval around that rate before you compare it to anything else.

I am Robert Galasso, and I lead Innovation and AI at Tandem Interactive, a Fort Lauderdale agency, where I built AI visibility tracking that runs in production every day. This is the full method: why screenshots mislead, how much sampling you need, and how to report numbers you would stake a budget on.

What is AI visibility?

AI visibility is the rate at which AI assistants such as ChatGPT, Gemini, Perplexity, and Claude mention, recommend, or cite your brand when people ask questions your brand should win, measured across repeated samples of the same prompts on each engine. The number is a rate, because a chat answer has no position one to hold. What you can measure is how often you show up when the question gets asked.

Rank trackers could check a position once and move on, because the results page was mostly stable. An LLM’s answer is generated fresh every time, with randomness built in on purpose, so measuring it takes the tools you would use for any noisy process, starting with repeated samples and error bars.

Why single screenshots lie

Ask ChatGPT for the best CRM for a small law firm, then ask again in a fresh session, and fairly often you will get a different list. Temperature adds randomness by design, retrieval pulls different sources from one run to the next, models get updated quietly, providers run tests, and session context bleeds into answers when you are not careful.

That variance is the core problem with most brand mention tracking in ChatGPT and every other engine. A screenshot of one answer is an anecdote, and a weekly screenshot is an anecdote on a schedule. The failure I worry about is a client cutting spend because their brand “dropped out of ChatGPT” when they landed on the wrong side of sampling variance twice in a row.

Classic rank tracking had variance too (data centers, localization, personalization), but the industry got twenty years to build intuition for it. AI answer tracking is new enough that one confident screenshot still passes for evidence in a deck. The engines are random by design, so the measurement has to be statistical.

I train generative models on my own time, and nothing cures faith in a single output faster than watching one prompt produce a hundred different results. LLM answers work the same way, so treat every prompt as a distribution.

How many samples do you need?

Run Wilson score intervals on an observed 60% mention rate and watch what sample size buys you:

  • 3 hits in 5 runs: the 95% interval spans roughly 23% to 88%. You know almost nothing.
  • 12 hits in 20 runs: roughly 39% to 78%. Now you can catch big moves.
  • 30 hits in 50 runs: roughly 46% to 72%. Tight enough for competitive comparisons.
5 runs 3 hits 23–88%
20 runs 12 hits 39–78%
50 runs 30 hits 46–72%
The same observed 60% mention rate at three sample sizes. More runs never move the point estimate; they narrow the interval around it.

Five samples per prompt per engine will catch a large move but will not measure a small one. It is still worth running, because a wide interval at least shows you how little you know.

Every query costs tokens. Moderate samples across many prompts beat huge samples on three vanity prompts, because you can pool related prompts whenever you need more sensitivity. If one prompt carries a real business decision, buy extra samples for that prompt.

How to measure AI visibility, step by step

This is the process I run in production. None of it requires custom software, and a spreadsheet plus some discipline gets you 80% of the way.

  1. Build the prompt set from real buyer questions. Pull from sales calls, support tickets, People Also Ask, and your GA4 site-search queries. Skip prompts nobody types.
  2. Freeze the definition of a hit before you run anything. Decide whether a hit means mentioned by name, actively recommended, or cited as a source, and write the rule down. Changing it partway through ruins the trend.
  3. Pick your engines. ChatGPT, Gemini, Perplexity, Claude, and Google AI Overviews if they matter in your vertical. Track them separately, because they don’t move together.
  4. Sample every prompt multiple times per engine, per run. Use a fresh session every time, with no memory, no logged-in personalization, and no follow-up questions.
  5. Record hits over trials, and keep the raw answers. You will want the evidence the first time someone challenges a number.
  6. Compute the mention rate and a Wilson confidence interval for every rate you plan to show another human being.
  7. Compare intervals, never points. Across time, against competitors, and between engines.
  8. Alert only when intervals separate. Overlap means “not a story yet,” and the report should say exactly that in plain words.
Prompt set real buyer questions Sampled runs fresh session, N per engine Hit rate hits over trials Wilson interval around every rate Compare intervals never points Alert or watch separation decides
next run, same frozen definitions
The production loop. Definitions stay frozen between runs so the trend stays readable, and interval separation decides the alert.

Most tracking falls apart at step 8. An alert fires only when two intervals separate, and everything short of that is logged as a watch item.

A worked example with the math shown

These numbers are made up to show the mechanics, so run your own before you trust anything here.

Say you track one prompt on one engine, 20 fresh runs per month.

  • Month one: your brand appears in 12 of 20 runs. That’s 60%, with a Wilson 95% interval of roughly 39% to 78%.
  • Month two: 8 of 20. That’s 40%, interval roughly 22% to 61%.

The point estimate dropped 20 points, and everybody wants to panic. But the intervals overlap between 39% and 61%, and a brand with a stable true rate of 50% could easily produce both months, so this one is logged as a watch item.

  • Month three: 3 of 20. That’s 15%, interval roughly 5% to 36%.

Now the whole interval sits below month one’s floor of 39%. The intervals have separated, so the alert fires, and you can defend the call to anyone who asks. Month two was “possible slide, watching,” and month three was “confirmed, here is the evidence.”

Month one 12 of 20 60% (39–78)
Month two 8 of 20 40% (22–61)
Month three 3 of 20 15% (5–36)
Months one and two overlap between 39% and 61%, so nothing provably moved and the report says so. Month three's whole interval sits below month one's floor, and the separation fires the alert.

When a brand slips across 30 prompts at once, pool them. Aggregated trials tighten the interval and catch broad moves far earlier than any single prompt can.

What a real AI visibility report looks like

The format is plain: every headline number carries its range. A rate of 54% on 100 trials reads as 54% (44% to 63%), which tells the reader what we know and how firmly we know it. Alerts appear only when intervals separate, so when one shows up, people act instead of debating whether it is real.

The uncomfortable feature is a status that says “we cannot tell yet.” Clients do not love it at first, and it pays for itself the first time a scare turns out to be sampling noise.

I shipped this in Nexus, the platform I built for my team at Tandem Interactive. Every prompt is sampled several times per engine, every rate carries a Wilson interval, and drop alerts stay quiet until the intervals separate. If you are evaluating AI visibility tracking tools instead of building one, ask every vendor for two things: the sample count behind each number, and the interval around it. If they cannot answer, the number is decoration.

The strictness comes from years of owning revenue numbers. Budgets move on the strength of a slide, and when the slide is noise, real money follows the noise.

FAQ

How is AI visibility different from SEO rank tracking?

Rank tracking measures a position on a mostly stable results page, so a single check was defensible. LLM brand tracking measures a probability: how often a model chooses to include you in generated text. Positions get spot-checked, while probabilities have to be sampled.

How many times should you sample each prompt?

Enough that the interval can answer your question. Five runs per engine flags only huge moves, twenty catches meaningful shifts, and pooling related prompts buys sensitivity without more spend. When the interval is too wide to support the decision, the fix is more samples.

What counts as a brand mention in ChatGPT?

Whatever you defined before the run: named in the answer, actively recommended, or cited as a linked source. I track those as separate rates because they mean different things commercially. The exact definition matters less than freezing it, since a moving definition makes every trend unreadable.

Why use Wilson confidence intervals instead of the normal approximation?

The normal approximation misbehaves at small samples and extreme rates. At 0% or 100% it produces a zero-width interval, which claims perfect certainty from a handful of runs. Wilson behaves at the sample sizes token budgets allow, and any stats library or spreadsheet can compute it.

How often should you run AI visibility tracking?

Weekly or monthly both work, as long as each run carries enough samples to mean something. A thin daily run reports noise on a faster schedule. Match the cadence to how fast you would act on a confirmed change.