"We appear in 46 percent of AI answers and our main competitor appears in 56." It sounds like a finding. It means very little until you know how many answers were counted, who wrote the questions and what "appear" meant.
Comparing brands in AI search is harder than measuring your own, for three reasons. The answers change from one run to the next. The questions are easy to tilt towards yourself without noticing. And a gap that looks large on a slide is often smaller than the noise in the sample.
This guide is a manual method you can run in a spreadsheet. It covers which competitors to include, how to write prompts that favour nobody, four things to score for each brand, how large a gap must be before it is real, and how to find the outside pages that get competitors named. It builds on the monthly routine in our guide to measuring AI search visibility, so set that up first if you have not.
An AI search competitor analysis needs a fixed method. To benchmark competitors in AI search, fix three to five direct competitors, run the same neutral buyer prompts on each engine, and score every brand on four things: named, cited with a link, described accurately and recommended. Print the answer count beside every rate. With 90 answers per engine, a gap under about 15 points is noise. Sort the links cited beside each competitor to find where to earn a mention.
Choose a Fair Set of Competitors
The comparison is only as fair as the list of brands in it. A list drawn up from memory in a meeting tends to hold the rivals sales loses to most often and to miss the ones buyers hear about from an assistant.
Three rules keep the set honest.
- Three to five direct competitors. They sell to the same kind of buyer, in the same market, at a similar size of deal. A company that wins a different customer is not a competitor for this purpose, even if it sells a similar product.
- One larger name for reference. Include one well-known brand in the category and label it as the reference. It shows what a high rate looks like on your prompts. It is not a target for next quarter.
- A written rule for brands you did not list. Assistants will name companies that are not on your list. Decide in advance when one joins the scorecard. A workable rule: any brand named in at least a tenth of the answers in two months running is added from the following month.
Apart from brands added under that rule, fix the set for at least a quarter. If the list changes every month, no trend can be read from it.
Write Prompts That Favour Nobody
The prompt set decides the result more than anything else. Questions written by your own marketing team carry your own vocabulary: your product names, your category label and the features you lead with. An assistant answers the question it was asked, so a tilted question produces a tilted answer.
Use the buyer's wording, and keep brand names out of every category prompt. Sales call notes, enquiry forms and the queries in Search Console are better sources of wording than a brainstorm. Then sort the prompts into three groups, so the set covers different moments in a purchase.
- Shortlist prompts. The buyer asks who to consider. No brand is named.
- Problem prompts. The buyer describes a difficulty and asks how to solve it. No brand is named.
- Comparison prompts. The buyer names two or more brands and asks how they differ. Score these as a separate group, because every brand named in the question will be named in the answer.
| Group | Tilted prompt | Why it is tilted | Neutral version |
|---|---|---|---|
| Shortlist | Which suppliers offer [a feature only you have]? | It asks for your differentiator, so it finds you | Which suppliers should a mid-sized company in [country] consider for [category]? |
| Shortlist | Is [your brand] a good choice for [category]? | It names you and invites agreement | What should a buyer look for when choosing a [category] supplier, and which companies meet that? |
| Problem | How does [your product name] solve [problem]? | It uses your product name as the subject | Our team has [problem]. What are the usual ways to fix it, and who provides them? |
| Comparison | Why is [your brand] better than [competitor]? | It states the conclusion in the question | How do [brand] and [brand] differ for a company that needs [requirement]? |
In comparison prompts, rotate the order in which the brands are named, so that no brand is always first in the question. Before the first run, give the full set to someone outside marketing and ask one thing: could a competitor have written these? If the answer is no, rewrite.
Score Four Things for Each Brand
For every answer, record four things about every brand in the set. Each one is a yes or a no, which keeps two people scoring the same answer in agreement.
| Score | Count a yes when | What it shows |
|---|---|---|
| Named | The brand appears anywhere in the answer | Whether the assistant connects the brand with the question at all |
| Cited with a link | The answer links to a page on the brand's own website | Whether the brand's own pages are being used as a source |
| Described accurately | What the answer says about the brand is correct today. Score this only where the brand is named | Whether being named is helping or misleading the buyer |
| Recommended | The answer puts the brand forward as a choice for the buyer's need, and does not only list it or mention it in passing | The difference between being suggested and being present |
"Recommended" needs a written rule, because it is a judgement. One that works: the answer says the brand suits the buyer's stated need, or includes it in a list the answer itself presents as its suggestions. A brand mentioned as an alternative, as background or with a warning is named and not recommended.
You can only check accuracy for competitors from what they publish, so keep that score to plain facts: what they sell, where they operate and what they say they charge.
Do not score "named first". It is tempting to give extra credit to the brand at the top of a list. A study published by SparkToro in January 2026, carried out with a co-author from Gumshoe, an AI tracking company, had 600 volunteers run 12 prompts through ChatGPT, Claude and Google's AI features in November and December 2025, for 2,961 runs in total. The chance of getting the same list of brands twice was under 1 in 100, and about 1 in 1,000 for the same list in the same order. How often a brand appeared across many runs was much steadier. The study is not peer reviewed, and its authors disclose the link to a company that sells tracking. Its conclusion still matches the arithmetic: count how often, not where.
How Large a Gap Has to Be Before It Is Real
Every rate on the scorecard is an estimate from a sample, so it carries a margin of error. When you compare two brands, both rates carry one, and the margin on the difference is wider than the margin on either rate alone.
The sum is the standard one for the difference between two proportions. Take each brand's rate as a fraction, p, and the number of answers, n:
margin on the gap = 1.96 x square root of ( pA x (1 - pA) / n + pB x (1 - pB) / n )
Example: both brands near 50 percent, 90 answers
0.5 x 0.5 / 90 + 0.5 x 0.5 / 90 = 0.005556
square root of 0.005556 = 0.0745
1.96 x 0.0745 = 0.146, or 14.6 pointsThe margin is widest when both rates are near 50 percent, so that case gives a safe figure to plan with.
The gap two brands need before they can be told apart
Approximate 95 percent margin on the difference between two rates, in points, when both are near 50 percent
- 30 answers each About 25 points About 25 points30 answers each
- 60 answers each About 18 points About 18 points60 answers each
- 90 answers each About 15 points About 15 points90 answers each
- 180 answers each About 10 points About 10 points180 answers each
- 360 answers each About 7 points About 7 points360 answers each
- 720 answers each About 5 points About 5 points720 answers each
| Answers counted for each brand | Smallest gap worth reporting | What that sample is |
|---|---|---|
| 30 answers | About 25 points | 10 prompts, three runs each |
| 60 answers | About 18 points | 20 prompts, three runs each |
| 90 answers | About 15 points | 30 prompts, three runs each |
| 180 answers | About 10 points | 60 prompts, three runs each |
| 360 answers | About 7 points | 60 prompts, three runs, two months combined |
| 720 answers | About 5 points | 60 prompts, three runs, four months combined |
With 90 answers, a gap under roughly 15 points is inside the noise. Halving the margin takes four times as many answers, which is why a small prompt set cannot settle a close race however carefully it is run.
These margins are wider than the ones in our measurement guide, and the two agree. That guide gives the margin on one rate: about 10 points at 90 answers. A comparison involves two rates, each with its own error, so the margin on the gap is about 1.4 times larger.
Now read the invented scorecard again, at 90 answers:
- Brand A and Brand B, named: 46 against 56 percent. The gap is 10 points and the margin is about 14.5. Treat them as level.
- Brand A and Brand C, named: 46 against 32 percent. The gap is about 13 points and the margin is about 14. Still inside the noise, though close to the edge. Watch it next month.
- Brand A and Brand D, named: 46 against 20 percent. The gap is about 26 points and the margin is about 13. This one can be reported.
Calculator
Check Whether a Gap Is Larger Than the Noise
The numbers for Brand A and Brand B from the invented example are filled in. Replace them with your own counts for two brands on the same answers. Nothing you type is stored or sent.
- First brand's rate
- Second brand's rate
- Gap between them
- Margin of error on the gap
The calculator works for any of the four scores, as long as both counts come from the same set of answers. It uses the sum shown above, so it shares the same caution: repeated runs are not fully independent, and the true margin is somewhat wider than the one printed.
Find the Sources Behind the Names
A scorecard says who is named. It does not say why. The closest thing to a why that you can observe is the set of pages an assistant links to when it names a competitor.
Each time an answer names a competitor and shows sources, copy the links into a second sheet, with the brand and the prompt beside each one. After a month, sort the links into five groups.
| Group | What it is | What to do with it |
|---|---|---|
| The competitor's own site | Service pages, comparison pages, pricing pages, articles | Note the page type. If their comparison page is cited and you have none, that is a gap in your own site |
| Directories and lists | Supplier directories and "top suppliers" articles | Check whether you are listed, and whether the listing is complete and current |
| Review sites | Platforms where customers rate suppliers | Compare the number and age of reviews, not only the score |
| Forums and communities | Threads where practitioners answer each other | Read what is said. It shows the wording buyers use and the objections they raise |
| News and trade press | Coverage, interviews and contributed articles | A list of publications that cover your category and may cover you |
Read the result with care. A cited link shows what the assistant displayed beside its answer. It does not prove that the page caused the brand to be named. It is a list of places worth being present, and it is a better starting point than a guess. Our article on how brands get recommended in ChatGPT explains why mentions on other sites matter.
One first-party report helps here. Bing Webmaster Tools has an AI Performance report, in public preview since February 2026, which counts citations of your own pages in Copilot and Bing's AI answers. In June 2026 Microsoft added Citation Share and a Compare view. Citation Share is your share of all the citations given for a query the AI used to find sources. Microsoft says it does not reveal competitor domains and does not represent a share of traffic. So it tells you how much of the citing you receive on a topic, and not who receives the rest. The manual sort above fills that gap.
Report Each Engine Separately Before Any Total
Assistants draw on different indexes and different sources, so they disagree about the same category. A brand can be named often on one and rarely on another. An average across engines hides exactly that.
Keep one scorecard for each engine. Give a combined figure only after the separate ones, and only if every engine was run with the same prompts and the same number of answers.
The disagreement is information. If a competitor leads on one engine alone, look at the sources that engine cites for them. It may be one directory, one review site or one set of pages that the other engines use less. If you trail on every engine, the cause is more likely to be on your own site, starting with whether the crawlers can read it. Our guide to AI crawler access problems covers that check.
A Report That Fits on One Page
Leadership asked one question, so the report should answer it on one page. Collect the answers every month, alongside your own tracking, and report the comparison once a quarter on the combined answers. One month on a small sample mostly reports noise.
- The scorecard, for each engine, with the number of answers printed beside every rate.
- The gaps that are larger than the margin, and a plain statement that the others are level.
- What changed since last quarter, including any brand added under the written rule and any change to the prompts.
- What the assistants got wrong about you, since an inaccurate description is the most direct thing you can fix.
- Three actions, each tied to a line in the source sort: a listing to complete, a page to write, a publication to approach.
Leave out the single combined score, the rank order of brands and any chart without a sample size. Each of them invites a conclusion the data cannot carry. The next question from leadership is usually whether any of this brings visitors, and our article on AI citations vs website traffic covers what can and cannot be shown.
What a Benchmark Cannot Tell You
State these limits in the report itself. They are easier to accept there than when someone else raises them.
- It is not market share. Being named in 46 percent of 90 answers to your prompts says nothing about what share of buyers choose you.
- It is not a ranking. Assistants do not hold a fixed order of brands, and the order inside one answer changes between runs.
- It does not show why a brand was named. The source sort gives leads to follow, not causes.
- It describes your prompts. A different, equally fair set of questions would give different rates. The comparison holds only within the set.
- It does not predict next month. Assistants change their models and sources without notice. A lead today is an observation, and acting on the source list does not guarantee that an assistant will name you.
Within those limits, the benchmark does one useful thing. It replaces "they seem to be everywhere in ChatGPT" with a count, a margin and a short list of places to work on. The wider method sits in our complete guide to AI SEO.