How to check your AI visibility by hand with 20–50 buyer questions
A manual method any team can run in a day or two: choose buyer questions by intent, ask each AI engine three times, record what you see, and turn the tally into a rate with an honest range.
Short answer
- Write 20–50 questions your buyers really ask, split into unaided, comparison and branded intent, and ask each engine every question three times.
- For every answer, record whether you were mentioned, whether your site was cited, which competitors were named, which sources appeared and how the category was described.
- Report mentions as a rate with a confidence interval, per intent, and treat overlapping intervals as “no clear change”.
Technical figure
AI visibility audit scorecard
Audit work connects buyer questions, answer evidence, issue priority, and proof checks.
Why check by hand first
Before you pay for any tool, it helps to see AI answers with your own eyes. A manual check shows you how assistants describe your category, which competitors they reach for, and which websites they lean on. It also teaches you what a tool should be measuring, so you can judge one later.
The method below needs a spreadsheet, a few hours and some discipline. It will not be as fast or as repeatable as software, but it is honest, cheap and enough to decide whether AI visibility is a real problem for you.
Pick 20–50 buyer questions by intent
Start from the words buyers already use. Sales call notes, support tickets, demo requests, community threads and the search queries in your analytics are better sources than a brainstorm. Write each question the way a person would type it into a chat window, not as a keyword.
Sort the questions into three intents. Unaided questions describe a need without naming any brand, such as “what tools help a small agency report on client SEO?”. Comparison questions weigh options, such as “alternatives to a named competitor for small teams”. Branded questions name you directly, such as “how much does your product cost?”.
Keep the intents apart when you report. Unaided and comparison questions measure whether buyers discover you. Branded questions only measure recall and accuracy, because the buyer has already heard of you. A list that is mostly branded questions will make almost any brand look visible.
Twenty questions is enough for a first read. Fifty gives you room to cover several products, buyer types or markets. Aim for roughly two thirds unaided and comparison questions and one third branded.
Choose engines and ask each question three times
Pick the assistants your buyers are likely to use. For many B2B teams that means ChatGPT, Perplexity, Gemini, Claude and Google’s AI Overviews or AI Mode. Two or three engines done carefully beat six done in a hurry.
Answers vary from one run to the next, so ask every question three times on each engine. Use a fresh chat each time, sign out or use a clean browser profile where you can, and note the date and your location. Personal history and location can change what an assistant says.
Sometimes there is no usable answer: the engine errors, refuses, or Google shows no AI answer for that search. Record that as a failed or empty trial. Do not count it as “not mentioned”, because it tells you nothing about your brand.
Record five things for every answer
For each trial, write down whether your brand was mentioned, whether your website was cited as a source, which competitors were named, which sources or domains appeared, and the words the answer used to describe your category. Add a notes column for anything wrong or surprising.
Copy short quotes rather than summaries when an answer gets a fact about you wrong. A quote is evidence you can act on and show to a colleague; a summary is an opinion.
Turn the tally into a rate with an interval
Your mention rate is the number of answers that mentioned you divided by the number of usable answers. Leave failed and empty trials out of both numbers. Work it out separately for unaided, comparison and branded questions, and for each engine if you have enough answers.
A rate from a small sample is uncertain, so give it a range. The Wilson interval is a standard way to do that for yes-or-no counts. As a worked example, 9 mentions in 30 usable answers is a rate of 30%, with a 95% Wilson interval of roughly 17% to 48%. That wide range is the honest answer: with 30 answers you cannot tell 25% from 35%.
When you repeat the check later, compare intervals, not single numbers. If the before and after intervals overlap, call the result inconclusive rather than a win or a loss.
Read what the answers are telling you
If you appear for branded questions but rarely for unaided ones, assistants know you exist but do not connect you with the need. If you are mentioned but your site is never cited, the answers are drawing on other people’s pages about you. If the same few competitors and the same few domains keep appearing, those sources are shaping how your category is explained.
Also read the category descriptions. If an assistant describes your product with an outdated feature, the wrong price or the wrong audience, that is a fact problem to fix at the source, not a ranking problem.
What to do with the result
Measurement only tells you where you stand. Tools measure and diagnose; the work itself is content, accessibility, authority and sources.
Content means pages that answer the questions you missed plainly and early, with specifics a buyer can check. Accessibility means crawlers can reach and read those pages, so check your robots rules and make sure important text is not hidden behind scripts or logins. Authority and sources mean being described accurately on the third-party pages the answers already cite, such as review sites, directories, comparison articles and community threads.
After you publish a change, wait long enough for it to be crawled, then rerun the same questions the same way and compare the intervals.
When a tool starts to pay off
The manual method gets expensive once you track many questions, several engines, weekly reruns or several clients. At that point a tool that repeats the questions for you, keeps the answers and sources, and calculates the intervals will save more time than it costs. Until then, a spreadsheet is a perfectly good start.
Common questions
How many buyer questions do I need?
Twenty is enough for a first read, and fifty lets you cover several products, buyer types or markets. More questions narrow the interval, but only if they are questions real buyers ask.
Why ask the same question three times?
AI answers vary between runs. Repeating each question shows how stable an answer is and gives you enough answers to put a range around the rate.
Should I count answers where the engine failed?
No. Leave failed, refused and empty answers out of the rate. Counting them as “not mentioned” would make your visibility look worse than the evidence shows.