How do you know if AEO is working? Learn how to measure AI search visibility, citations, recommendations and business impact without relying on one score.
I’ve sat through enough marketing presentations to know that almost anything can look impressive if nobody agreed on the rules beforehand. AI search has created an entirely new opportunity for this. We now have AI visibility scores, citation counts, mentions, recommendations, share of voice and dashboards capable of showing very precise improvements in something most businesses couldn’t have defined a year ago.
So when somebody tells me a company’s AI visibility improved 38%, I don’t really care about the 38% until I know what changed underneath it. Did the company appear more often when people asked for a provider? Did its website earn more citations? Did one AI platform start mentioning it while the others remained unchanged? I’ve been around marketing long enough to know that a percentage without that context can tell a wonderful story. It just may not be the right one.
How Do You Know Whether AEO Results Are Actually Improving?
AEO measurement should tell you whether your company is becoming more visible for questions that matter to the business, whether that visibility is becoming more useful and whether anything outside the measurement platform supports the idea that something meaningful is changing. That doesn’t require perfect attribution, but it does require enough transparency to understand why the numbers moved.
I’m not looking for certainty here. I’m looking for enough evidence to make a decent business decision, which is a much more reasonable standard.
Ask What’s Hiding Behind the AI Visibility Score
The first thing I want to know when somebody shows me an AI visibility score is what had to happen for that number to move. Marketing loves composite scores because complicated things become wonderfully easy to sell once you turn them into a number. There’s nothing inherently wrong with using those scores as benchmarks. The problem starts when nobody can explain what went into them.
There isn’t one standard way to calculate AI visibility. As of August 2026, AirPulse’s AI Search Visibility Measurement Methodology recommends separating measurements such as mention rate, first-party citation rate, third-party citation rate and source stability rather than treating one AI response as a universal visibility score. Other platforms may combine measurements into their own weighted scores. The number isn’t necessarily the problem. Not knowing what it represents is.
If an agency can’t explain which questions are being tested, which AI platforms are included and what constitutes a mention, citation or recommendation, I wouldn’t put much faith in a percentage change. If we’re going to celebrate the score, we should at least know how we got it.
Make Sure the Questions Didn’t Get Easier
Suppose an agency begins with questions where your company rarely appears. Three months later, the dashboard looks dramatically better. Before celebrating, I’d want to know whether we’re still measuring the same thing.
Were poorly performing questions removed? Were new questions added where the company already had stronger visibility? Did informational questions replace harder commercial ones? Did the geography, AI platforms or testing frequency change? There may be legitimate reasons to update the measurement set, but those changes should be documented. Otherwise we’re comparing two different tests and calling the difference improvement.
That matters because AI results themselves can vary. A 2026 academic study, Quantifying Uncertainty in AI Visibility, repeatedly sampled generative search platforms and found substantial variation in citation results. As of September 2026, Zion Labs’ published AI visibility methodology similarly emphasizes repeated testing and evaluating individual AI engines rather than assuming one response represents a brand’s visibility everywhere.
Marketing has enough people turning correlations into strategies already. We don’t need to start turning moving goalposts into AEO results.
Measure Visibility Where a Customer Might Actually Choose You
Not all AI visibility has the same business value. Imagine a commercial roofing company starts appearing frequently in answers about the history of roofing materials. That’s real visibility. But unless the company has suddenly decided to enter the roofing-history business, I’d also want to know what happens when someone asks questions closer to actually hiring a contractor.
A business might make enormous gains across informational questions while barely moving for comparisons, local searches or direct recommendations. For a Kansas City business trying to become more visible when local customers ask AI systems who they should consider, that distinction matters. Another company might make a smaller overall gain while beginning to appear consistently around questions customers ask immediately before contacting a provider. If I owned the second company, I’d probably find that smaller number considerably more interesting.
Being cited is different from being mentioned, and being mentioned is different from being recommended. AEO reporting should make those distinctions clearer, not bury them inside one impressive percentage.
Look for Something Outside the AEO Tool
I don’t expect an AEO platform to prove that every new customer came from AI. Someone can discover a company through an AI answer, remember the name and search for it later, visit directly or make a phone call. The AI interaction may never appear in conventional analytics. Marketing attribution wasn’t exactly flawless before AI showed up, either.
What I do want is supporting evidence. If measured AI visibility is improving, what’s happening with branded search, direct traffic, identifiable AI referrals, leads and sales conversations? Are customers beginning to say they encountered the company through an AI research tool? None of those things alone proves that AEO caused the change. I’m looking for independent signals that begin telling a compatible story.
I’d also ask customers how they found the business. It isn’t sophisticated, but sometimes a person saying, “Gemini gave me your company when I was researching this,” tells you something your analytics never could. We’ve spent years trying to make attribution more sophisticated. Sometimes asking the customer is still remarkably useful.
Don’t Let the Dashboard Grade Its Own Homework
The same company or agency doing the optimization may also choose the questions, determine the weighting, define the visibility score and decide which improvements get featured in the report. That doesn’t mean the results are dishonest. It means the client should understand the methodology well enough to evaluate them.
If the dashboard says visibility improved dramatically, show me which questions improved, whether the change happened repeatedly and where nothing changed. One of the quickest ways an agency can make me trust its reporting is to show me something that didn’t work. Maybe recommendation visibility stayed flat, new pages haven’t been cited or one AI system responded while another didn’t. That’s useful because it tells us what to investigate next.
I’ve never believed a marketing report becomes less credible because it contains bad news. Usually the opposite is true. A report designed exclusively to prove success isn’t really measurement. It’s marketing.
Don’t Confuse Better Measurement With Better Results
AI search tools will get better at measuring visibility. We’ll have more questions, more platforms, better historical data and more sophisticated ways to categorize mentions, citations and recommendations. That’s useful, but better measurement doesn’t automatically mean better marketing.
If an agency shows major gains for questions nobody who buys from you ever asks, I wouldn’t be particularly impressed. A smaller change may be far more meaningful if the business begins appearing consistently when potential customers are deciding whom to consider.
Visibility is an intermediate outcome. The eventual objective is awareness, consideration, leads, opportunities, customers or whatever else the organization actually cares about. If the measurement system can’t eventually help us understand that relationship, we’ve built a very sophisticated way to measure something we haven’t established matters.
So, Is Your AEO Actually Working?
I wouldn’t answer that question with a single visibility score or one screenshot showing that an AI platform recommended the company. I’d want to know what changed underneath the numbers, whether the same things were measured consistently, whether improvement occurred around questions that matter commercially and whether anything outside the AEO dashboard supports what we’re seeing.
AI search optimization is working when the evidence becomes increasingly difficult to dismiss. The business is appearing more consistently around the right subjects and in more valuable parts of the customer journey, while independent measurements begin pointing in the same direction. The people doing the work should also be able to explain why they believe the change is happening without hiding behind a proprietary score.
We may not be able to connect every AI answer to a customer. That’s okay. I’ve been doing this long enough to know marketing rarely gives us that kind of certainty anyway. But we should still be able to tell the difference between progress and a prettier dashboard.
Related:
Ask us! What questions do you have about content, strategy, pop culture, lifestyle, wellness, history or more? We may use your question in an upcoming article!
Like MediaFeed’s content? Be sure to follow us.
This article originally appeared on Fireball Agency and was syndicated by MediaFeed.co.
