A new wave of AEO and GEO tools has arrived, and they all promise the same thing: tell you exactly how visible your brand is in ChatGPT, Perplexity, and Gemini, right down to your ‘AI share of voice’ and ‘prompt search volume.’
It sounds powerful. It looks good in a dashboard. And it is costing some businesses hundreds or thousands of dollars a month for data that, in many cases, simply does not reflect reality.
SEO professionals who have dug into the methodology behind these tools have found a troubling common thread: the numbers are often based on tiny data samples, flawed methodology, and assumptions that do not match how AI search actually works.
In this post, we break down exactly why most AI visibility metrics are unreliable, what you should be tracking instead, and how Alev Digital approaches visibility in ChatGPT and AI search results in a way that actually connects to real business outcomes.
The Problem: AI Search Is a Black Box
With traditional SEO, we have reasonably reliable data. Google Search Console shows impressions, clicks, and average positions. Keyword research tools estimate monthly search volumes based on clickstream data, panel surveys, and Google’s own keyword planner.
AI search has none of this infrastructure.
OpenAI does not publish how many times ChatGPT receives a specific query. Anthropic does not share Claude’s usage data. Perplexity does not release prompt logs. Every major AI platform keeps its search behavior completely private.
This creates a data vacuum. And into that vacuum, a rush of tools have stepped in with dashboards full of numbers that look authoritative but are built on shaky foundations.
Where Are These ‘AI Search Volume’ Numbers Coming From?
When you dig into the methodology behind most AI keyword volume tools, you typically find one or more of these approaches:
- Browser extension panels: Some tools rely on data collected through small browser extension user panels. These panels represent a tiny fraction of actual AI users, sometimes less than one percent, and are subject to significant selection bias.
- Blended Google search volume: Many tools are simply applying a multiplier or modifier to traditional Google keyword volume data and calling it ‘AI prompt volume.’ The problem is that people talk to AI completely differently from how they type into Google. Conversational queries, follow-up questions, and multi-turn prompts have no equivalent in Google search data.
- API simulations: Some tools query AI APIs directly and track their responses, but API calls do not replicate real user sessions.
- They lack personalization, session history, and location context, all of which influence what AI platforms actually return.
As Ahrefs noted in a community discussion: tracking a tiny, biased sample of prompts is mathematically meaningless for calculating an overarching share of voice.
The Routing Problem: Web Search vs. Training Data
There is a deeper technical issue that most AI visibility tools ignore entirely: how AI platforms decide whether to search the web or answer from training data.
When a user asks an AI a question, the model makes a decision. It either:
- Retrieves live web content to generate its answer (RAG, or Retrieval Augmented Generation)
- Answers directly from the knowledge encoded in its training weights
⋆ If the AI answers from training data, and your brand launched or changed significantly after the model’s knowledge cutoff, you effectively do not exist to that model. No amount of on-page optimization will help.
Most AI tracking tools do not account for this routing decision at all. They query the API, get an answer, and report that as your ‘visibility.’ But without knowing whether the AI ran a web search or answered from memory, the data is incomplete at best and misleading at worst.
The Volatility Problem: Less Than 1% Consistency
Research published by SparkToro tested AI response consistency across multiple sessions. They found that for a given brand-related query, there was less than a one percent chance that ChatGPT or Google AI would produce the same list of brands in two separate answers.
AI responses are influenced by:
- The user’s location and local context
- The model’s temperature settings and randomness parameters
- Prior conversation context and session history
- Real-time web retrieval variations
What this means in practice is that a tool tracking your ‘AI share of voice’ by sending 50 API prompts per month is not capturing your real visibility. It is capturing a tiny, unrepresentative sample of a system that behaves differently for every user.
What Should You Actually Track?
This is not an argument against measuring AI visibility. It is an argument for measuring it more honestly and more usefully.
Here is what actually matters and can be tracked with reasonable accuracy:
1. Direct Citation Tracking
Rather than measuring ‘share of voice,’ track specific instances where your brand, content, or website is cited by AI platforms. Use manual spot-checks across ChatGPT, Perplexity, and Gemini for your key service queries on a regular schedule.
This is not automated. It is not scalable to thousands of keywords. But it is real.
2. Referral Traffic From AI Platforms
Check your Google Analytics or whatever analytics platform you use for direct referral traffic from AI sources. Perplexity and some other AI platforms do pass referral data. This is actual, verifiable traffic, not an estimate.
3. Brand Mention Monitoring
Use tools like Google Alerts, Mention, or Brand24 to track where your brand is being mentioned across the web. Since AI engines pull from these sources, growing your off-site mentions is both trackable and directly correlated with AI visibility improvement.
4. Qualitative AI Response Audits
Monthly or quarterly, run structured queries about your brand and industry through all major AI platforms. Document what comes up. Track changes over time. This gives you a qualitative picture of your AI narrative that no automated tool can fully replace.
Alev Digital conducts structured AI visibility audits as part of our AEO Services, giving clients a clear, honest picture of where they stand in AI search.
5. SEO Metrics That Correlate With AI Visibility
Certain traditional SEO signals are strongly correlated with AI citation likelihood:
- Domain Rating (DR) and overall domain authority
- Number and quality of referring domains
- opical authority scores in your primary subject area
- Brand search volume growth over time
Improving these metrics in Google also tends to improve your AI search visibility, because the underlying signals (trust, authority, relevance) are what both systems are measuring.
Our Technical SEO Services are designed to build these foundational signals that support both traditional and AI search performance.
The Real Cost of Chasing Fake Metrics
Beyond the wasted budget, there is a strategic cost to relying on AI volume metrics that do not reflect reality.
When teams optimize for metrics that are not real, they make decisions based on false signals. They might conclude that a campaign is working because their ‘AI share of voice’ score went up, while their actual referral traffic from AI platforms stayed flat.
Or they deprioritize activities that actually build AI visibility, like off-site brand building and content quality improvement, in favor of technical optimizations that chase the wrong indicators.
The best AI search optimization services focus on building the foundations that make AI engines trust you, not on gaming metrics that do not connect to how AI search actually works.
Want an honest assessment of your AI search visibility? Our team at Alev Digital focuses on signals that matter. Book a consultation here.
What Good AEO Measurement Actually Looks Like
At Alev Digital, we believe in honest measurement. Before recommending any AEO investment, we establish a clear baseline using metrics that are verifiable:
- Manual AI platform audit: What are ChatGPT, Perplexity, and Gemini currently saying about your brand?
- Referral traffic audit: Are any AI platforms currently sending measurable traffic to your site?
- Off-site mention audit: How many times is your brand mentioned across Reddit, Quora, review sites, and publications?
- Technical accessibility check: Can AI crawlers access your site? Are there any blocks in robots.txt or CDN settings?
- Content quality assessment: Does your content have enough factual density and direct-answer structure to be cited by AI engines?
This gives us a real starting point. Progress is then measured against these baselines, not against dashboard numbers generated by tools that cannot actually see inside AI platforms.
If you are currently paying for AEO or GEO tracking tools and are unsure whether the data is meaningful, our SEO Audit Services include a full review of your measurement setup and a recommendation for what is actually worth tracking.
Authoritative External References:
- SparkToro Research: AI Response Consistency — data on AI answer volatility across sessions
- Ahrefs Blog: The Truth About AI Keyword Volume — critical analysis of AI search data reliability
Frequently Asked Questions
Not entirely. Tools that track actual referral traffic from AI platforms, monitor brand mentions across the web, and run structured citation audits can provide useful data. The tools that are unreliable are those claiming to show precise ‘AI search volume’ or ‘share of voice’ based on API queries or tiny data panels. Always ask a tool how their data is generated before buying.
No. AEO is still critical, especially as AI search usage grows. The point is to invest in the right activities, building content quality, off-site authority, and brand presence, rather than chasing metrics that do not reflect reality. Measurement should inform strategy, not distort it.
The most reliable indicators are: growth in branded search volume on Google, increases in referral traffic from AI platforms in your analytics, positive changes in what AI engines say about your brand in manual audits, and growth in off-site brand mentions over time.
Focus your budget on foundational tools: a solid SEO platform, brand monitoring software, and analytics. Be very cautious about expensive AI-specific tracking dashboards until the underlying methodology is proven. The activity matters more than the tracking at this stage.
We use a combination of manual AI audits, referral traffic analysis, brand mention monitoring, and traditional SEO signals to give clients an honest picture of their AI search visibility. No inflated dashboards, just real data.
