# AI search visibility you can actually measure Which AI crawlers read your pages, which AI answers cite you, and how many people those answers send back. Three sources, one chat, and a baseline you can hold up against any GEO pitch. [Start free](https://app.aidata.stream/register) [Read the baseline post](/blog/are-you-cited-in-ai-answers/) ## Reading rarely becomes visiting The same 30 days on one of our own sites, seen from the edge, from GA4 and from Search Console. Each layer answers a question the others cannot. 1. AI crawler requests Cloudflare, every bot that fetched a page 38,223 2. Visits from AI tools GA4 sessions referred by an assistant 209 3. Google organic clicks Search Console, same 30 days 334 - ~60 live assistant fetches for every visit an AI tool sent - 206 of 209 visits came from ChatGPT. PerplexityBot made 3,775 requests and Perplexity sent 2. - ~2,400 requests wore an AI user agent but asked for `/.env` and credential files One of our own sites, 30 days, October 2026. Bars are on a log scale; AI visits ran at 63% of organic clicks. Figures from [the dashboards post](/blog/claude-dashboards-analytics-data/). ## Three layers, three sources Connect what you have and the chat joins them: a crawler read, a citation and a visit for the same page can sit in one answer. [ ### Who reads your site Cloudflare, Pro and Business Requests and bandwidth per AI crawler per day, the pages they fetch most, and the URLs they keep asking for that no longer exist. Verified bots are split from anything merely wearing a crawler’s name. Setup](/docs/connecting-data-sources/#cloudflare)[ ### Who cites you DataForSEO, every plan The Google AI Overview and ChatGPT answers that cite your domain, the questions behind them, the pages that earn the citations, and up to nine competitors side by side. Setup](/docs/connecting-data-sources/#dataforseo)[ ### Who comes back GA4 and Cloudflare Visits that arrive from ChatGPT, Perplexity, Claude, Copilot and Gemini, by day or landing page, and an opt-in beta that finds the AI visits GA4 files under Referral or Unassigned. Setup](/docs/connecting-data-sources/#google-analytics-4) ## AI crawlers at the edge, scanners removed Anyone can send a crawler’s user agent string, and vulnerability scanners do, by the thousand. On our own marketing site, just over 1,000 requests in 48 hours carried an AI crawler name, and roughly six in ten of them were asking for credential files that do not exist. The same probe arrived under 8 different crawler names. - Every crawler row splits successful fetches into verified requests, confirmed by Cloudflare as the bot’s operator, and unverified ones. - Failed requests are reported separately, so scanner probes never inflate the crawler count or the bandwidth figure. - Ask for AI crawler paths with status 400 and above to see exactly what the scanners asked for. - Four tools: crawler activity, crawler paths, AI referrals and a whole-site traffic overview, plus a dashboard widget and the AI Crawler Visibility skill. Requests with an AI crawler user agent, 48 hours, aidata.stream - ~6 in 10 scanner probes for credential files - ~4 in 10 crawlers and assistants fetching real pages Scanner requests return 404 on a static site, which is how the tools tell them apart. What the scanners asked for - 404/.env - 404/.env.production - 404/.ssh/id\_rsa - 404/.aws/credentials - 404/firebase-admin.json - 404/terraform.tfstate From [our two days of logs](/blog/most-of-your-ai-crawler-traffic-isnt-ai/), with two free Cloudflare rules that block them. A yacht brokerage client, never any “GEO work” 1,004 AI answers cite the site - 977 Google AI Overviews - 27 ChatGPT answers ChatGPT fetched a page from the site in 128 answers - 27 cited it as a source - 101 read it and quoted a competitor instead That gap is a content brief, not a technical problem. From [Are you cited in AI answers?](/blog/are-you-cited-in-ai-answers/) ## Citations you can count, before anyone sells you GEO DataForSEO keeps a corpus of real AI answers: the question, the answer, the pages it cited, and how often the question is asked. Ask the chat how visible you are, which pages earn it, and who the AI treats as your peers. - Visibility summary with a month-by-month history, the actual cited answers, the pages that earn citations, a comparison with up to nine competitors, and the domains cited most for a topic. - For ChatGPT, whether it cited your page or only retrieved it without quoting it. - The AI Search Visibility skill turns it into one report: a measured baseline, then a way to check any agency’s claims afterwards. - Coverage is Google AI Overviews and ChatGPT only, with ChatGPT limited to the United States in English. Data starts August 2025 and the current month is always incomplete. ## The visits GA4 files in the wrong place GA4’s AI Assistant channel only catches visits that arrive with an `ai-assistant` medium. Perplexity, Copilot and untagged ChatGPT links land in Referral or Unassigned, next to your own subdomains, link scanners and referrer spam. - With the beta on, every traffic-source lookup gives those rows a normalised channel, an AI-assistant flag your AI can total, and a confidence score. - Each source is judged once and cached for 30 days, so the extra second only applies to the first lookup. - GA4 stays the source of truth. If the model is unavailable you get plain GA4 results, never an error. - Beta, off by default. A team owner switches it on under Beta Features on Pro and Business. It runs on our key, not your LLM key. On a paid Cloudflare zone, the AI referrals tool shows the same visits from the edge, by day or landing page, before GA4 and consent banners get involved. GA4 source / medium With the beta on 1. perplexity.ai / referral GA4: Unassigned AI assistant is\_ai\_assistant 2. copilot.microsoft.com / referral GA4: Referral AI assistant is\_ai\_assistant 3. chatgpt.com / referral GA4: Referral AI assistant is\_ai\_assistant 4. staging.yoursite.com / referral GA4: Referral Self-referral 5. link-checker / (not set) GA4: Unassigned Link scanner or tool Every reclassified row carries a confidence score. Below the threshold, GA4’s grouping is kept and the suggestion is only noted. ## What GA4 alone cannot show - Crawler reads. A bot fetching a page never runs your GA4 tag, so the request only exists at the edge. - Scanner noise. Probes wearing a crawler’s user agent return 404s and never appear anywhere but the edge log. - Traffic GA4 never counts: ad blockers, consent declines and failed tags. Cloudflare counts every request. - Referrer-stripped visits. Apps that drop the referrer show up as Direct, so AI visit counts in GA4 are a floor. Two skills put it together: AI Crawler Visibility, and Edge vs Analytics Gap for the traffic that exists at the edge but not in GA4. [Skills and reports](/docs/reports/). ![Stacked bar chart of AI crawler requests per day by operator, beside a table of requests by bot with operator, type, requests, share and megabytes](/images/blog/claude-dashboards/ai-visibility-crawlers.webp) Requests per day by operator, with each bot typed as a live assistant fetch, a search-index bot or a bulk crawler. Built in Claude from the connector. ![Table of most-fetched paths with AI requests, the main bots, AI visits and Google clicks for each path](/images/blog/claude-dashboards/ai-visibility-paths.webp) Where the three sources meet: for each path, its AI requests from Cloudflare, the AI visits that landed there from GA4, and its Google clicks from Search Console. Cloudflare Pro and Business. One API token with a single permission, Zone, Analytics, Read, scoped to one zone. AI referrals need a paid Cloudflare zone. DataForSEO Every plan, including Free. Billed to your own DataForSEO account at around $0.10 per AI visibility call, with a row ceiling and an allowed-domains list. AI traffic classification Beta on Pro and Business, opt-in under Beta Features, runs on our key. GA4 itself is on every plan. [Compare plans](/pricing/) · [How the chat joins sources](/features/ai-chat/) · [Same tools from your own AI client](/features/mcp-server/) ## Questions ### Does this show Search Console AI Mode or AI Overview impressions? No. Search Console shows AI Overview impressions in its own interface but offers no API for them, so no tool can pull that figure. The citation data here comes from DataForSEO’s corpus of real AI answers instead. ### Which AI platforms are covered? Crawlers: whichever bots hit your Cloudflare zone, including GPTBot, ClaudeBot, PerplexityBot, CCBot and Bytespider. Citations: Google AI Overviews in around 94 countries and ChatGPT in the United States in English, since August 2025; Perplexity, Gemini, Claude and Copilot are not included. Referrals: ChatGPT, Perplexity, Claude, Copilot and Gemini. ### Do I need a paid Cloudflare plan? Any proxied Cloudflare zone gives you crawler activity, crawler paths and the traffic overview. Cloudflare only exposes the referrer host on paid zones, so the AI referrals tool needs one. Retention of per-request data also depends on your Cloudflare plan; every tool checks what your zone keeps and says so. ### What does the citation data cost? DataForSEO is a bring-your-own-key connection billed to your own DataForSEO account: around $0.10 per AI visibility call plus a fraction of a cent per row. The AI Search Visibility skill makes five to seven calls, and every answer shows what the call cost. A row ceiling and an allowed-domains list keep spending in check. ### Can it block the scanners it finds? No. The Cloudflare token has Zone, Analytics, Read as its only permission, so the connection cannot change firewall rules or anything else. Our post on scanner traffic shows two free Cloudflare rules you can add yourself. ### Does the classification beta use my AI key? No. It runs a System One model from TypeSafe AI on our key, not on your LLM key. It is off by default, a team owner switches it on under Beta Features on Pro and Business, and GA4 stays the source of truth: below a confidence threshold the GA4 grouping is kept. ### Can my own AI client see this data? Yes. The Cloudflare, DataForSEO and GA4 tools are available through the connector, with the same plan rules and permissions as the built-in chat. ## Launch history - 2026-10-19 Guide [AI search visibility with DataForSEO: five tools and the AI Search Visibility skill](/blog/are-you-cited-in-ai-answers/) - 2026-10-11 Guide [Most of your "AI crawler" traffic isn’t AI; counts now exclude failed fetches](/blog/most-of-your-ai-crawler-traffic-isnt-ai/) - 2026-10-10 Guide [An AI visibility dashboard built in Claude from the connector](/blog/claude-dashboards-analytics-data/) - 2026-09-18 Shipped [AI traffic classification beta (System One model)](/blog/adding-system-one-models/) - 2026-09-15 Shipped [Cloudflare: AI crawlers and edge traffic](/blog/cloudflare-now-available/) Page last updated 2026-10-25. ## Get your baseline Connect DataForSEO and run the AI Search Visibility skill in your free month of Business. Add Cloudflare to see who is reading. [Start free](https://app.aidata.stream/register) [Connect DataForSEO](/docs/connecting-data-sources/#dataforseo) --- Source: https://aidata.stream/features/ai-visibility/