Skip to content

See where you stand

Change it

Works withWordPressShopifyWebflowGitHubGoogleSlackAll integrations
FixesFixes now ship to WordPress, Shopify, Webflow and GitHub

By team

By goal

Done for you

Not sure where to start? Take a guided tour.Book a demo
Customer storyHow the voice AI company Futuro grew its traffic3×AI traffic in 60 days

Learn

Data and research

FreeAI Site GraderAI Query Scannerllms.txt GeneratorAll free tools
Open dataWhere AI sends people, across more than 6,000 sites

Cloudflare AI crawler monitoring

Turn Cloudflare AI Crawl Control into an AI visibility workflow: see which AI services reach your site, which paths they request, and whether crawler access lines up with Trakkr visibility changes.

Explore crawler monitoringRead crawler docs
AI Crawl Controllope.example · last 24 hours
BotPathStatusRequests
PerplexityBot/guides/wide-fit403312
GPTBot/pricing2001,204
ClaudeBot/products/daily200648
OAI-SearchBot/pricing200402
Google-Extended/blog/sizing200220
PerplexityBot/pricing40396
In Trakkr

Asked in Trakkr

“Which AI crawlers hit our pricing pages last week?”

GPTBot1,204
OAI-SearchBot402
PerplexityBot40396

PerplexityBot is blocked on 38 pages

A robots.txt rule from Sep 28 turns it away. Trakkr drafted the fix.

A practical workflow.

Start with the source, check what it can show, then compare it with your AI visibility.

Before you start

A site proxied through Cloudflare and a read-only API token for the zone.

A clear list of high-value pages, docs, pricing pages, and policies AI systems should find.

A Trakkr brand with crawler tracking, visibility prompts, or citation monitoring enabled.

Open AI Crawl Control for the zone

Start with visibility before enforcement. Confirm Cloudflare is seeing AI crawler activity on the domain and note the time window you will compare in Trakkr.

Review AI traffic analytics

Inspect request paths, bot groupings, status and time windows. Keep the analysis page-path based, not just bot-name based.

Connect the read-only source

Connect Cloudflare to Trakkr with the requested read-only scope. Trakkr reads server-side analytics, which can be aggregated or sampled rather than exact request logs.

Compare crawler access to visibility outcomes

Map crawler hits to the pages Trakkr sees cited, recommended, or missing in AI answers. Look for important pages that never receive AI crawler requests.

Create an action loop

Use Trakkr actions and workflows to investigate request spikes, blocked paths, access gaps, and pages with request evidence but no separately observed citation.

Cloudflare is one of 29 integrations.See them all

What to measure.

Keep source signals, AI answers and business outcomes separate. Compare the same pages and time window.
SignalWhy it mattersCadence
AI crawler requests by pathShows whether crawlers reach the pages that matter for recommendations and citations.Weekly
User agents and bot familiesSeparates training, search, assistant, and preview traffic where the platform exposes it.Weekly
Response status and request timingHelps investigate bursts, blocks and response changes around launches.Daily during launches
Crawler access versus AI visibilityTurns raw crawl logs into a decision: fix access, improve content, or monitor the prompt set.Monthly

Turn the signal
into a next step.

Cloudflare supplies one part of the picture. Use Trakkr to investigate the answers around it.

Read crawler docs

How Trakkr fits

  1. Trakkr connects Cloudflare crawler signals to prompts, citations, competitors, and actions.
  2. Crawler spikes can become workflows for review, stakeholder updates, or technical fixes.
  3. Access checks keep robots.txt, WAF order, HTTP status and advisory llms.txt evidence separate.

Common mistakes

  • Blocking an AI crawler before you understand which pages it is requesting.
  • Treating every AI user agent as the same kind of traffic.
  • Looking only at total hits instead of the paths tied to visibility and citations.

Sources and
further reading.

Cloudflare’s own documentation first, then the Trakkr data, tools and guides that sit around it.

Related workflows

Vercel · Edge logsVercel AI crawler logsUse a Trakkr project Log Drain for request delivery. Treat Vercel Bot Management and its AI-bot ruleset as a separate control surface for log or deny policy.Open workflow Netlify · Bot trackingNetlify AI bot trackingTrakkr installs a Netlify Edge Function that records matching crawler requests before browser analytics would run. Upstream blocks remain outside its view.Open workflow Google Search Console · Search reportingGoogle Search Console AI visibilityGoogle is adding generative AI performance reporting for AI Overviews, AI Mode, and Discover generative features. Use it with Trakkr to connect search exposure to AI answer visibility.Open workflow

Questions,
answered.

The ones we hear most. Need a hand?
Talk to our team

Yes. Cloudflare AI Crawl Control exposes AI crawler analytics such as crawler groupings, user-agent signals, request paths and response outcomes. Detail and retention can vary by plan.

Ask anything else, like “What does Trakkr need from Cloudflare?”Ask anything else

The Agent answers from our docs, methods and pricing, and links to the source.

Start with crawler visibility, then decide what to control

Use Cloudflare for the raw request signal and Trakkr for the visibility, citation, and action layer around it.

Explore crawler monitoringRead crawler docs
Cloudflare is connectedJust now
Open AI Crawl Control for the zoneStart with visibility before enforcement2 min
Review AI traffic analyticsInspect request paths, bot groupings, status and time windowsRunning
Connect the read-only sourceConnect Cloudflare to Trakkr with the requested read-only scopeToday
Compare crawler access to visibility outcomesMap crawler hits to the pages Trakkr sees cited, recommended, or missing in AI answersThis week
Create an action loopUse Trakkr actions and workflows to investigate request spikes, blocked paths, access gaps, and pages with request evidence but no separately observed citationOngoing