Skip to content

See where you stand

Change it

Works withWordPressShopifyWebflowGitHubGoogleSlackAll integrations
FixesFixes now ship to WordPress, Shopify, Webflow and GitHub

By team

By goal

Done for you

Not sure where to start? Take a guided tour.Book a demo
Customer storyHow the voice AI company Futuro grew its traffic3×AI traffic in 60 days

Learn

Data and research

FreeAI Site GraderAI Query Scannerllms.txt GeneratorAll free tools
Open dataWhere AI sends people, across more than 6,000 sites

CloudflareAI search crawler

Cloudflare-AutoRAG

Cloudflare-AutoRAG is a Cloudflare AutoRAG crawler used to index configured content for AI search applications.

Needed to be found in its search index.

Get the robots.txt ruleCloudflare documentation
Is Cloudflare-AutoRAG allowed on your site?
  • Cloudflare-AutoRAGSearch
    Not checked

Reads your live robots.txt and applies the same rules these crawlers use. Nothing is stored.

robots.txt token
Cloudflare-AutoRAG
Follows robots.txt
Yes, documented
How to verify it
Not published
Seen on
Not measured
Last checked
11 June 2026

Should you block Cloudflare-AutoRAG?

Not if you want to be found. A hit usually reflects a site or app configured to use Cloudflare AutoRAG over that content.

robots.txt
User-agent: Cloudflare-AutoRAG
Disallow: /

Asks Cloudflare-AutoRAG not to crawl any page. Add it to the robots.txt file at the root of your site, above any wider rules.

Ignored robots.txt?

Whether to allow Cloudflare-AutoRAG is a policy call. Either way, check which AI platforms actually send you human visitors before assuming crawl volume equals value.

Spot it in your logs.

Match the name, then check where the request came from. Anyone can copy a user-agent.

User-agentsource
Cloudflare-AutoRAG

Version numbers change. Match the highlighted name, not the whole string.

  1. 1

    Match the name

    Search your server or CDN log for Cloudflare-AutoRAG, case-insensitive. A match is a clue, not proof.

  2. 2

    Check the source

    Cloudflare publishes no IP ranges, reverse DNS or signatures for Cloudflare-AutoRAG in the sources here, so it cannot be verified. Treat matches as unconfirmed.

Serve the content in HTML

The cited operator material does not verify JavaScript rendering. Serve useful HTML before client-side JavaScript where possible.

Questions, answered.

Cloudflare-AutoRAG is a Cloudflare AutoRAG crawler used to index configured content for AI search applications.

Sources.

Officially documented. Operator facts and Trakkr’s measurements keep separate dates.

  1. [1]
    Cloudflare crawler documentation

    Officially documented · checked 11 June 2026

    developers.cloudflare.com/autorag/

First verified 11 June 2026. Last source check 11 June 2026.

All AI crawlers

See which AI crawlers read your site.

Trakkr shows which bots, Cloudflare-AutoRAG included, visit which pages each day, and whether the answers they feed mention you.

14-day free trial. Cancel anytime.