Skip to content

See where you stand

Change it

Works withWordPressShopifyWebflowGitHubGoogleSlackAll integrations
FixesFixes now ship to WordPress, Shopify, Webflow and GitHub

By team

By goal

Done for you

Not sure where to start? Take a guided tour.Book a demo
Customer storyHow the voice AI company Futuro grew its traffic3×AI traffic in 60 days

Learn

Data and research

FreeAI Site GraderAI Query Scannerllms.txt GeneratorAll free tools
Open dataWhere AI sends people, across more than 6,000 sites

xAIAI search crawler

xAI-SearchBot

xAI-SearchBot is a crawler token observed retrieving pages for xAI’s Grok search and answer features.

No current xAI operator page was found for this crawler name. Treat the string as reported until xAI publishes a crawler reference.

Get the robots.txt ruleRecord source
Is xAI-SearchBot allowed on your site?
  • xAI-SearchBotSearch
    Not checked

Reads your live robots.txt and applies the same rules these crawlers use. Nothing is stored.

robots.txt token
xAI-SearchBot
Follows robots.txt
Not documented
How to verify it
Not published
Seen on
Not measured
Last checked
11 August 2026

Should you block xAI-SearchBot?

You can, but it may not listen. xAI has not documented xAI-SearchBot, so treat any rule as a request. Blocking it may reduce whether your pages can be retrieved and cited in Grok answers.

About robots.txtxAI has not documented whether xAI-SearchBot follows robots.txt, so a rule is a request, not a guarantee.

robots.txt
User-agent: xAI-SearchBot
Disallow: /

Asks xAI-SearchBot not to crawl any page. Add it to the robots.txt file at the root of your site, above any wider rules.

Ignored robots.txt?

Spot it in your logs.

Match the name, then check where the request came from. Anyone can copy a user-agent.

Reported user-agentsource
Mozilla/5.0 (compatible; xAI-SearchBot/1.0; +https://x.ai)

This string comes from a public crawler registry or observed naming, not current operator documentation.

  1. 1

    Match the name

    Search your server or CDN log for xAI-SearchBot, case-insensitive. A match is a clue, not proof.

  2. 2

    Check the source

    xAI publishes no IP ranges, reverse DNS or signatures for xAI-SearchBot in the sources here, so it cannot be verified. Treat matches as unconfirmed.

Serve the content in HTML

The cited operator material does not verify JavaScript rendering. Serve useful HTML before client-side JavaScript where possible.

xAI publishes no crawler documentation, so this user-agent is recorded from observed requests rather than an operator page.

Questions, answered.

xAI-SearchBot is a crawler token observed retrieving pages for xAI's Grok search and answer features. This record has no current xAI source, so treat a matching request as unverified.

Sources.

Reported, not operator verified. Operator facts and Trakkr’s measurements keep separate dates.

  1. [1]
    Reported crawler reference

    Reported, not operator verified · checked 11 August 2026

    github.com/ai-robots-txt/ai.robots.txt/blob/main/table-of-bot-metrics.md

Last source check 11 August 2026.

All AI crawlers

See which AI crawlers read your site.

Trakkr shows which bots, xAI-SearchBot included, visit which pages each day, and whether the answers they feed mention you.

14-day free trial. Cancel anytime.