Skip to content

See where you stand

Change it

Works withWordPressShopifyWebflowGitHubGoogleSlackAll integrations
FixesFixes now ship to WordPress, Shopify, Webflow and GitHub

By team

By goal

Done for you

Not sure where to start? Take a guided tour.Book a demo
Customer storyHow the voice AI company Futuro grew its traffic3×AI traffic in 60 days

Learn

Data and research

FreeAI Site GraderAI Query Scannerllms.txt GeneratorAll free tools
Open dataWhere AI sends people, across more than 6,000 sites

GoogleLive fetcher

Google-Agent

Google-Agent is a Google user-triggered fetcher used by agents that act on a person’s request, including Project Mariner.

Reads a page when someone asks Gemini about it.

Get the robots.txt ruleGoogle documentation
Is Google-Agent allowed on your site?4 Google bots

Reads your live robots.txt and applies the same rules these crawlers use. Nothing is stored.

robots.txt token
Google-Agent
Follows robots.txt
Not always
Last checked
18 August 2026

Should you block Google-Agent?

Usually not. It reads a page only when a person asks Gemini about it, so blocking it hides your page from someone already looking for it.

What blocking each Google bot does
Google botJobIf you block it
Google-ExtendedControl tokenDisallowing it limits future Gemini training and grounding uses covered by the token. Google says it does not affect Search inclusion or ranking.
GooglebotSearchBlocking Googlebot can remove pages from Google Search and eligibility for Google AI features that depend on Search indexing.
Google-AgentThis pageUser fetchBlocking it can prevent a Google agent from retrieving a page while completing a user-directed task. It is separate from Google Search crawling.
Google-GeminiNotebookUser fetchA hit usually means a person added your page as a source, not that Google is crawling your site broadly.
Google-CloudVertexBotOtherA hit usually relates to a configured Vertex AI workflow rather than broad public search crawling.
GoogleOtherOtherBlocking it may affect Google product-specific crawling but is separate from primary Search indexing policy.
GoogleOther-ImageOtherBlocking it can limit image-oriented Google product crawls without targeting regular Googlebot directly.
GoogleOther-VideoOtherBlocking it can limit video-oriented Google product crawls without targeting regular Googlebot directly.

From Google’s documentationGoogle says user-triggered fetchers generally ignore robots.txt, because a person asked for the page.

About robots.txtGoogle classifies this as a user-triggered fetcher, so automated robots.txt rules may not apply in the same way as they do to common crawlers.

robots.txt
User-agent: Google-Agent
Disallow: /

Asks Google-Agent not to crawl any page. Other Google bots are unaffected. Add it to the robots.txt file at the root of your site, above any wider rules.

Ignored robots.txt?

Google-Agent access decides whether Google can fetch your pages. Whether those pages get used in generated answers is a separate question, so track your presence in Google's AI results rather than treating crawl volume as visibility.

Also seen under Google’s name, but not in its current documentation: GoogleAgent-Mariner, Gemini-Deep-Research, Google-NotebookLM, Google-Gemini-CLI, Google-Firebase. Rules naming them control no documented Google crawler.

Spot it in your logs.

Match the name, then check where the request came from. Anyone can copy a user-agent.

Desktop agent fetchersource
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Safari/537.36; compatible; Google-Agent

Version numbers change. Match the highlighted name, not the whole string.

Mobile agent fetchersource
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36; compatible; Google-Agent

Version numbers change. Match the highlighted name, not the whole string.

  1. 1

    Match the name

    Search your server or CDN log for Google-Agent, case-insensitive. A match is a clue, not proof.

  2. 2

    Check the published user-triggered fetcher IP ranges

    Google also documents experimental Web Bot Auth signatures for a subset of requests. Not every request is signed. developers.google.com/static/search/apis/ipranges/user-triggered-fetchers.json

Is this really Google-Agent?

Addresses and reverse DNS

Paste an address that claims to be Google-Agent. It is checked against Google’s published list and its reverse DNS. Nothing is stored.

Check any crawler’s address

Serve the content in HTML

The cited operator material does not verify JavaScript rendering. Serve useful HTML before client-side JavaScript where possible.

How active is Google-Agent?

Measured across the sites connected to Trakkr, 27 July 2026 to 25 October 2026.

18%

of connected sites saw Google-Agent in the four weeks to 25 October 2026. That is 19 of 104.

100%75%50%25%8%
Google-Agent visited between 4% and 13% of connected sites each week from 27 July 2026 to 25 October 2026.

Share of connected sites it visited each week

Rank by reach
20 of 26
Typical site, per week
10 requests
Sites reached, 13 weeks
33 of 135

Most widespread AI crawlers

Share of connected sites each one visited, last four weeks.

  1. 1OAI-SearchBot84%
  2. 2ClaudeBot84%
  3. 3GPTBot67%
  4. 4Amazonbot61%
  5. 5ChatGPT-User56%
  6. 6PerplexityBot56%
  7. 7Applebot51%
  8. 20Google-Agent18%
All 26 measured crawlers
How this is measured

Finalized daily crawler summaries from sites connected to Trakkr, grouped by the crawler named in each request. Trakkr’s own site is excluded. A site counts as reached in a week when it logged at least one request from Google-Agent. The typical figure is the median weekly requests among the sites it reached.

  • This is a sample of connected sites, not the whole web.
  • A matching user-agent does not prove the named operator sent the request.
  • Requests are not unique pages, training use or citations.
  • Crawlers seen on fewer than 10 sites are not shown. Classic search crawlers such as Googlebot are not collected on every site, so they are left out.

Questions, answered.

Google-Agent is a Google user-triggered fetcher used by agents that act on a person's request, including Project Mariner.

Sources.

Officially documented. Operator facts and Trakkr’s measurements keep separate dates.

  1. [1]
    Google crawler documentation

    Officially documented · checked 18 August 2026

    developers.google.com/crawling/docs/crawlers-fetchers/google-user-triggered-fetchers#google-agent

First verified 18 August 2026. Last source check 18 August 2026. Activity measured to 25 October 2026.

Changes

Google refreshed its crawler referenceThe current reference separates common crawlers, special-case crawlers, user-triggered fetchers, and product control tokens.
All AI crawlers

See which AI crawlers read your site.

Trakkr shows which bots, Google-Agent included, visit which pages each day, and whether the answers they feed mention you.

14-day free trial. Cancel anytime.