Skip to content

See where you stand

Change it

Works withWordPressShopifyWebflowGitHubGoogleSlackAll integrations
FixesFixes now ship to WordPress, Shopify, Webflow and GitHub

By team

By goal

Done for you

Not sure where to start? Take a guided tour.Book a demo
Customer storyHow the voice AI company Futuro grew its traffic3×AI traffic in 60 days

Learn

Data and research

FreeAI Site GraderAI Query Scannerllms.txt GeneratorAll free tools
Open dataWhere AI sends people, across more than 6,000 sites

Webz.ioTraining crawler

Webzio-Extended

Webzio-Extended is a Webz.io token covering whether crawled content may be included in the datasets it resells for AI and machine learning use.

Training only. It does not decide whether you appear in answers.

Get the robots.txt ruleWebz.io documentation
Is Webzio-Extended allowed on your site?
  • Webzio-ExtendedTraining
    Not checked

Reads your live robots.txt and applies the same rules these crawlers use. Nothing is stored.

robots.txt token
Webzio-Extended
Follows robots.txt
Yes, documented
How to verify it
Not published
Seen on
Not measured
Last checked
11 August 2026

Should you block Webzio-Extended?

Only if you want your pages kept out of training data. Blocking it signals that your content should not be included in the AI training datasets Webz.io sells on.

What blocking each Webz.io bot does
Webz.io botJobIf you block it
Webzio-ExtendedThis pageTrainingBlocking it signals that your content should not be included in the AI training datasets Webz.io sells on.
omgiliOtherYour content can enter downstream monitoring, research, and AI-training data products through Webz.io.
robots.txt
User-agent: Webzio-Extended
Disallow: /

Asks Webzio-Extended not to crawl any page. Other Webz.io bots are unaffected. Add it to the robots.txt file at the root of your site, above any wider rules.

Ignored robots.txt?

Also seen under Webz.io’s name, but not in its current documentation: omgilibot. Rules naming them control no documented Webz.io crawler.

Spot it in your logs.

Match the name, then check where the request came from. Anyone can copy a user-agent.

User-agentsource
Webzio-Extended

Version numbers change. Match the highlighted name, not the whole string.

  1. 1

    Match the name

    Search your server or CDN log for Webzio-Extended, case-insensitive. A match is a clue, not proof.

  2. 2

    Check the source

    Webz.io publishes no IP ranges, reverse DNS or signatures for Webzio-Extended in the sources here, so it cannot be verified. Treat matches as unconfirmed.

Serve the content in HTML

The cited operator material does not verify JavaScript rendering. Serve useful HTML before client-side JavaScript where possible.

Replaces the older omgili and omgilibot tokens. Webz.io publishes no exact user-agent string, so only the token is recorded here.

Questions, answered.

Webzio-Extended is a Webz.io token covering whether crawled content may be included in the datasets it resells for AI and machine learning use.

Sources.

Officially documented. Operator facts and Trakkr’s measurements keep separate dates.

  1. [1]
    Webz.io crawler documentation

    Officially documented · checked 11 August 2026

    webz.io/blog/company/an-overview-of-the-webz-io-duo-of-crawlers/

First verified 11 August 2026. Last source check 11 August 2026.

All AI crawlers

See which AI crawlers read your site.

Trakkr shows which bots, Webzio-Extended included, visit which pages each day, and whether the answers they feed mention you.

14-day free trial. Cancel anytime.