What is Perplexity-User? AI crawler guide
Learn what Perplexity-User is, who operates it, its verified user-agent, robots.txt posture, and how blocking it can affect AI search, citations, training, or agent visibility.
User-triggered Perplexity fetcher for pages needed to answer a specific question.
What is Perplexity-User?
Perplexity-User is a web fetcher operated by Perplexity that retrieves pages in response to a specific user question. It is triggered when a person asks Perplexity something and the system needs to fetch live web content to formulate an answer. The fetcher identifies itself with the user-agent token Perplexity-User. According to Perplexity, this bot generally ignores robots.txt rules because the request is considered user-initiated rather than autonomous crawling. It is not a traditional search crawler; it only visits pages that are directly relevant to a query someone has just asked.
What it's for
For a site owner, a visit from Perplexity-User means your content is being retrieved to help answer a real-time user question on Perplexity. This can drive visibility and traffic if your page is cited in the answer. Because the fetch is user-driven, it may occur even if you have blocked other crawlers. The impact is immediate and contextual, potentially surfacing your content to users who might not otherwise find it through conventional search.
How to handle Perplexity-User
Perplexity states that Perplexity-User generally ignores robots.txt. Adding a Disallow rule for this token may not prevent fetches. The page will render a robots.txt snippet for reference, but its effectiveness is limited for this bot.
robots.txt rule
User-agent: Perplexity-User Disallow: /
Blocking cost
Blocking Perplexity-User may prevent your content from being used in Perplexity answers, potentially reducing visibility and referral traffic from user queries.
Examples
- A user asks Perplexity for the latest weather forecast, and Perplexity-User fetches a weather service page to extract current conditions.
- Someone queries Perplexity about a historical event, prompting Perplexity-User to retrieve a relevant encyclopedia article.
- A developer asks Perplexity for documentation on a Python library, and Perplexity-User fetches the official docs page to provide code examples.
Related bots
- ChatGPT-User: Also tracked as a interaction crawler.
- MistralAI-User: Also tracked as a interaction crawler.
- Meta-ExternalFetcher: Also tracked as a interaction crawler.
- Claude-User: Also tracked as a interaction crawler.
- Google-NotebookLM: Also tracked as a interaction crawler.
- ChatGPT Agent: Also tracked as a interaction crawler.
- Claude-Code: Also tracked as a interaction crawler.
- cohere-ai: Also tracked as a interaction crawler.
- DuckAssistBot: Also tracked as a interaction crawler.
- Perplexity: Perplexity-User connects this operator term to its crawler behavior.
- PerplexityBot: Perplexity-User connects this operator term to its crawler behavior.
Frequently Asked Questions
Does Perplexity-User obey robots.txt?
Perplexity documents that Perplexity-User generally ignores robots.txt rules because the request is user-initiated.
How can I block Perplexity-User?
Since Perplexity-User typically ignores robots.txt, blocking via that method may not work.
What is the difference between Perplexity-User and other Perplexity crawlers?
Perplexity-User is specifically for user-triggered fetches when answering a question, while other Perplexity bots may perform autonomous crawling for indexing or other purposes.
Will blocking Perplexity-User affect my site's visibility in Perplexity?
Yes, blocking Perplexity-User could prevent your content from being retrieved and cited in Perplexity answers, potentially reducing referral traffic.
Is Perplexity-User a search engine crawler?
No, Perplexity-User is not a traditional search engine crawler. It only fetches pages on demand when a user asks a specific question that requires live web content.
Data & Sources
- Perplexity documentation - Primary source for Perplexity-User crawler details.
- Perplexity-User live crawler data - Trakkr crawler telemetry for this user agent.