What is PerplexityBot?
PerplexityBot is Perplexity's web crawler that retrieves content for real-time answers. Learn how it works and whether to allow or block it.
PerplexityBot crawls websites to power Perplexity's answer engine, which serves over 15 million monthly users. Unlike traditional search crawlers that just index pages, PerplexityBot retrieves content that gets directly quoted and cited in AI-generated responses. Allowing PerplexityBot means your content can appear as a source in Perplexity answers.
Why it matters
PerplexityBot represents the clearest example of how AI search creates new visibility dynamics. With 15M+ monthly active users and growing, Perplexity is becoming a meaningful traffic and brand awareness channel. The decision to allow or block PerplexityBot is a strategic one. Unlike training crawlers where the value exchange is unclear, PerplexityBot offers direct attribution: your content gets cited, your brand gets mentioned, and you can track it happening. For brands building AI visibility strategies, Perplexity provides the clearest feedback loop on what content performs in AI contexts.
What people get wrong
- PerplexityBot trains AI models with your content. PerplexityBot is a retrieval crawler, not a training crawler. It fetches content to generate real-time answers, similar to how Bing fetches pages for search results. Your content isn't used to train Perplexity's underlying models.
- Blocking PerplexityBot has no downside. Unlike blocking training crawlers like GPTBot, blocking PerplexityBot removes your content from a citation-driven answer engine. You lose visibility in one of the fastest-growing AI search platforms, which may matter more as Perplexity's user base grows.
- PerplexityBot ignores robots.txt. Perplexity has publicly committed to honoring robots.txt directives for PerplexityBot. While there have been controversies about edge cases, the standard robots.txt disallow directive works for controlling access.
Related terms
AI Crawlers: PerplexityBot is one of several AI crawlers alongside GPTBot and ClaudeBot, but differs in being retrieval-focused rather than training-focused.
Perplexity: PerplexityBot is the crawler that powers Perplexity's answer engine, fetching content that appears in Perplexity's cited responses.
AI Citations: PerplexityBot enables AI citations by retrieving content that Perplexity then references in its answers with direct URL attribution.
Track when PerplexityBot leads to citations
Trakkr monitors when your brand and content appear in Perplexity answers, connecting the dots between PerplexityBot crawling your content and actual citations in user queries. See which pages get cited, how often, and for what types of questions. This helps you understand the ROI of allowing PerplexityBot and optimize content that performs well in AI answer contexts.
Frequently Asked Questions
What is PerplexityBot?
PerplexityBot is the web crawler operated by Perplexity AI to fetch content for its answer engine. When users ask Perplexity questions, PerplexityBot retrieves relevant web pages so Perplexity can synthesize answers and cite sources. It identifies itself with the user-agent string "PerplexityBot" and respects robots.txt.
How do I block PerplexityBot?
Add these lines to your robots.txt file: "User-agent: PerplexityBot" followed by "Disallow: /" on the next line. To also block user-initiated research queries, add the same for "User-agent: PerplexityBot-User". The crawler checks robots.txt and should stop crawling within a few days.
Should I allow or block PerplexityBot?
It depends on your priorities. Allowing PerplexityBot means your content can be cited in Perplexity answers, providing brand visibility and potential referral traffic. Blocking prevents this visibility but also stops Perplexity from summarizing your content. Unlike training crawlers, the value exchange here is more direct and measurable.
What's the difference between PerplexityBot and GPTBot?
PerplexityBot fetches content for real-time answer generation with citations, while GPTBot crawls for OpenAI's model training. PerplexityBot creates immediate, trackable visibility through citations. GPTBot's impact is indirect and harder to measure since it feeds training data rather than generating cited answers.
Does PerplexityBot respect rate limits?
Yes, PerplexityBot honors crawl-delay directives in robots.txt. If you're experiencing heavy crawling, add "Crawl-delay: 10" (or your preferred seconds) under the PerplexityBot user-agent rules. Perplexity has also implemented server-side rate limiting following publisher feedback.
How can I see if PerplexityBot is crawling my site?
Check your server access logs for the user-agent string "PerplexityBot" or "PerplexityBot-User". Most analytics platforms also show bot traffic if you have bot filtering disabled. You'll see requests from Perplexity's documented IP ranges accessing your pages.