robots.txt Generator
Marketing & GrowthEmit a robots.txt with a sitemap line and optional AI-crawler blocks.
Calculated Result
User-agent: *
Allow: /
User-agent: GPTBot
Disallow: /
User-agent: CCBot
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: anthropic-ai
Disallow: /
Sitemap: https://calnora.site/sitemap.xml
Formula: User-agent groups + optional Sitemap
About this calculator
Disallow empty means allow all for * . Optional GPTBot and similar rows are a policy choice, not a guarantee crawlers obey.
How It Works & Formula
FormulaUser-agent groups + optional Sitemap
Concatenate User-agent / Disallow blocks plus Sitemap.
Related Calculators
View All (17)robots.txt & Sitemap Viewer
Read robots.txt and sample linked sitemaps.
UTM Builder
Append utm_source, medium, campaign, term, and content.
JSON-LD Schema Generator
Fill Article, FAQPage, or Organization JSON-LD.
CAC & LTV Calculator
Analyze Customer Acquisition Cost vs Lifetime Value and discover your business unit economics health ratio.