← Back to Tools
Free Tool
robots.txt Generator
Build a modern robots.txt with CMS presets, AI training vs retrieval controls, multi-bot rules, and an in-browser RFC 9309 URL tester. Nothing is uploaded.
Quick presets
Apply CMS path templates or AI / SEO bot policies. You can edit anything after.
CMS / platform
AI & SEO policies
Default policy
Rules for User-agent: *. Specific bots below can override this.
Quick add
Sitemap URLs optional · one per row
Bot catalog
Override the default for search, AI training, AI retrieval, SEO, and social crawlers. Only bots that differ from the default emit their own block.
Draft & share
Autosaved in this browser. Export or import JSON to move a full config between machines.