Robots.txt Generator

Generate a robots.txt file for your website. Free online robots.txt generator with allow/disallow rules and sitemap support.

Allow paths (one per line)

Disallow paths (one per line)

Sitemap URL

User-agent: *
Allow: /
Disallow: /admin/
Disallow: /private/

Sitemap: https://example.com/sitemap.xml

Frequently Asked Questions

What does robots.txt do?

Robots.txt tells search engine crawlers which pages or sections of your site they can or cannot access.