Robots.txt generator

Build a practical robots.txt file for small and medium websites: choose a user-agent, list Allow and Disallow paths, then append Host and multiple Sitemap URLs. Output is generated locally and ready to copy.

Updated: 2026-05-05
Your robots rules are processed locally in the browser. This static page does not upload your input.

User-agent Disallow paths (one per line) Allow paths (optional, one per line) Host (optional) Sitemap URLs (one per line)

Robots.txt controls crawling behavior, not access control for private data.

Output Generated robots.txt will appear here.

Related SEO tools

Meta Robots Tag Generator

XML Sitemap Generator

Sitemap Index XML Generator

Hreflang Tag Generator

Privacy And Limitations

Your robots rules are processed locally in the browser. This static page does not upload your input.

robots.txt is only a crawl directive, not an access control mechanism for sensitive content.
Search engines can interpret directives differently, so always verify behavior in Search Console and crawler logs.
Incorrect wildcards or broad Disallow rules can accidentally block important pages from crawling.

FAQ

Does robots.txt hide private pages?

No. robots.txt requests crawler behavior but does not protect URLs from direct access.

Can I add more than one Sitemap line?

Yes. It is common to add multiple sitemap files or sitemap indexes.

Should I block all query parameters?

Not blindly. Parameter crawling strategy depends on your site architecture and canonicalization setup.

Do all bots follow robots.txt?

Major search engines usually do, but malicious bots may ignore it.

Is this file generated server-side?

No. The output is assembled in your browser.