◧ DEVELOPER Updated July 25, 2026

Robots.txt Generator

Write a correct robots.txt without memorising the syntax.

Paths to block (one per line, prefix with ! to allow)
Sitemap URL
Crawl delay (seconds, 0 = none)
robots.txt
Ad space · belowTool · horizontal

How it works

Each path becomes a Disallow line under a wildcard user-agent, paths prefixed with an exclamation mark become Allow overrides, and the sitemap reference is appended at the end where crawlers expect it.

Related tools

◨Base64 Encoder / Decoder◧JSON Formatter◪JWT Decoder

Frequently asked questions

Does robots.txt hide a page from Google?

No. It stops crawling, not indexing — use a noindex meta tag to keep a page out of results.

Where does the file go?

At the root of the domain, exactly at /robots.txt; crawlers do not look anywhere else.

Comments

Comments aren't enabled yet. When you're ready, this is where a service like Giscus, Disqus, or a custom form would go — the slot is already reserved on every tool page.