🤖 robots.txt Checker
Enter a domain to fetch its live robots.txt, or paste one. Then test whether a path is allowed for a given crawler and get basic sanity linting. Parsing and testing run entirely in your browser.
1. robots.txt content
https://<domain>/robots.txt into the box below. Or paste your own.
2. Test a request
Just the path (and optional query string), e.g.
/private/file.pdf — no scheme or host.
Matched case-insensitively as a prefix against each
User-agent: line, falling back to * if nothing matches.
Paste a robots.txt and enter a path to test.