🤖 robots.txt Checker
Paste a robots.txt file, test whether a path is allowed for a given crawler, and get basic sanity linting. Entirely client-side.
1. robots.txt content
2. Test a request
Just the path (and optional query string), e.g.
/private/file.pdf — no scheme or host.
Matched case-insensitively as a prefix against each
User-agent: line, falling back to * if nothing matches.
Paste a robots.txt and enter a path to test.