Robots.txt Rule Tester
Test whether a robots.txt rule allows or blocks a given URL before you rely on it.
Checking a rule does what you think
Robots.txt rules use wildcards and path matching that are easy to get subtly wrong — a rule you believe blocks one folder might block far more, or nothing at all. This checks a specific URL against your rules and tells you whether it would be allowed or disallowed, so you can confirm the behaviour before a crawler acts on it.
Enter your rules and a URL, and it reports whether that URL is blocked.
Where the surprises hide
The common traps are in the matching. Order and specificity matter, wildcards match more than people expect, and a missing or stray slash changes everything. It is also easy to accidentally block your CSS and JavaScript, which stops Google rendering your page properly. Testing individual URLs — especially important ones and ones near a disallow rule — is the way to catch a rule that blocks too much before it quietly costs you traffic.
Questions & answers
Why test a robots.txt rule?
Because path matching and wildcards are easy to get wrong, and a rule can block far more, or less, than you intended.
What is a common robots.txt mistake?
Accidentally blocking CSS and JavaScript, which stops Google rendering the page, or a stray slash that changes what a rule matches.
Is anything uploaded?
No. The test runs in your browser.