Can AI crawlers read your site?
Checks robots.txt for every known AI crawler, then fetches the page with that crawler's user agent to see what actually happens.
No sign-up← All tools
A robots.txt that allows a crawler is not the same as a page that crawler can use. This check reads the rules first, then fetches the URL with the crawler's own user agent and reports the status, the word count and any block page it meets.