Site files

Prepare or inspect robots.txt rules and sitemap XML from source material you supply. Check paths, entries and syntax with the relevant browser tool before placing the file on a site. A valid file does not guarantee crawling or indexing. Verify the published response and your intended exclusions after making a deployment.

9 tools in this category.

Available tools

Related guides