Run the Robots Txt Checker on TagBuilder.io for a focused website, SEO, domain, header or technical diagnostic.