Robots.txt
A file that tells search engine crawlers which URLs they can or cannot access.
Topics: SEO and CRO
Browse definitions. Filter by starting letter or by topic.
A file that tells search engine crawlers which URLs they can or cannot access.
Topics: SEO and CRO