The Robots.txt file is an important tool for controlling which pages on your site search engines can crawl. This simple text file, located in the foundation Listing within your site, offers Guidance to Internet robots like Google’s crawler. When your website is situated in a country other than the US, not an issue! Just pick your country through