# The robots.txt file is used to control how search engines index your live URLs. # See http://www.robotstxt.org/wc/norobots.html for more information. # To prevent search engines from seeing the site altogether, uncomment the next two lines: User-Agent: * crawl-delay: 6 Sitemap: https://www.marketcheck.com/static/sitemap_used_cars_top_1000.xml Sitemap: https://www.marketcheck.com/static/sitemap_new_cars_top_1000.xml