# BEGIN JCAWLER-XML-SITEMAP-COMPONENT Sitemap: http://www.agenciabanana.com/sitemap.xml # END JCAWLER-XML-SITEMAP-COMPONENT sitemap: http://cdn.attracta.com/sitemap/569617.xml.gz #Wildcard robots.txt agent (ask all bots to crawl slowly but still index everything). User-agent: * Crawl-delay: 1 Disallow: