 |
 |
A robots.txt file will take precedence over sitemaps. It's also worth noting that inclusion in a sitemap doesn't guarantee it will be included in the index but matching against robots.txt will (eventually) lead to a page's removal. Cheers, Mike >>> "Simon Marshall [som]" <[log in to unmask]> 18/12/2009 13:12 >>> Hi all, Does anyone know about the relationship between Google Sitemaps and the robots.txt file? Specifically, when a site is being spidered does the data in the sitemap over-ride the robots.txt file? We have an automatically generated sitemap which includes some pages that *are* in our robots.txt file. I'm trying to figure out whether we're effectively over-riding our own robots.txt in the context of Google (or any other search engine that can use our map). Cheers, Simon -- Simon Marshall Swyddog Datblygu'r We Prifysgol Aberystwyth E-bost: [log in to unmask] Ffon: (01970) 62 2459 Ffacs: (01970) 62 1554 -- Simon Marshall Web Development Officer Aberystwyth University E-mail: [log in to unmask] Tel: (01970) 62 2459 Fax: (01970) 62 1554 ----------------------------------------------------- This email and any attachments are confidential and intended solely for the use of the individual to whom it is addressed. Any views or opinions presented are solely those of the author and do not necessarily represent those of Edge Hill University or associated companies. If you are not the intended recipient, please contact the sender as soon as possible and delete it and all copies of it. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. The message content of in-coming emails is automatically scanned to identify Spam and viruses otherwise Edge Hill University do not actively monitor content. However, sometimes it will be necessary for Edge Hill University to access business communications during staff absence. Edge Hill University has taken steps to ensure that this email and any attachments are virus free. However, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by Edge Hill University for any loss or damage arising in any way from its use. -----------------------------------------------------