This may interest webmasters:
Search engine robots that visit your web site
http://www.jafsoft.com/searchengines/webbots.html
'Search engines and other sites send robots to read and index your pages.
This page
reverses that process and indexes the robots. This information has been
gleaned by looking at the server logs for www.jafsoft.com. Whenever a page
is
read from a web site, the log file records a number of details including the
time, the IP address and usually the referrer page and the user agent. You
can see this in our analysis of a server log sample.
Unlike many pages that list web robots, this page actually tries to
go visit the robots themselves. Where possible links are provided to the
robots home pages, and descriptions are given of what they're up to.'
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Ian Winship
Learning Resources, University of Northumbria at Newcastle
City Campus Library, Newcastle upon Tyne NE1 8ST, UK
----------------
e-mail: [log in to unmask]
phone: 0191 227 4150 fax: 0191 227 4563
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|