Check the list below. What’s the common thread?
Googlebot Yahoo! Slurp YahooFeedSeeker/1.0 Mozilla Aggregator MSNbot AppleSyndication YouReadMe Feedeater 1.0 ZyBorg/1.0 Dead Link Checker (wn.dlc@looksmart.net; http://www.WISEnutbot.com) NetNewsWire/2.0 Bloglines/2.0 kinjabot (http://www.kinja.com) Lektora 1.0 Feedster Crawler/1.0; Feedster, Inc. Mediapartners-Google/2.1 (if you run Google ads, you’ll see this) lmspider lmspider@scansoft.com A2B Location-Based Search Engine (+http://www.a2b.cc) BlogSearch/1.1 +http://www.icerocket.com/ GeoURL bot (http://geourl.org/bot) Blogslive (info@blogslive.com) OmniExplorer_Bot/1.09 (+http://www.omni-explorer.com) Internet Categorizer
(this is a nasty spider: it hit something like 150 pages at once)SharpReader/0.9.5.1 (.NET CLR 1.1.4322.573; WinNT 5.1.2600.0)
They’re all either spiders or link aggregators or RSS agents of some kind that I found in my user agent webstats with just a cursory glance. The interesting thing is that some days I get more traffic from these than from actual people using actual web browsers, actually surfing the web.
Interesting.
PS: While checking the user agents, I was shocked and thrilled that someone (or someones) uses Lynx to access sparkplug9.com. Trés cool!