What’s up with Yahoo!’s Slurp?
That’s the question I was asking myself last week as I was peering at my blog stats. More specifically, the stats for which robots had been visiting, sucking, spidering my sites.
Yahoo! Slurp is visiting my sites almost constantly … in this 7-day period it requested a file (image, page, you name it) 12,521 times. That’s about 9000 more hits than the Googlebot.
Here’s the graph. (Note: this is only the Mozilla-compatible spiders, which in total account for about half of the spidering on my sites.)
There are a ton of bots that hit any blogs that have more than 20 posts and a few links to them. I think I once counted 50 or 60 – and that was a year or more ago.
But why would Yahoo! Slurp account for 65% of the spider hits to my site? During the same period, the MSN bot accounted for about 1200 hits. I confess I don’t get it.
Somehow, Yahoo! wants to know everything, immediately, repeatedly. That’s OK, I’m just wondering why.
. . .
. . .
Interestingly enough, the Technorati bot is almost nowwhere to be seen, with something like 90 hits on my site in the 7-day period.
[tags] technoratibot, yahoo slurp, googlebot, spiders, MSNbot, john koetsier [/tags]