[geeklog-devel] Yahoo bot becoming a pain

[Oliver Spiesshofer]

Yahoo got through to my site now:

May Full Month (Hits    MB)
Yahoo Slurp    11,454    49.78 MB
Googlebot    11,369    97.45 MB

June 1-13
Yahoo Slurp    7,085    33.98 MB
Googlebot    5,820    22.02 MB

500 hits of the 7,085 came in just yesterday.

Oliver

Web Site Master wrote:

> I am having the same issue with one of my sites. Yesterday and last

> Wednesday the Yahoo bot increased my traffic by over 25% and was mainly

> going after the forum.

>

> -----Original Message-----

> From: geeklog-devel-bounces at lists.geeklog.net

> [mailto:geeklog-devel-bounces at lists.geeklog.net] On Behalf Of Dirk Haun

> Sent: June 8, 2007 4:50 PM

> To: geeklog-devel

> Subject: Re: [geeklog-devel] Yahoo bot becoming a pain

>

> Dirk Haun wrote:

>

>   

>> This bot has gone completely bonkers ...

>>     

>

> Still no change, btw. Since the majority of its requests are for /forum/

> viewtopic.php, I've modified that file now to give Slurp a 503. Doesn't seem

> to impress it too much yet:

>

> 74.6.69.31 - - [08/Jun/2007:16:35:39 -0400] "GET /forum/viewtopic.php?

> forum=3&showtopic=47113 HTTP/1.0" 503 0 "-" "Mozilla/5.0 (compatible; Yahoo!

> Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"

> 74.6.21.164 - - [08/Jun/2007:16:35:39 -0400] "GET /forum/viewtopic.php?

> showtopic=14219 HTTP/1.0" 503 0 "-" "Mozilla/5.0 (compatible; Yahoo!

> Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"

> 74.6.18.103 - - [08/Jun/2007:16:35:39 -0400] "GET /forum/viewtopic.php?

> forum=3&showtopic=31759 HTTP/1.0" 503 0 "-" "Mozilla/5.0 (compatible; Yahoo!

> Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"

>

> That was the best I could think of, short of blocking it entirely. And who

> wants to block the #2 search engine, even if it's a pain? :-/

>

> I hope it slows down a bit eventually, so that I can let at least get some

> requests through again ...

>

> bye, Dirk

>

>

> --

> http://www.geeklog.net/

> http://geeklog.info/

>

> _______________________________________________

> geeklog-devel mailing list

> geeklog-devel at lists.geeklog.net

> http://eight.pairlist.net/mailman/listinfo/geeklog-devel

>

> __________ NOD32 2319 (20070608) Information __________

>

> This message was checked by NOD32 antivirus system.

> http://www.eset.com

>

>

> _______________________________________________

> geeklog-devel mailing list

> geeklog-devel at lists.geeklog.net

> http://eight.pairlist.net/mailman/listinfo/geeklog-devel

>

>

>