[geeklog-devel] Yahoo bot becoming a pain

[Oliver Spiesshofer]

Yahoo got through to my site now:

May Full Month (Hits    MB)
Yahoo Slurp    11,454    49.78 MB
Googlebot    11,369    97.45 MB

June 1-13
Yahoo Slurp    7,085    33.98 MB
Googlebot    5,820    22.02 MB

500 hits of the 7,085 came in just yesterday.

Oliver

Web Site Master wrote:
> I am having the same issue with one of my sites. Yesterday and last
> Wednesday the Yahoo bot increased my traffic by over 25% and was mainly
> going after the forum.
>
> -----Original Message-----
> From: geeklog-devel-bounces at lists.geeklog.net
> [mailto:geeklog-devel-bounces at lists.geeklog.net] On Behalf Of Dirk Haun
> Sent: June 8, 2007 4:50 PM
> To: geeklog-devel
> Subject: Re: [geeklog-devel] Yahoo bot becoming a pain
>
> Dirk Haun wrote:
>
>   
>> This bot has gone completely bonkers ...
>>     
>
> Still no change, btw. Since the majority of its requests are for /forum/
> viewtopic.php, I've modified that file now to give Slurp a 503. Doesn't seem
> to impress it too much yet:
>
> 74.6.69.31 - - [08/Jun/2007:16:35:39 -0400] "GET /forum/viewtopic.php?
> forum=3&showtopic=47113 HTTP/1.0" 503 0 "-" "Mozilla/5.0 (compatible; Yahoo!
> Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"
> 74.6.21.164 - - [08/Jun/2007:16:35:39 -0400] "GET /forum/viewtopic.php?
> showtopic=14219 HTTP/1.0" 503 0 "-" "Mozilla/5.0 (compatible; Yahoo!
> Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"
> 74.6.18.103 - - [08/Jun/2007:16:35:39 -0400] "GET /forum/viewtopic.php?
> forum=3&showtopic=31759 HTTP/1.0" 503 0 "-" "Mozilla/5.0 (compatible; Yahoo!
> Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"
>
> That was the best I could think of, short of blocking it entirely. And who
> wants to block the #2 search engine, even if it's a pain? :-/
>
> I hope it slows down a bit eventually, so that I can let at least get some
> requests through again ...
>
> bye, Dirk
>
>
> --
> http://www.geeklog.net/
> http://geeklog.info/
>
> _______________________________________________
> geeklog-devel mailing list
> geeklog-devel at lists.geeklog.net
> http://eight.pairlist.net/mailman/listinfo/geeklog-devel
>
> __________ NOD32 2319 (20070608) Information __________
>
> This message was checked by NOD32 antivirus system.
> http://www.eset.com
>
>
> _______________________________________________
> geeklog-devel mailing list
> geeklog-devel at lists.geeklog.net
> http://eight.pairlist.net/mailman/listinfo/geeklog-devel
>
>
>