Looking for some apache config help to block evil spiders

[Tony Nelson]

On 09-10-10 14:37:29, Steven W. Orr wrote:
> ... Here's what I added to my httpd.conf:
> 
> RewriteLog    logs/rewrite_log
> RewriteLogLevel 1
> 
> RewriteCond %{HTTP_USER_AGENT}  ^Baiduspider.* [OR]
> RewriteCond %{HTTP_USER_AGENT}	^msnbot.* [OR]
> RewriteCond %{HTTP_USER_AGENT}	^NaverBot.* [OR]
> RewriteCond %{HTTP_USER_AGENT}	^Sogou-Test-Spider.*
> RewriteCond %{HTTP_USER_AGENT}	^Mozilla/4.0.*
> RewriteCond %{HTTP_USER_AGENT}	^T-Mobile Dash.*
> RewriteRule .* - [F,L]
> 
> and inside each of the virtual domains, I added:
> 
>     RewriteEngine On
>     RewriteOptions Inherit
> 
> Here's the problem. What I want to see is the rewrite_log telling me
> what it has redirected or failed. Instead, I'm getting a line telling 
> me every link that it does NOT rewrite. For example:
> 
> 72.30.65.61 - - [10/Oct/2009:14:28:24 --0400] \
> [vdom.syslang.net/sid#b7298ed0][rid#b6b488e8/initial] (1) pass 
> through /d1/fn
> 
> I have googled my brains out and it seems like others have had the
> same questions. I see no answers. If anyone has any idea I love to 
> hear it.

WAG:  The RewriteRule doesn't actually rewrite anything.  Perhaps 
something would be logged if it did.  You'd probably still have the 
other log lines as well.

-- 
____________________________________________________________________
TonyN.:'                       <mailto:tonynelson at georgeanelson.com>
      '                              <http://www.georgeanelson.com/>