mirroring websites solution? wget problems

[Rick Stevens]

Matthew Benjamin wrote:
> Am Di, den 29.03.2005 schrieb Matthew Benjamin um 20:39:
> 
> 
>>I guess I need to figure out the authentication scheme. I am 
>>attempting to mirror a website that is housed on IIS, and the response
> 
> 
>>is
>>
>>... 401 Unauthorized
>>Unknown Authentication Scheme
> 
> 
> wget would need to know and handle NTLM
> 
> http://davenport.sourceforge.net/ntlm.html
> 
> AFAIK it does not know about NTLM yet.
> 
> You can check what the IIS server says by running
> 
> wget --server-response <target-server>
> 
> and you probably will find a line like "WWW-Authenticate: NTLM". Maybe
> you should use
> 
> http://apserver.sourceforge.net/
> 
> which is available through Dag Wieers' repository.
> 
> Alexander
> 
> One other problem, it is also trying to retrieve/mirror urls that are
> not native to my site. The absolute URLs (example: www.redhat.com)

I'm assuming the web site owners want you to mirror their sites.  If so,
why not just set up rsync and be done with it?
----------------------------------------------------------------------
- Rick Stevens, Senior Systems Engineer     rstevens at vitalstream.com -
- VitalStream, Inc.                       http://www.vitalstream.com -
-                                                                    -
-     Is that a buffer overflow or are you just happy to see me?     -
----------------------------------------------------------------------