Download tool for multiple files or wildcards in http
chomiak at charter.net
Wed Apr 7 00:36:21 UTC 2004
I'm not sure whether lftp can be used for this. I'm probably not doing a
very good job of explaining this, but I think the supposed directory for
the books really isn't the actual directory and direct access to anything
but specific urls might not be possible. Using lftp with the -u option to
give my username and password, I can go to www.loc.gov/nls/braille, but no
.brf books appear in the directory list or if you try mget. But I can
directly download with wget or edbrowse for example,
www.loc.gov/nls/braille/12269v01.brf. I can see why they want to protect
the files and it's not my desire to crash security, but it still would be
nice not to have to download each file individually.
Lftp would be easier to use than the suggested script if it would work, I
think, but I'm not sure it will.
As for the script to pipe to wget, I'm getting errors with that, but those
may be my own mistakes in setting it up; will post and ask about it if i
don't figure it out. The filenames are five-digit numbers plus v01 v02 v03
etc., depending on how many volumes, ending in .brf.
I actually did write NLS once and ask them about using an ftp setup; they
said they had considered it but had security concerns.
Anyway, it's not like what I'm doing doesn't work, but I'm always looking
for better and less time-consuming ways to do things.
"Where your treasure is, there will your heart be also."
More information about the Blinux-list