command line fan fiction program?
blinux.list at thechases.com
Thu Mar 23 16:15:53 UTC 2017
On March 23, 2017, Karen Lewellen wrote:
> An example of a mass downloader included with a Linux shell?
> I want to test this, but am unsure of what tool to use.
If, as Jeffrey suggests, there's a sensible pattern to the chapter
breakdowns (an actual sample URL would help), you can either use
"curl" which knows how to expand numeric ranges or wrap it in a "for"
loop in the shell. Additionally, a little testing suggests that the
trailing story-title slug is optional. So you can do:
I created variables to clarify what's going where and what's easy to
change. It could easily be put in a script such as "fanfiction.sh":
curl "https://m.fanfiction.net/s/$1/[1-$2]/" -o "$3_#1.html"
make it executable:
$ chmod ugo+x ./fanfiction.sh
and then you can invoke it with
$ ./fanfiction.sh 8045114 87 MauradersPlan
Though now that I better understand the problem, wget might be an
even better solution since it can pull down all the chapters *and*
update the internal links so that they link to each other. And all
you need is the story ID:
wget -c --no-parent --mirror --trust-server-names --convert-links https://m.fanfiction.net/s/8045114/
Again, as per my previous message, you might want to use the
options as well to be a little kinder to the server instead of
You might then want to make a directory in which you put symlinks so
that you have a sensible name pointing to that directory so you can
remember that as "MauradersPlan" instead of "8045114":
ln -s m.fanfiction.net/s/8045114 MauradersPlan
The wget and curl utilities are pretty prevalent, so I imagine
they're already available to you (and if not, should be uneventful
I did the above, giving me an 87-chapter book that I was able to
navigate offline with lynx (or whichever other browser) and
encountered no major issues.
More information about the Blinux-list