command line fan fiction program?

Tim Chase blinux.list at
Thu Mar 23 16:15:53 UTC 2017

On March 23, 2017, Karen Lewellen wrote:
> An example of a mass downloader included with a  Linux shell?
> I want to test this, but am unsure of what tool to use.

If, as Jeffrey suggests, there's a sensible pattern to the chapter
breakdowns (an actual sample URL would help), you can either use
"curl" which knows how to expand numeric ranges or wrap it in a "for"
loop in the shell.  Additionally, a little testing suggests that the
trailing story-title slug is optional.  So you can do:

  $ STORY_ID=8045114
  $ SLUG=MauradersPlan
  $ curl
  "${STORY_ID}/[1-${TOTAL_CHAPTERS}]/" -o

I created variables to clarify what's going where and what's easy to
change.  It could easily be put in a script such as "":

  curl "$1/[1-$2]/" -o "$3_#1.html"

make it executable:

  $ chmod ugo+x ./

and then you can invoke it with

  $ ./ 8045114 87 MauradersPlan

Though now that I better understand the problem, wget might be an
even better solution since it can pull down all the chapters *and*
update the internal links so that they link to each other.  And all
you need is the story ID:

  wget -c --no-parent --mirror --trust-server-names --convert-links

Again, as per my previous message, you might want to use the


options as well to be a little kinder to the server instead of
hammering them.

You might then want to make a directory in which you put symlinks so
that you have a sensible name pointing to that directory so you can
remember that as "MauradersPlan" instead of "8045114":

  ln -s MauradersPlan

The wget and curl utilities are pretty prevalent, so I imagine
they're already available to you (and if not, should be uneventful
for installing).

I did the above, giving me an 87-chapter book that I was able to
navigate offline with lynx (or whichever other browser) and
encountered no major issues.

Best wishes,



More information about the Blinux-list mailing list