command line fan fiction program?

Linux for blind general discussion blinux-list at redhat.com
Fri Apr 21 09:14:59 UTC 2017


Again, sorry for the month late reply, but others might find this useful. 
After I wrote my last reply, I remembered something, see below.

On 3/23/2017 6:42 AM, Karen Lewellen wrote:
> An example of a mass downloader included with a  Linux shell?

If you mean for grabbing thousands of files, wget works well and runs almost 
anywhere. Other good choices are aria2c and curl. I prefer aria2c, 
especially for servers which use a lot of redirects and hide the final name 
of the downloaded file. I'm seeing this a lot with podcasts. Most 
downloaders give the downloaded mp3 file a useless name like audio.mp3 or a 
bunch of hex numbers. I don't have this problem with aria2c. It also 
supports torrents and I think parallel downloading. It's updated often, but 
Debian testing should have a current version. There are Windows binaries but 
I couldn't get them to run here. I'm not sure if it's in cygwin.

> I want to test this, but am unsure of what tool to use.

I suggest wget, but see below.

> The editing is not a problem, I am far from picky about it having, with the
> work was smaller, used the m.edition  of the site to secure things one at a
> time.

I didn't know about the mobile site, or I forgot about it. I'll have to take 
a look. Also, w3m and links the chain do a far better job of dumping the 
HTML to text without as much junk. If you pass the -nolist option to Lynx 
the cat, you don't get the often huge list of links at the end. Finally, for 
getting a large number of pages, don't forget the -crawl option. I think 
wget is more popular. I almost never see the -crawl option of Lynx the cat 
mentioned.

> That I might  grab all the chapters at once would be a fine solution.
> Certainly for the works I cannot get in epub.


I might have a solution. Archive Team did a huge fanfiction.net grab a few 
years ago. They published a script to get huge blocks of stories en mass 
(we're talking thousands) by range. It should work for a single story or a 
group of any size as long as the range is sequential. It of course downloads 
all chapters. I'm not sure if it gets .epub, but I seem to remember reading 
them, so I think it does. I have to do some digging to see if it's still 
around. My machine with the fanfiction archive crashed, but archive.org 
should have it. Anyone interested in mass downloads of sites should look at 
Archive Team, but be warned there is strong language. They preserve sites 
which would otherwise be lost. http://www.archiveteam.org/

If you want a 2016-06-26 fanfiction repack (not sure of the original 
download date), look here but be warned that most of the zip files are over 
1 GB in size. I'm not sure about format.

https://archive.org/download/fanfiction_collection_repack_2016

Here is a general Internet Archive fanfiction search which is how I found 
the above link:

https://archive.org/search.php?query=fanfiction

I can't seem to find the fanfiction download script. I think some grabs 
might have been moved or deleted. You could try archiveteam.org. I did find 
some Firefly fan fiction though. It looks like the 2016 repack link above is 
the best bet for a huge, mass download.




More information about the Blinux-list mailing list