Need to download lots of files, Whats the most lightweight download manager?
Solution 1:
I routinely do 3k <.5MB files, and I use Free Download Manager.
I can't say if there is better or more light weight solutions out there, but, this works and does what I need - I go page by page, right click, add all to IDM, then when I finished, I highlight them all in IDM and do start!
I have never done 30k all in one go, but, I hope this helps!
Solution 2:
As @Shiki already said: wget
wget -i
will read all URLs from external files and process them
Solution 3:
Jdownloader http://jdownloader.org/download/index Is an interesting, multi-platform Java downloader that is being developed continually. It is Not lightweight at all, and could be a security risk for all I know. 30k would probably have to be done in sections. but it has a way to backup the list, and load lists in. Its main design seems to be more about downloading from "public servers", but it works good for normal stuff.
I have seen 5000 items in it, but that was because I asked it to analise a link that went on forever, ooops.
Solution 4:
I use curl (http://curl.haxx.se/). It has a bunch of good features for mass downloads. It allows you to specify many files on one invocation, like this:
CURL.EXE http://example.com/file[1-10].mp3
That will download "file1.mp3", "file2.mp3", and so on.
I've used it to download many files, but not 5,000. However, in principle it should work.