Need to download lots of files, Whats the most lightweight download manager?

Solution 1:

I routinely do 3k <.5MB files, and I use Free Download Manager.

I can't say if there is better or more light weight solutions out there, but, this works and does what I need - I go page by page, right click, add all to IDM, then when I finished, I highlight them all in IDM and do start!

I have never done 30k all in one go, but, I hope this helps!

Solution 2:

As @Shiki already said: wget

wget -i will read all URLs from external files and process them

Solution 3:

Jdownloader http://jdownloader.org/download/index Is an interesting, multi-platform Java downloader that is being developed continually. It is Not lightweight at all, and could be a security risk for all I know. 30k would probably have to be done in sections. but it has a way to backup the list, and load lists in. Its main design seems to be more about downloading from "public servers", but it works good for normal stuff.

I have seen 5000 items in it, but that was because I asked it to analise a link that went on forever, ooops.

Solution 4:

I use curl (http://curl.haxx.se/). It has a bunch of good features for mass downloads. It allows you to specify many files on one invocation, like this:

CURL.EXE http://example.com/file[1-10].mp3

That will download "file1.mp3", "file2.mp3", and so on.

I've used it to download many files, but not 5,000. However, in principle it should work.