Download files from a list

How can I download files (that are listed in a text file) using wget or some other automatic way?

Sample file list:

www.example.com/1.pdf
www.example.com/2.pdf
www.example.com/3.pdf

wget has a built-in flag for this: wget -i your_list, where your_list is a file containing URL's delimited by linebreaks. You can find this kind of thing by reading man wget

Get them in parallel with

cat urlfile | parallel --gnu "wget {}"

By default it will run as many processes as you have cores, you can probably ramp this up another 10x if you really want to pull them down quickly by adding "-j 20" after parallel.

parallel has a built-in flag --arg-file (-a) that will use an input-file as the source, so you can avoid cat |. You can use

parallel --gnu -a urlfile wget

Or simply parallel --gnu wget < urlfile

xargs -i wget 'http://{}'  < your_list