How to get past the login page with Wget?

I am trying to use Wget to download a page, but I cannot get past the login screen.

How do I send the username/password using post data on the login page and then download the actual page as an authenticated user?

Solution 1:

Based on the manual page:

# Log in to the server.  This only needs to be done once.
wget --save-cookies cookies.txt \
     --keep-session-cookies \
     --post-data 'user=foo&password=bar' \
     --delete-after \
     http://server.com/auth.php

# Now grab the page or pages we care about.
wget --load-cookies cookies.txt \
     http://server.com/interesting/article.php

Make sure the --post-data parameter is properly percent-encoded (especially ampersands!) or the request will probably fail. Also make sure that user and password are the correct keys; you can find out the correct keys by sleuthing the HTML of the login page (look into your browser’s “inspect element” feature and find the name attribute on the username and password fields).

Solution 2:

You can log in via browser and copy the needed headers afterwards:

Use "Copy as cURL" in the Network tab of browser developer tools and replace curl's flag -H with wget's --header (and also --data with --post-data if needed).

Solution 3:

I directly gave cookies of an existing connection to wget with --no-cookies and the Cookie HTTP request header. In my case it was a Moodle university login where logging in looks more complex (using multiple requests with a login ticket). I added --post-data because it was a POST request.

For example, get all Moodle users list:

wget --no-cookies --header "Cookie: <name>=<value>" --post-data 'tab=search&name=+&personsubmit=Rechercher&keywords=&keywordsoption=allmine' https://moodle.unistra.fr/message/index.php

Solution 4:

I had the same problem. My solution was to do the login via Chrome and save the cookies data to a text file. This is easily done with this Chrome extension: Chrome cookie.txt export extension.

When you get the cookies data, there is also an example on how to use them with wget. A simple copy-paste command line is provided to you.