Wget a page that requires logging in

Solution 1:

Another solution, after being logged in in the browser, and if you don't want to use the Firefox cookie extractor in Python, is to open your web inspector and check what session headers are sent.

For example, in Chrome:

Remote Address:111.11.111.111:80
Request URL:http://example.com
Request Method:GET
Status Code:200 OK
Request Headersview source
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip, deflate, sdch
Accept-Language:fr,en-US;q=0.8,en;q=0.6,es;q=0.4,de;q=0.2
Cache-Control:no-cache
Connection:keep-alive
Cookie:_ga=GA1.2.228078207.1409667791; mp_d6ebe82547b18c335122656ad5df6c0e_mixpanel=%7B%22distinct_id%22%3A%20%221492964fd1e75-0b7e66217-39740157-15f900-1492964fd1f1b8%22%2C%22%24initial_referrer%22%3A%20%22%24direct%22%2C%22%24initial_referring_domain%22%3A%20%22%24direct%22%7D; rack.session=BAh7B0kiD3Nlc3Npb25faWQGOgZFVEkiRTMyZGMwMTc0OWMwNmE2YzBjYWQ4%0AMjM1ODdjNGZlNzY4NDdmZjNkY2ZhYWIzNWNiYmYxZWM1MjkwMGM0YTNhYzQG%0AOwBGSSIcd2FyZGVuLnVzZXIuZGVmYXVsdC5rZXkGOwBUVToZV2FyZGVuOjpH%0AaXRIdWI6OlVzZXJ7BzoMYXR0cmlic3sNSSIKbG9naW4GOwBGSSISYXVnLXJp%0AZWRpbmdlcgY7AFRJIgdpZAY7AEZpA%2BwPHkkiD2F2YXRhcl91cmwGOwBGSSI4%0AaHR0cHM6Ly9hdmF0YXJzLmdpdGh1YnVzZXJjb250ZW50LmNvbS91LzE5NzAx%0ANTY%2Fdj0zBjsAVEkiEGdyYXZhdGFyX2lkBjsARkkiAAY7AFRJIg9zaXRlX2Fk%0AbWluBjsARkZJIgluYW1lBjsARkkiF0F1Z3VzdGluIFJpZWRpbmdlcgY7AFRJ%0AIgxjb21wYW55BjsARkkiC0NvcGFzcwY7AFRJIgplbWFpbAY7AEZJIgAGOwBU%0AOgp0b2tlbkkiLTExMzg4NDkzNGIzZDkxNTMzOGJlOTU3YjcxZTA3OTU3ZDhh%0AYWQ2YjEGOwBU%0A--cf66a01faadf81e2cf2997a9e01c7dccdc5c67ba
Host:example.com
Pragma:no-cache
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36

Here the following command will work:

wget --header "Cookie: rack.session=BAh7B0kiD3Nlc3Npb25faWQGOgZFVEkiRTMyZGMwMTc0OWMwNmE2YzBjYWQ4%0AMjM1ODdjNGZlNzY4NDdmZjNkY2ZhYWIzNWNiYmYxZWM1MjkwMGM0YTNhYzQG%0AOwBGSSIcd2FyZGVuLnVzZXIuZGVmYXVsdC5rZXkGOwBUVToZV2FyZGVuOjpH%0AaXRIdWI6OlVzZXJ7BzoMYXR0cmlic3sNSSIKbG9naW4GOwBGSSISYXVnLXJp%0AZWRpbmdlcgY7AFRJIgdpZAY7AEZpA%2BwPHkkiD2F2YXRhcl91cmwGOwBGSSI4%0AaHR0cHM6Ly9hdmF0YXJzLmdpdGh1YnVzZXJjb250ZW50LmNvbS91LzE5NzAx%0ANTY%2Fdj0zBjsAVEkiEGdyYXZhdGFyX2lkBjsARkkiAAY7AFRJIg9zaXRlX2Fk%0AbWluBjsARkZJIgluYW1lBjsARkkiF0F1Z3VzdGluIFJpZWRpbmdlcgY7AFRJ%0AIgxjb21wYW55BjsARkkiC0NvcGFzcwY7AFRJIgplbWFpbAY7AEZJIgAGOwBU%0AOgp0b2tlbkkiLTExMzg4NDkzNGIzZDkxNTMzOGJlOTU3YjcxZTA3OTU3ZDhh%0AYWQ2YjEGOwBU%0A--cf66a01faadf81e2cf2997a9e01c7dccdc5c67ba"  http://example.com

Solution 2:

Does the page have a "Remember me" option. If so, you can export the cookie file (see this: http://blog.mithis.net/archives/python/90-firefox3-cookies-in-python) and use --load-cookies in wget.