Proxy Blocking apt-get, Allowing wget/curl

I'm having an issue getting passed my corporate proxy server. I had an exception put in place to allow my IP address to pass through, but haven't been able to get apt-get to work.

wget can get to the Ubuntu repos without any issues.

root@server:/tmp# http_proxy=http://<PROXY>:8080 wget http://us.archive.ubuntu.com/ubuntu/dists/precise-backports/multiverse/binary-i386/Packages.bz2
--2014-01-24 09:17:38--  http://us.archive.ubuntu.com/ubuntu/dists/precise-backports/multiverse/binary-i386/Packages.bz2
Resolving <PROXY> (<PROXY>)... x.x.x.25, x.x.x.24
Connecting to <PROXY> (<PROXY>)|x.x.x.25|:8080... connected.
Proxy request sent, awaiting response... 200 OK
Length: 5178 (5.1K) [application/x-bzip2]
Saving to: `Packages.bz2'

100%[========================================>] 5,178       --.-K/s   in 0.001s

2014-01-24 09:17:38 (7.78 MB/s) - `Packages.bz2' saved [5178/5178]

root@server:/tmp# ll -h
total 16K
drwxrwxrwt  2 root root 4.0K Jan 24 09:17 ./
drwxr-xr-x 23 root root 4.0K Jan 16 14:14 ../
-rw-r--r--  1 root root 5.1K Jan 24 09:05 Packages.bz2
root@server:/tmp#

I setup my apt-conf.d proxy configurations.

root@server:/tmp# cat /etc/apt/apt.conf.d/30proxy
Acquire::http::proxy "http://<PROXY>:8080";
Acquire::ftp::proxy "ftp://<PROXY>:8080";
Acquire::https::proxy "https://<PROXY>:8080";
root@server:/tmp#

apt-get still fails with a 403 Forbidden error.

root@server:/tmp# apt-get update
Ign http://us.archive.ubuntu.com precise Release.gpg
<...snipped excess...>
Ign http://us.archive.ubuntu.com precise-backports/universe TranslationIndex
Err http://security.ubuntu.com precise-security/main Sources
  403  Forbidden [IP: x.x.x.25 8080]
<...snipped excess...>
Err http://security.ubuntu.com precise-security/multiverse i386 Packages
  403  Forbidden [IP: x.x.x.25 8080]
W: Failed to fetch http://us.archive.ubuntu.com/ubuntu/dists/precise/main/source/Sources  403  Forbidden [IP: x.x.x.24 8080]
<...snipped excess...>
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/precise-security/multiverse/binary-i386/Packages  403  Forbidden [IP: x.x.x.25 8080]
E: Some index files failed to download. They have been ignored, or old ones used instead.
root@server:/tmp#

^{PROXY replaces my proxy server's FQDN}

Solution 1:

The problem is with the reported user-agent. This is why the proxy worked with wget and not apt-get.

I ran tcpdump -Ai eth0 port 8080 to see what the http looked like and received a 'non approved user-agent' message from our corporate proxy.

<...snip...>
 <h1>ACCESS DENIED</h1>
 <p>The software you are accessing the internet with is not reporting an 
    approved "User-Agent"</p>
<...snip...>

The Ubuntu Manpage had a section that explains that configuration parameter.

Acquire::http::User-Agent can be used to set a different User-Agent for the http download method as some proxies allow access for clients only if the client uses a known identifier.

apt-get 307 error provided the needed syntax, you simply add the below syntax to /etc/apt/apt.conf.d/30proxy (or whatever you choose).

Acquire::http::User-Agent "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)";

Now, apt-get updates go through flawlessly.

root@server:/etc/apt/apt.conf.d# apt-get update
Hit http://security.ubuntu.com precise-security Release.gpg
Hit http://us.archive.ubuntu.com precise Release.gpg
<...snip...>
Hit http://us.archive.ubuntu.com precise-backports/multiverse Translation-en
Hit http://us.archive.ubuntu.com precise-backports/restricted Translation-en
Hit http://us.archive.ubuntu.com precise-backports/universe Translation-en
Reading package lists... Done