apt.conf "Acquire::http:Proxy "proxyserver:port" seems not to be used (Ubuntu 13.04 under Virtual Box on Win7)

networking apt software-center proxy virtualbox-networking

This is for an installation of Ubuntu 13.04 Desktop in a VirtualBox VM. I'm on a corprate network with an HTTP proxy at x.y.z.251:9090. I've configured the guest OS with the autoconfiguration script, so Firefox works fine, but I can't get "apt-get" or "Ubuntu Software Center" to connect using the proxy.

I created the file "/etc/apt/apt.conf", it now contains a single line of text:

Acquire::http:Proxy "http://x.y.z.251:9090"

which is what the proxy.pac file that I specified in "System Settings/Network/Network Proxy" and applied to the entire system.

When I use Firefox, a packet trace shows that the TCP connection is done via the corporate proxy at port 9090, when I run "Ubuntu Software Center" and "apt-get", the packet trace shows TCP SYN packets with the eventual destination IP address and port 80. There, is of course, no response to the SYN packets, since the corporate firewall blocks TCP connections on port 80 from getting in or out.

With "sudo apt-get" in a terminal window, I always see:

Ign cdrom://Ubuntu 13.04 _Raring Ringtail_ - Release amd64 (20130424) raring/main Translation-en_US
Ign cdrom://Ubuntu 13.04 _Raring Ringtail_ - Release amd64 (20130424) raring/main Translation-en
Ign cdrom://Ubuntu 13.04 _Raring Ringtail_ - Release amd64 (20130424) raring/restricted Translation-en_US
Ign cdrom://Ubuntu 13.04 _Raring Ringtail_ - Release amd64 (20130424) raring/restricted Translation-en
Err http://extras.ubuntu.com raring Release.gpg 
  Could not connect to extras.ubuntu.com:80 (91.189.92.152), connection timed out
Err http://archive.canonical.com raring Release.gpg 
  Cannot initiate the connection to archive.canonical.com:80 (2001:67c:1360:8c01::1b). - connect (101: Network is unreachable) [IP: 2001:67c:1360:8c01::1b 80]
Err http://us.archive.ubuntu.com raring Release.gpg 
  Cannot initiate the connection to us.archive.ubuntu.com:80 (2001:67c:1562::14). - connect (101: Network is unreachable) [IP: 2001:67c:1562::14 80]
Err http://us.archive.ubuntu.com raring-updates Release.gpg
  Cannot initiate the connection to us.archive.ubuntu.com:80 (2001:67c:1562::14). - connect (101: Network is unreachable) [IP: 2001:67c:1562::14 80]
Err http://us.archive.ubuntu.com raring-backports Release.gpg
  Cannot initiate the connection to us.archive.ubuntu.com:80 (2001:67c:1562::14). - connect (101: Network is unreachable) [IP: 2001:67c:1562::14 80]
59% [Connecting to security.ubuntu.com (91.189.91.14)]

Eventually, I see:

Err http://security.ubuntu.com raring-security Release.gpg
  Cannot initiate the connection to security.ubuntu.com:80 (2001:67c:1360:8c01::18). - connect (101: Network is unreachable) [IP: 2001:67c:1360:8c01::18 80]
Reading package lists... Done                          
W: Failed to fetch http://us.archive.ubuntu.com/ubuntu/dists/raring/Release.gpg  Cannot initiate the connection to us.archive.ubuntu.com:80 (2001:67c:1562::14). - connect (101: Network is unreachable) [IP: 2001:67c:1562::14 80]

W: Failed to fetch http://us.archive.ubuntu.com/ubuntu/dists/raring-updates/Release.gpg  Cannot initiate the connection to us.archive.ubuntu.com:80 (2001:67c:1562::14). - connect (101: Network is unreachable) [IP: 2001:67c:1562::14 80]

W: Failed to fetch http://us.archive.ubuntu.com/ubuntu/dists/raring-backports/Release.gpg  Cannot initiate the connection to us.archive.ubuntu.com:80 (2001:67c:1562::14). - connect (101: Network is unreachable) [IP: 2001:67c:1562::14 80]

W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/raring-security/Release.gpg  Cannot initiate the connection to security.ubuntu.com:80 (2001:67c:1360:8c01::18). - connect (101: Network is unreachable) [IP: 2001:67c:1360:8c01::18 80]

W: Failed to fetch http://archive.canonical.com/ubuntu/dists/raring/Release.gpg  Cannot initiate the connection to archive.canonical.com:80 (2001:67c:1360:8c01::1b). - connect (101: Network is unreachable) [IP: 2001:67c:1360:8c01::1b 80]

W: Failed to fetch http://extras.ubuntu.com/ubuntu/dists/raring/Release.gpg  Could not connect to extras.ubuntu.com:80 (91.189.92.152), connection timed out

W: Some index files failed to download. They have been ignored, or old ones used instead.

I'm not at all certain what I'm doing wrong.

--- Update ---- I have also tried:

Acquire::http:proxy "http://user:[email protected]:9090/"

"x.y.z" are, of course, place-holders; I doubt my employer's security people would look kindly on me posting the actual numbers. I see no difference in the packet trace -- "apt-get" is still using the real IP addresses and port 80. I made a deliberate mistake in the apt.conf file and "apt-get" exited with an error, so I'm certain that it sees this configuration line for the proxy, it just doesn't seem to honor it.

I am using the "NAT" networking connection, as it is against the IT policy here to bridge between networks, and the DHCP server only provides addresses to known MAC addresses. I don't want IT security banging on my cube (again).


Based on my past experience, overall Ubuntu (GNOME) doesn't work very well with PAC (Automatic Config). Pac (Proxy Auto-config) is basically JavaScript logic to determine the closest or most appropriate proxy for you, I strongly recommend using the proxy servers directly (in your case Network - Network Proxy).

A few things to check

Check your /etc/apt/apt.conf

Network -Network Proxy GUI will update /etc/apt/apt.conf and set the proxy info there.

Find out the proxy server info from pac

You can find the details by reading the proxy.pac file anyway.

So the /etc/apt/apt.conf should look like below (suppose the proxy server is => proxy.company.com port 80), do NOT use pac URL.

Acquire::http::proxy "http://proxy.company.com:80/";
Acquire::https::proxy "https://proxy.company.com:80/";
Acquire::ftp::proxy "ftp://proxy.company.com:80/";

NOTE: If no proxy is specified in the apt config files, apt-get will fall back to http_proxy environment variable.

From apt.conf man page, respond to the comment from @Braiam

   http
       HTTP URIs; http::Proxy is the default http proxy to use. It is in
       the standard form of http://[[user][:pass]@]host[:port]/. Per host
       proxies can also be specified by using the form http::Proxy::<host>
       with the special keyword DIRECT meaning to use no proxies. If no
       one of the above settings is specified, http_proxy environment
       variable will be used.

Check proxy environment variables

You can check proxy settings by running the following in terminal

echo $http_proxy
echo $https_proxy
echo $ftp_proxy

If you set env variables correctly, you should be should be able to update in CLI.

Prefer IPv4

I noticed apt-get was trying to connect to the update servers using their ipv6 address. This could cause problems (sometimes the IPV6 addresses expire or change).

You can edit /etc/gai.conf and add precedence ::ffff:0:0/96 100 to prefer ipv4 over ipv6.

More details

  • https://unix.stackexchange.com/a/13263/30622
  • https://serverfault.com/a/177649/145918

I believe I have had this very problem and the answer is simple. The syntax is critical. It must look like this:

Acquire::http::Proxy "http://x.y.z.251:9090";

(There is an extra ":" and a terminating ";")


@Braiam

I think it's better to start a separate answer to clarify things.

I did a quick test using Vagrant and a Ubuntu 13.04 Raring x86_64 VM behind firewall.

apt-get version is 0.9.7.7ubuntu4

root@raring:~# apt-get --version 
apt 0.9.7.7ubuntu4 for amd64 compiled on Apr 12 2013 23:49:05
Supported modules:
*Ver: Standard .deb
*Pkg:  Debian dpkg interface (Priority 30)
 Pkg:  Debian APT solver interface (Priority -1000)
 S.L: 'deb' Standard Debian binary tree
 S.L: 'deb-src' Standard Debian source tree
 Idx: Debian Source Index
 Idx: Debian Package Index
 Idx: Debian Translation Index
 Idx: Debian dpkg status file
 Idx: EDSP scenario file

Update: It works the same way on Precise 12.04.3

root@support:/etc/apt# uname -a
Linux support 3.8.0-30-generic #44~precise1-Ubuntu SMP Fri Aug 23 17:33:45 UTC 2013 i686 i686 i386 GNU/Linux
root@support:/etc/apt# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 12.04.3 LTS
Release:        12.04
Codename:       precise
root@support:/etc/apt# apt-get --version
apt 0.8.16~exp12ubuntu10.14 for i386 compiled on Sep  8 2013 03:26:42

1st - clear apt configuration files

I cleared the proxy settings in /etc/apt/apt.conf. I also did ack and ag scan of all files and sub-directories /etc/apt to make sure there is no proxy set in apt configuration files.

By default {http_proxy,https_proxy,ftp_proxy} env variables are not set.

apt-get was not able to connect to the update servers.

root@raring:~# apt-get -o Debug::Acquire::http=true update
0% [Connecting to au.archive.ubuntu.com (202.158.214.106)] [Connecting to security.ubuntu.com (91.189.91.13)] [Connecting to ppa.launchpad.net (91.189.95.83)]

2nd - set environment variables

Simply set the proxy variables

export {http_proxy,https_proxy,ftp_proxy}="http://10.xxx.xxx.231:80"

apt-get is now able to connect!

See the debug output (this is something that I have never tried).

root@raring:~# apt-get -o Debug::Acquire::http=true update
0% [Working]GET http://security.ubuntu.com/ubuntu/dists/raring-security/Release.gpg HTTP/1.1
Host: security.ubuntu.com
Cache-Control: max-age=0
User-Agent: Debian APT-HTTP/1.3 (0.9.7.7ubuntu4)


GET http://au.archive.ubuntu.com/ubuntu/dists/raring/Release.gpg HTTP/1.1
Host: au.archive.ubuntu.com
Cache-Control: max-age=0
User-Agent: Debian APT-HTTP/1.3 (0.9.7.7ubuntu4)


GET http://ppa.launchpad.net/git-core/ppa/ubuntu/dists/raring/Release.gpg HTTP/1.1
Host: ppa.launchpad.net
Cache-Control: max-age=0
User-Agent: Debian APT-HTTP/1.3 (0.9.7.7ubuntu4)


HTTP/1.1 200 OK
Date: Fri, 27 Sep 2013 12:12:20 GMT
ETag: "16e20bb4-3a5-4db2e154a1dc0"
Server: Apache/2.2.3 (Red Hat)
Content-Type: text/plain; charset=UTF-8
Accept-Ranges: bytes
Last-Modified: Thu, 25 Apr 2013 11:54:39 GMT
Content-Length: 933
Proxy-Connection: Keep-Alive

Get:1 http://au.archive.ubuntu.com raring Release.gpg [933 B]
0% [1 Release.gpg 0 B/933 B 0%] [Waiting for headers] [Waiting for headers]GET http://au.archive.ubuntu.com/ubuntu/dists/raring-updates/Release.gpg HTTP/1.1
Host: au.archive.ubuntu.com
Cache-Control: max-age=0
User-Agent: Debian APT-HTTP/1.3 (0.9.7.7ubuntu4)


99% [Waiting for headers] [Waiting for headers]HTTP/1.1 200 OK             
Date: Fri, 27 Sep 2013 12:12:20 GMT
ETag: "16e20ee7-3a5-4e7594ace9200"
Server: Apache/2.2.3 (Red Hat)
Content-Type: text/plain; charset=UTF-8
Accept-Ranges: bytes
Last-Modified: Fri, 27 Sep 2013 08:30:00 GMT
Content-Length: 933
Proxy-Connection: Keep-Alive

Get:2 http://au.archive.ubuntu.com raring-updates Release.gpg [933 B]
50% [2 Release.gpg 0 B/933 B 0%] [Waiting for headers] [Waiting for headers]GET http://au.archive.ubuntu.com/ubuntu/dists/raring-backports/Release.gpg HTTP/1.1
Host: au.archive.ubuntu.com
Cache-Control: max-age=0
User-Agent: Debian APT-HTTP/1.3 (0.9.7.7ubuntu4)


100% [Waiting for headers] [Waiting for headers]HTTP/1.1 200 OK             
Date: Fri, 27 Sep 2013 12:12:21 GMT
ETag: "16e0083d-3a5-4e6844f477fc0"
Server: Apache/2.2.3 (Red Hat)
Content-Type: text/plain; charset=UTF-8
Accept-Ranges: bytes
Last-Modified: Mon, 16 Sep 2013 18:24:07 GMT
Content-Length: 933
Proxy-Connection: Keep-Alive

Get:3 http://au.archive.ubuntu.com raring-backports Release.gpg [933 B]
67% [3 Release.gpg 0 B/933 B 0%] [Waiting for headers] [Waiting for headers]HTTP/1.1 200 OK
Date: Fri, 27 Sep 2013 12:12:20 GMT
ETag: "3a5-4e75c5d969600"
Server: Apache/2.2.22 (Ubuntu)
Expires: Fri, 27 Sep 2013 13:05:00 GMT
Accept-Ranges: bytes
Cache-Control: max-age=3159, s-maxage=3300, proxy-revalidate
Last-Modified: Fri, 27 Sep 2013 12:10:00 GMT
Content-Length: 933
Proxy-Connection: Keep-Alive

......

3rd - unset the environment variables

unset the env vars => unset {http_proxy,https_proxy,ftp_proxy}

Now apt-get was unable to connect.

Conclusion

Just like what man apt.conf says, if http::Proxy is NOT set, http_proxy environment variable will be used.

BTW: I am also an Arch Linux user. Pacman works in a similar way, if I do not use wget or curl with proxy in pacman.conf, it'll use `{http_proxy,https_proxy,ftp_proxy}' environment variables.

Update sudo does NOT preserve environment variables. That's why sudo apt-get update fails. To workaround the issue (preserve proxy environment variables), use sudo -E apt-get update.

Related

How to get longest line from a file?
Vim cannot find syntax.vim
Problem with printing (hplip) in Ubuntu 14.04
How do you sideload apk files in Windows 11?
What is the practical usage difference between "grep -e" and "grep"?
Sabertooth 990FX R3.0 won't enable TPM
Robocopy does not copy the root folder and its time stamp
Windows 10 Font folder displaying as files instead of fonts
Remote desktop authentication on home network - Microsoft account vs Windows account?
Disable Mac typing Accent Menu
How do you open a specific folder in Finder without applescript?
Where does Mac Messages store older Attachments?