Apt - strange requests to d16r8ew072anqo.cloudfront.net:80
On Saturday (May 18) I started one of my VMs (running Ubuntu 18.04 Server).
To my surprise, the VM almost immediately attempted to connect to d16r8ew072anqo.cloudfront.net:80,
something that I have never seen before--it's a pretty "pristine" installation of Ubuntu,
with no custom apt repositories, and just a few packages installed.
It had never connected to anything suspicious before--mostly to Ubuntu and Snap domains.
(I use Little Snitch to monitor network traffic so I see connections in real time
and can deny them as well.)
I spent some time trying to figure out what happened
and I believe I narrowed it down to unattended-upgrades installing security patches.
Specifically, when I manually reinstalled intel-microcode:amd64 package I was able to
reproduce the strange connection to CloudFront (although it might have been
just a coincidence).
Then, on Monday, I wanted to document the issue in case something similar happens again, but to my surprise I couldn't reproduce the strange connection anymore.
And the only observable difference on Monday was that the output from
sudo apt-get install --reinstall intel-microcode:amd64 [1] did't have the Ign:1 line.
I searched the web, including http://archive.ubuntu.com/ubuntu, grep-ed
the VM's disk, checked the DNS records of misc. ubuntu.com subdomains,
tried wget-ting different URLs to find a redirect to the suspicious domain--but I couldn't find any clue about the strange connection to CloudFront.
My question is: does anyone know what happened, or at least noticed the same connection in their logs?
(By the way, I'm aware of one example where the Ubuntu team used CloudFront to relieve their servers: MD5 mismatch on my 12.04 ISO, what is going on? --so I'm hoping that maybe this is a similar case?)
[1]:
$ sudo apt-get install --reinstall intel-microcode:amd64
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not upgraded.
Need to get 1,801 kB of archives.
After this operation, 0 B of additional disk space will be used.
Ign:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 intel-microcode amd64 3.20190514.0ubuntu0.18.04.2
Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 intel-microcode amd64 3.20190514.0ubuntu0.18.04.2 [1,801 kB]
Fetched 1,801 kB in 20s (89.2 kB/s)
(Reading database ... 107477 files and directories currently installed.)
Preparing to unpack .../intel-microcode_3.20190514.0ubuntu0.18.04.2_amd64.deb ...
Unpacking intel-microcode (3.20190514.0ubuntu0.18.04.2) over (3.20190514.0ubuntu0.18.04.2) ...
Setting up intel-microcode (3.20190514.0ubuntu0.18.04.2) ...
update-initramfs: deferring update (trigger activated)
intel-microcode: microcode will be updated at next boot
Processing triggers for initramfs-tools (0.130ubuntu3.7) ...
update-initramfs: Generating /boot/initrd.img-4.15.0-50-generic
Solution 1:
I made some inqurires to the Security and Archive teams about this, as they'd be the authoritative source on whether this was expected behavior or not. The following is a summary of what they shared with me:
This observed behavior was offloading traffic load from the archive mirrors to Cloudfront, and was done between Wednesday May 15th and Monday May 20th in order to alleviate load from the Archive Servers, specifically for the Kernel and Microcode updates.
According to those teams, this is the first time such offloading was done via CloudFront, and in this particular case was expected behavior.
While it looks suspicious, the teams have confirmed with me via IRC that this was expected behavior, and they are surprised that more people hadn't noticed the behavior in the past.
ULTIMATELY: Not malicious behavior, but instead was expected behavior, and was required in order for things to not overload on the archive servers. (It was also the first time they've had to do this so at least nothing blew up heh)
Standard behavior of NOT offloading to Cloudfront should be back in place now.