How do I build an Ubuntu/Ruby image that also contains foreman? [duplicate]
My team uses Docker (with ubuntu:14.04
base image) for local development and we often have to rebuild some or all of our images. But we often get failures downloading packages with apt-get install
, even immediately after running apt-get -y update
. For instance, today I see
Err http://archive.ubuntu.com/ubuntu/ trusty-security/main libxml2 amd64 2.9.1+dfsg1-3ubuntu4.7
404 Not Found [IP: 91.189.88.161 80]
Err http://archive.ubuntu.com/ubuntu/ trusty-security/main libxml2-dev amd64 2.9.1+dfsg1-3ubuntu4.7
404 Not Found [IP: 91.189.88.161 80]
Fetched 84.7 MB in 1min 6s (1281 kB/s)
Unable to correct missing packages.
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libx/libxml2/libxml2_2.9.1+dfsg1-3ubuntu4.7_amd64.deb 404 Not Found [IP: 91.189.88.161 80]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libx/libxml2/libxml2-dev_2.9.1+dfsg1-3ubuntu4.7_amd64.deb 404 Not Found [IP: 91.189.88.161 80]
E: Aborting install.
Apparently the specific version of a particular package has been deleted from the archive and replaced with a slightly differently named patch version. For instance, the above error is looking for libxml2_2.9.1+dfsg1-3ubuntu4.7_amd64.deb
but the version on the server is libxml2_2.9.1+dfsg1-3ubuntu4.8_amd64.deb
.
Often this is solvable by removing the base image (docker rmi ubuntu:14.04
) and rebuilding; the newly downloaded ubuntu image has the correct patch number and finds the right archive file. But even this doesn't always work -- probably due to a delay between a new minor upgrade to Ubuntu's dependency db and the deployment of that new ubuntu:14.04
image onto Docker Hub.
We've tried using apt-get
flags --fix-missing
and --fix-broken
and those don't consistently work either.
Any other ideas?
apt-get install fails with Not Found error because package removed from repository is a similar problem but the accepted answer is unacceptable because it's not possible to be automated. Our daily development process, including automatic build and deploy, is all scripted and using Docker and it's not practical to hack around inside a Dockerfile every time a particular archive goes missing (then remove the hack after a few hours or days).
In response to @prateek05, here's the /etc/apt/sources.list
from the official ubuntu:14.04
docker image:
root@72daa1942714:/# cat /etc/apt/sources.list
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://archive.ubuntu.com/ubuntu/ trusty main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty main restricted
## Major bug fix updates produced after the final release of the
## distribution.
deb http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted
## Uncomment the following two lines to add software from the 'universe'
## repository.
## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu/ trusty universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty universe
deb http://archive.ubuntu.com/ubuntu/ trusty-updates universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty-updates universe
## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
# deb http://archive.ubuntu.com/ubuntu/ trusty-backports main restricted
# deb-src http://archive.ubuntu.com/ubuntu/ trusty-backports main restricted
deb http://archive.ubuntu.com/ubuntu/ trusty-security main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty-security main restricted
deb http://archive.ubuntu.com/ubuntu/ trusty-security universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty-security universe
# deb http://archive.ubuntu.com/ubuntu/ trusty-security multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ trusty-security multiverse
Solution 1:
You have stated that your Dockerfile contains RUN apt-get -y update
as its own RUN
instruction. However, due to build caching, if all changes to the Dockerfile occur later in the file, when docker build
is run, Docker will reuse the intermediate image created the last time RUN apt-get -y update
executed instead of running the command again, and so any recently-added or -edited apt-get install
lines will be using old data, leading to the errors you've observed.
There are two ways to fix this:
Pass the
--no-cache
option todocker build
, forcing every statement in the Dockerfile to be run every time the image is built.Rewrite the Dockerfile to combine the
apt-get
commands in a singleRUN
instruction:RUN apt-get update && apt-get install foo bar ...
. This way, whenever the list of packages to install is edited,docker build
will be forced to re-execute the entireRUN
instruction and thus rerunapt-get update
before installing.
The Dockerfile best practices page actually has an entire section on apt-get
commands in Dockerfiles. I suggest you read it.
Solution 2:
The issue could be potentially with the ubuntu sources
Check /etc/apt/sources.list
If you see deb http://archive.ubuntu.com/ubuntu main universe restricted multiverse
, that could be the potential issue.
Fix that by replacing it with deb http://archive.ubuntu.com/ubuntu/ trusty main universe restricted multiverse
Alternatively it could be the mirror itself doesn't respond. archive.ubuntu.com
Name: archive.ubuntu.com
Address: 91.189.88.152
Name: archive.ubuntu.com
Address: 91.189.88.161
Name: archive.ubuntu.com
Address: 91.189.88.149
Replace archive.ubuntu.com with a more trusted mirror say us.archive.ubuntu.com
Name: us.archive.ubuntu.com
Address: 91.189.91.23
Name: us.archive.ubuntu.com
Address: 91.189.91.26
(edit by the oiriginal asker):
Thanks, prateek05! My Dockerfile now starts:
FROM ubuntu:14.04
RUN sed -i'' 's/archive\.ubuntu\.com/us\.archive\.ubuntu\.com/' /etc/apt/sources.list
RUN apt-get -y update
and it seems to be working. But since this is a sporadic issue, only time will tell...
Solution 3:
found something that works!
so i found this: https://www.mail-archive.com/[email protected]/msg5682159.html
The solution is to create a pinning_file
with the contents
# Written by ubuntu-advantage-tools
Package: *
Pin: release o=UbuntuESM, n=trusty
Pin-Priority: never
then add
COPY pinning_file /etc/apt/preferences.d/ubuntu-esm-infra-trusty
to you Dockerfile
before you run the sudo apt-get -y update
Solution 4:
In my case the problem was caused by the parent image since it had not cleared the apt cache properly.
I solve the problem including cleaning commands before the first apt update ...
RUN apt clean && \
rm -rf /var/lib/apt/lists/* && \
apt update && \
...
Hope this helps