How can I backup a Docker-container with its data-volumes?

I've been using this Docker-image tutum/wordpress to demonstrate a Wordpress website. Recently I found out that the image uses volumes for the MySQL-data.

So the problem is this: If I want to backup and restore the container I can try to commit an image, and then later delete the container, and create a new container from the committed image. But if I do that the volume gets deleted and all my data is gone.

There must be some simple way to backup my container plus its volume-data but I can't find it anywhere.

Solution 1:

if I want to revert the container I can try to commit an image, and then later delete the container, and create a new container from the committed image. But if I do that the volume gets deleted and all my data is gone

As the docker user guide explains, data volumes are meant to persist data outside of a container filesystem. This also ease the sharing of data between multiple containers.

While Docker will never delete data in volumes (unless you delete the associated container with docker rm -v), volumes that are not referenced by any docker container are called dangling volumes. Those dangling volumes are difficult to get rid of and difficult to access.

This means that as soon as the last container using a volume is deleted, the data volume becomes dangling and its content difficult to access.

In order to prevent those dangling volumes, the trick is to create an additional docker container using the data volume you want to persist so that there will always be at least that docker container referencing the volume. This way you can delete the docker container running the wordpress app without losing the ease of access to that data volume content.

Such containers are called data volume containers.

There must be some simple way to back up my container plus volume data but I can't find it anywhere.

backup docker images

To backup docker images, use the docker save command that will produce a tar archive that can be used later on to create a new docker image with the docker load command.

backup docker containers

You can backup a docker container by different means

by committing a new docker image based on the docker container current state using the docker commit command
by exporting the docker container file system as a tar archive using the docker export command. You can later on create a new docker image from that tar archive with the docker import command.

Be aware that those commands will only backup the docker container layered file system. This excludes the data volumes.

backup docker data volumes

To backup a data volume you can run a new container using the volume you want to backup and executing the tar command to produce an archive of the volume content as described in the docker user guide.

In your particular case, the data volume is used to store the data for a MySQL server. So if you want to export a tar archive for this volume, you will need to stop the MySQL server first. To do so you will have to stop the wordpress container.

backup the MySQL data

An other way is to remotely connect to the MySQL server to produce a database dump with the mysqldump command. However in order for this to work, your MySQL server must be configured to accept remote connections and also have a user who is allowed to connect remotely. This might not be the case with the wordpress docker image you are using.

Edit

Docker recently introduced Docker volume plugins which allow to delegate the handling of volumes to plugins implemented by vendors.

The docker run command has a new behavior for the -v option. It is now possible to pass it a volume name. Volumes created in that way are named and easy to reference later on, easing the issues with dangling volumes.

Edit 2

Docker introduced the docker volume prune command to delete all dangling volumes easily.

Solution 2:

UPDATE 2

Raw single volume backup bash script:

#!/bin/bash
# This script allows you to backup a single volume from a container
# Data in given volume is saved in the current directory in a tar archive.
CONTAINER_NAME=$1
VOLUME_NAME=$2

usage() {
  echo "Usage: $0 [container name] [volume name]"
  exit 1
}

if [ -z $CONTAINER_NAME ]
then
  echo "Error: missing container name parameter."
  usage
fi

if [ -z $VOLUME_NAME ]
then
  echo "Error: missing volume name parameter."
  usage
fi

sudo docker run --rm --volumes-from $CONTAINER_NAME -v $(pwd):/backup busybox tar cvf /backup/backup.tar $VOLUME_NAME

Raw single volume restore bash script:

#!/bin/bash
# This script allows you to restore a single volume from a container
# Data in restored in volume with same backupped path
NEW_CONTAINER_NAME=$1

usage() {
  echo "Usage: $0 [container name]"
  exit 1
}

if [ -z $NEW_CONTAINER_NAME ]
then
  echo "Error: missing container name parameter."
  usage
fi

sudo docker run --rm --volumes-from $NEW_CONTAINER_NAME -v $(pwd):/backup busybox tar xvf /backup/backup.tar

Usage can be like this:

$ volume_backup.sh old_container /srv/www
$ sudo docker stop old_container && sudo docker rm old_container
$ sudo docker run -d --name new_container myrepo/new_container
$ volume_restore.sh new_container

Assumptions are: backup file is named backup.tar, it resides in the same directory as backup and restore script, volume name is the same between containers.

UPDATE

It seems to me that backupping volumes from containers is not different from backupping volumes from data containers.

Volumes are nothing else than paths linked to a container so the process is the same.

I don't know if docker-backup works also for same container volumes but you can use:

sudo docker run --rm --volumes-from yourcontainer -v $(pwd):/backup busybox tar cvf /backup/backup.tar /data

and:

sudo docker run --rm --volumes-from yournewcontainer -v $(pwd):/backup busybox tar xvf /backup/backup.tar

END UPDATE

There is this nice tool available which lets you backup and restore docker volumes containers:

https://github.com/discordianfish/docker-backup

if you have a container linked to some container volumes like this:

$ docker run --volumes-from=my-data-container --name my-server ...

you can backup all the volumes like this:

$ docker-backup store my-server-backup.tar my-server

and restore like this:

$ docker-backup restore my-server-backup.tar

Or you can follow the official way:

How to port data-only volumes from one host to another?

Solution 3:

If you only need to backup mounted volumes you can just copy folders from your Dockerhost.

Note: If you are on Ubuntu, Dockerhost is your local machine. If you are on Mac, Dockerhost is your virtual machine.

On Ubuntu

You can find all folders with volumes here: /var/lib/docker/volumes/ so you can copy them and archive wherever you want.

On MAC

It's not so easy as on Ubuntu. You need to copy files from VM.

Here is a script of how to copy all folders with volumes from virtual machine (where Docker server is running) to your local machine. We assume that your docker-machine VM named default.

docker-machine ssh default sudo cp -v -R /var/lib/docker/volumes/ /home/docker/volumes

docker-machine ssh default sudo chmod -R 777 /home/docker/volumes

docker-machine scp -R default:/home/docker/volumes ./backup_volumes

docker-machine ssh default sudo rm -r /home/docker/volumes

It is going to create a folder ./backup_volumes in your current directory and copy all volumes to this folder.

Here is a script of how to copy all saved volumes from your local directory (./backup_volumes) to Dockerhost machine

docker-machine scp -r ./backup_volumes default:/home/docker

docker-machine ssh default sudo mv -f /home/docker/backup_volumes /home/docker/volumes

docker-machine ssh default sudo chmod -R 777 /home/docker/volumes

docker-machine ssh default sudo cp -v -R /home/docker/volumes /var/lib/docker/

docker-machine ssh default sudo rm -r /home/docker/volumes

Now you can check if it works by:

docker volume ls

Solution 4:

If your project uses docker-compose, here is an approach for backing up and restoring your volumes.

docker-compose.yml

Basically you add db-backup and db-restore services to your docker-compose.yml file, and adapt it for the name of your volume. My volume is named dbdata in this example.

version: "3"

services:
  db:
    image: percona:5.7
    volumes:
      - dbdata:/var/lib/mysql

  db-backup:
    image: alpine    
    tty: false
    environment:
      - TARGET=dbdata
    volumes:
      - ./backup:/backup
      - dbdata:/volume
    command: sh -c "tar -cjf /backup/$${TARGET}.tar.bz2 -C /volume ./"

  db-restore:
    image: alpine    
    environment:
      - SOURCE=dbdata
    volumes:
      - ./backup:/backup
      - dbdata:/volume
    command: sh -c "rm -rf /volume/* /volume/..?* /volume/.[!.]* ; tar -C /volume/ -xjf /backup/$${SOURCE}.tar.bz2"

Avoid corruption

For data consistency, stop your db container before backing up or restoring

docker-compose stop db

Backing up

To back up to the default destination (backup/dbdata.tar.bz2):

docker-compose run --rm db-backup

Or, if you want to specify an alternate target name, do:

docker-compose run --rm -e TARGET=mybackup db-backup

Restoring

To restore from backup/dbdata.tar.bz2, do:

docker-compose run --rm db-restore

Or restore from a specific file using:

docker-compose run --rm -e SOURCE=mybackup db-restore

I adapted commands from https://loomchild.net/2017/03/26/backup-restore-docker-named-volumes/ to create this approach.