Docker service starts before ZFS

I am using Docker CE on Ubuntu 16.04, with ZFS as storage for Docker. Setup is pretty much standard:

1. There's a zpool that has multiple disks
2. On the zpool, there's a zfs filesystem with mountpoint /var/lib/docker.
3. Docker sees that the filesystem is ZFS, and uses that automatically

Everything works most of the time. However, once in a while when my VM boots up, the zpool fails to mount. I believe (though I am not 100% sure) that this is because the Docker service starts before the ZFS filesystem is mounted, and in fact I see a /var/lib/docker folder created on the root FS.

How can I ensure that the Docker service does not start until all ZFS filesystems are mounted?

Solution 1:

I was able to solve this by doing two things. Note that one alone might be sufficient.

First, explicitly tell Docker to use ZFS as filesystem, by writing {"storage-driver": "zfs"} in the file /etc/docker/daemon.json. (If the file exists in your disk, then just add the storage-driver key)

Second, create the following systemd unit in the file /etc/systemd/system/docker-wait-zfs.service:

[Unit]
Description=Wait for ZFS before starting Docker
RequiredBy=docker.service
Before=docker.service
Requires=zfs.target
After=zfs.target

[Service]
Type=oneshot
ExecStart=/bin/true
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Then execute:

systemctl enable docker-wait-zfs.service

Solution 2:

The answer here did not work for me after a recent update. It seems you cannot have "{"storage-driver": "zfs"}" in your "/etc/docker/daemon.json" anymore, unless the root drive is using ZFS. This was not applicable to me, since I was just mapping folders to my containers which happen to reside on ZFS.

To fix this, the following worked:

1. sudo systemctl edit docker.service

2. Add the following contents:

   After=zfs-mount.service Requires=zfs-mount.service Wants=zfs-mount.service BindsTo=zfs-mount.service

The answer here, contains explanation for what each line does.