ZFS pool not mounting correctly; can't find all files
I have a ZFS pool (mirrored RAID) on two disks that I've transferred from a FreeBSD 11.0 server into an Ubuntu 16.04.2 LTS server.
After importing the pool, I see:
NAME USED AVAIL REFER MOUNTPOINT
zroot 993G 790G 104K /mnt/zfs
zroot/ROOT 989G 790G 96K none
zroot/ROOT/default 989G 790G 989G none
zroot/tmp 140K 790G 140K none
zroot/usr 3.37G 790G 96K none
zroot/usr/home 2.15G 790G 2.15G none
zroot/usr/ports 640M 790G 640M none
zroot/usr/src 608M 790G 608M none
zroot/var 1.66M 790G 96K none
zroot/var/audit 96K 790G 96K none
zroot/var/crash 96K 790G 96K none
zroot/var/log 476K 790G 476K none
zroot/var/mail 840K 790G 840K none
zroot/var/tmp 96K 790G 96K none
What I expect to see is a fully-populated directory tree under /mnt/zfs
on the Ubuntu system, containing 993GiB of files. What I see instead is a partially-populated tree of directories, with no files inside them:
$ tree /mnt/zfs
/mnt/zfs
├── tmp
├── usr
│ ├── home
│ ├── ports
│ └── src
└── var
├── audit
├── crash
├── log
├── mail
└── tmp
I'm pretty sure I'm missing something fundamental here.
Attempt #1: Setting a mountpoint for ROOT, as per @Zoredache's suggestion, just results in an empty ROOT directory:
$ sudo zfs set mountpoint=/mnt/zfs/ROOT zroot/ROOT
$ tree /mnt/zfs
/mnt/zfs
└── ROOT
Attempt #2: Setting a mountpoint for ROOT/default, as per @user121391's suggestion, also yields an empty directory:
$ sudo zfs set mountpoint=/mnt/zfs/ROOT zroot/ROOT/default
$ tree /mnt/zfs
/mnt/zfs
0 directories, 0 files
Attempt #3: Mounting an individual directory seems to work, as per @user131391's other suggestion:
$ sudo zfs set mountpoint=/mnt/zfs/usr/home zroot/usr/home
$ tree /mnt/zfs
/mnt/zfs
└── usr
└── home
└── duncan
├── tmp
│ ├── code
...
└── usb
17978 directories, 67539 files
Which is great, but unfortunately that's no help. The files I'm after are under /usr/local/
, which doesn't appear in the result of zfs list
. The space is clearly being used, though, if you look at the sizes:
zroot/ROOT/default 989G 790G 989G none
zroot/usr 3.37G 790G 96K none
Attempt #4:
Further information for Michael Kjörling:
$ sudo zfs set mountpoint=/mnt/zfs/ROOT zroot/ROOT/default
$ sudo zfs get mountpoint,mounted zroot -t filesystem -r
NAME PROPERTY VALUE SOURCE
zroot mountpoint none local
zroot mounted no -
zroot/ROOT mountpoint none local
zroot/ROOT mounted no -
zroot/ROOT/default mountpoint /mnt/zfs/ROOT local
zroot/ROOT/default mounted no -
zroot/tmp mountpoint none local
zroot/tmp mounted no -
zroot/usr mountpoint none local
zroot/usr mounted no -
zroot/usr/home mountpoint none local
zroot/usr/home mounted no -
zroot/usr/ports mountpoint none local
zroot/usr/ports mounted no -
zroot/usr/src mountpoint none local
zroot/usr/src mounted no -
zroot/var mountpoint none local
zroot/var mounted no -
zroot/var/audit mountpoint none local
zroot/var/audit mounted no -
zroot/var/crash mountpoint none local
zroot/var/crash mounted no -
zroot/var/log mountpoint none local
zroot/var/log mounted no -
zroot/var/mail mountpoint none local
zroot/var/mail mounted no -
zroot/var/tmp mountpoint none local
zroot/var/tmp mounted no -
$ tree /mnt/zfs
/mnt/zfs
0 directories, 0 files
Solution 1:
zroot and zroot/ROOT don't contain any files, they are just containers for the other data.
The bulk of your data is in zroot/ROOT/default.
The other zfs filesystems should inherit their mountpoint from zroot (not sure how this information got lost in the transition from FreeBSD to Ubuntu):
zfs inherit mountpoint zroot/tmp
zfs inherit mountpoint zroot/usr
zfs inherit mountpoint zroot/usr/home
zfs inherit mountpoint zroot/usr/ports
... and so on (it may be possible to do a -r on this but I haven't checked).
The only zfs filesystem which should have a mountpoint of none
is zroot/ROOT:
zfs set mountpoint=none zroot/ROOT
After doing all this, you'll need to export and re-import the pool, and mount the filesystems in the correct order:
zpool export zroot
Import the pool to the new mountpoint but don't actually mount anything:
zpool import -a -N -r /mnt/zfs
Mount the root pool:
zfs mount zroot/ROOT/default
Mount everything else:
zfs mount -a
Solution 2:
Check the MOUNTPOINT
column. It looks like there are no mountpoints for anything. You might want to try setting mountpoints for your datasets.
For example I believe this would mount zroot/ROOT
to /mnt/zfs/ROOT
.
zfs set mountpoint=/mnt/zfs/ROOT zroot/ROOT
I don't know enough about ZFS to tell you why this wouldn't happen automatically, or if it should when importing onto a Linux system.