BTRFS on RHEL8 - compiling kernel module or making userspace tools work
Context
I recently installed RHEL 8 without realizing that it no longer supports BTRFS.
Unfortunately, I have 4 disks in BTRFS RAID10. I don't have enough space on my other disks to hold the data present on the BTRFS disks, so copying it all elsewhere while booted from a USB drive is out the window.
I have my initial question, then some followup questions regarding the approaches I took that failed. Feel free to just focus on the "how to get BTRFS working" part of this problem, although I am curious to learn about the other issues if you have answers to any part of this.
Question 1
- Is there a (relatively) easy way to get BTRFS working on RHEL8?
Userspace btrfs-progs
My first attempt to make this work was to compile and install btrfs-progs with the following:
# Install deps
sudo dnf install libuuid-devel libblkid-devel lzo-devel zlib-devel libzstd-devel e2fsprogs-devel e2fsprogs-libs e2fsprogs libgcrypt-devel libsodium-devel libattr-devel
# Install deps for doc gen
sudo dnf install asciidoc xmlto source-highlight
# Shallow-clone latest release
git clone --depth 1 --branch v5.14.1 https://github.com/kdave/btrfs-progs.git
cd btrfs-progs
git switch -c v5.14.1
# Build
# --disable-zoned since that feature needs kernel >=5.10
export CFLAGS="-O3 -pipe -frecord-gcc-switches -mtune=native -march=native"
export CPPFLAGS=$CFLAGS
export SODIUM_CFLAGS=$CFLAGS
export ZSTD_CFLAGS=$CFLAGS
export ZLIB_CFLAGS=$CFLAGS
export UUID_CFLAGS=$CFLAGS
export PYTHON_CFLAGS=$CFLAGS
./autogen.sh
./configure --with-crypto=libsodium --disable-zoned
make -j12
sudo make install
It seems to install correctly and is accessible to my user:
$ which btrfs
/usr/local/bin/btrfs
$ ls -1 /usr/local/bin/ | grep btrfs
btrfs
btrfsck
btrfs-convert
btrfs-find-root
btrfs-image
btrfs-map-logical
btrfs-select-super
btrfstune
fsck.btrfs
mkfs.btrfs
$ btrfs version
btrfs-progs v5.14.1
However, root apparently doesn't have /usr/local/bin in its path by default. I added export PATH+=":/usr/local/bin" to /etc/profileand/root/.bash_profile, but neither of them seem to get sourced automatically when using sudoor when dropping into a root shell withsudo su`.
When specifying the full path to the binary, it complains it can't open /dev/btrfs-control. Querying my local search engine, some suggest needing udev, but that's already installed (maybe misconfigured?)
$ sudo btrfs version
sudo: btrfs: command not found
$ sudo /usr/local/bin/btrfs device scan
Scanning for Btrfs filesystems
WARNING: failed to open /dev/btrfs-control, skipping device registration: No such file or directory
WARNING: failed to open /dev/btrfs-control, skipping device registration: No such file or directory
WARNING: failed to open /dev/btrfs-control, skipping device registration: No such file or directory
WARNING: failed to open /dev/btrfs-control, skipping device registration: No such file or directory
ERROR: there were 4 errors while registering devices
Other BTRFS commands seem to work:
$ sudo /usr/local/bin/btrfs filesystem show /dev/sda
Label: 'wdred' uuid: aaaa-bbbb-cccc-dddd-eeee
Total devices 4 FS bytes used 2.13TiB
devid 1 size 5.46TiB used 1.07TiB path /dev/sda
devid 2 size 5.46TiB used 1.07TiB path /dev/sdc
devid 3 size 5.46TiB used 1.07TiB path /dev/sdb
devid 4 size 5.46TiB used 1.07TiB path /dev/sdd
However, I've been afraid to mount the partitions or perform any operations on them given the above error, for fear that its missing components that would cause it to mangle my data.
Questions 2, 3, 4
- Is it safe to attempt to use btrfs to mount the disks given the above errors?
- What's up with the
/dev/btrfs-controlerror onbtrfs device scan? - How can I get
sudoandsudo suto have/usr/local/binin its path by default?
BTRFS kernel module
I wondered if it might be better to compile a kernel module, but having almost no kernel hacking experience, it went badly.
It seems like I need to set CONFIG_BTRFS_FS=m in my kernel config to start. It's not there currently, and I seemed to remember being able to do this in menuconfig.
$ grep "BTRFS" /boot/config-4.18.0-305.19.1.el8_4.x86_64
# CONFIG_BTRFS_FS is not set
The RHEL documentation mentions how to load kernel modules and such, but not about how to build them. I consulted archwiki, and tried to download the RHEL8 kernel from the Red Hat site. The download page for RHEL8 had a "Sources" tab with a 20G .iso file. I downloaded it, mounted it, and found it was stuffed with .rpm files and didn't look anything like the linux kernel source repo. I was a little lost.
I then went to /usr/src/kernels/, initialized a git repo out of fear I would clobber something important, and went on trying to figure out how to build the kernel module or change things in menuconfig.
$ cd /usr/src/kernels/4.18.0-305.19.1.el8_4.x86_64
$ sudo su
# git init
# git add -A
# git commit -m "Unmodified kernel"
# make mrproper
HOSTCC scripts/basic/bin2c
scripts/kconfig/conf --syncconfig Kconfig
arch/x86/Makefile:184: *** Compiler lacks asm-goto support.. Stop.
make: *** [Makefile:1361: _clean_arch/x86] Error 2
Lacking asm-goto support, the internet suggested that I may need elfutils-libelf-devel but I appeared to already have that.
For funzies I tried building it with clang and with gcc-toolset-10, but both of those had the same error.
Question 5
- Any ideas why
Compiler lacks asm-goto support? - What's a good resource on how to build kernel modules / patch kernels / modify your system's kernel?
System info
$ uname -a
Linux rhel 4.18.0-305.19.1.el8_4.x86_64 #1 SMP Tue Sep 7 07:07:31 EDT 2021 x86_64 x86_64 x86_64 GNU/Linux
$ gcc --version
gcc (GCC) 8.4.1 20200928 (Red Hat 8.4.1-1)
$ scl run gcc-toolset-10 'gcc --version'
gcc (GCC) 10.2.1 20201112 (Red Hat 10.2.1-8)
$ clang --version
clang version 11.0.0 (Red Hat 11.0.0-1.module+el8.4.0+8598+a071fcd5)
Thanks for reading this far! Any help is appreciated.
In the comments, @MichaelHampton's answer worked.
I added kernel-ml from ELrepo like this:
sudo rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
sudo yum install https://www.elrepo.org/elrepo-release-8.el8.elrepo.noarch.rpm
sudo yum --enablerepo=elrepo-kernel install kernel-ml
Note that I already built the userspace tools btrfs-progs and outlined the build steps in the question above. If you're coming here from Google, you'll need to install btrfs-progs in addition to the kernel module.
Then restarted, checked that I was running the mainline kernel, and was able to mount my disks with BTRFS and read their data.
$ sudo reboot
$ uname -a
Linux rhel 5.14.8-1.el8.elrepo.x86_64 #1 SMP Sat Sep 25 10:32:52 EDT 2021 x86_64 x86_64 x86_64 GNU/Linux
$ sudo mount UUID=zzz -o defaults,noatime /mnt/hdd
$ ls /mnt/hdd
all my files :)