btrfs Integrity and Stress Testing

A DevOps colleague is recommending that we begin transitioning our production environment to using btrfs. We have primarily ext4 filesystems, though some low-usage servers using ZFS (on Linux). As one of the decision-makers, and as one responsible for our overall environment, I am hesitant based upon the number of comments and articles around btrfs in production on the web. To counter that argument Oracle has released their Enterprise Linux with btrfs support, SLES 12 (https://www.suse.com/releasenotes/x86_64/SUSE-SLES/12/) also indicates it will be using btrfs, and there is evidence that companies like Facebook are also using it in controlled production environments.

There are a number of arguments made as to why moving in this direction (adopting btrfs) will be a good thing, and I agree with them overall, however, I would like to be prudent, do due diligence, and get more operational familiarity and hours logged in a "small production" or staging environment before moving forward on a wider scale. Are there any tools that can help me build the case - as in stress testing followed by data integrity checks or something along those lines? Other than not seeing statements like: "Q. Is btrfs stable? Short answer: No, it's still considered experimental." on the btrfs wiki what else can I do to get a warmer fuzzy?

Solution 1:

I've given an elaborate answer already here: Is btrfs production ready?

In short: Btrfs is nothing you already want on a production file server.

Here are the reasons why: it becomes unpredictable if space is running out, though most core features are considered stable, others aren't yet and it is still a fast moving target. Being a fast moving target means you should always use the latest and greatest kernel, which is something you don't want on a server.

Also even still with kernel 3.16 it is known to produce deadlocks, which is for sure something you don't want on a production server (http://marc.merlins.org/perso/btrfs/post_2014-10-05_Btrfs-Tips_-Catch-Btrfs-Deadlocks.html). Also some RAIDs are still experimental, like RAID5/6. You can use those, but not scrub data there yet and other stuff.

Fedora wants finally to make Btrfs maybe the default file system with v23, which is going to happen at the end of 2015. Red Hat switched from Ext4 to XFS as default now.

If you really need a COW production ready file system, do yourself a favor and take ZFS, either as ZFS on Linux or under FreeBSD.

And read Russel Coker's blog about his experience with Btrfs. http://etbe.coker.com.au/

Solution 2:

I'd stick with mainstream filesystems like XFS (as promoted by Red Hat) and rely on ZFS for advanced filesystem needs.

This is mainly due to:

• Mindshare: ZFS knowledge is held in the Linux, FreeBSD and Solaris/OpenIndiana communities.
• Maturity: The ZFS codebase is proven and has been around for awhile. Best-practices have evolved and I can't recall seeing horror stories about data loss.
• Portability: I've definitely changed platforms and retained ZFS compatibility throughout.

Short of specific integration with Linux containers, I don't know that BtrFS offers much over ZFS today. If you have hesitation about using BtrFS, listen to your gut. I don't think I'd bet the business on it, but things can change over time. Evaluate your specific needs. Are there compelling features in BtrFS that your environment can take advantage of?

Take a sample view of Server Fault questions about BtrFS to see what issues people routinely face.