ZFS vs XFS

Solution 1:

I've found XFS more well suited to extremely large filesystems with possibly many large files. I've had a functioning 3.6TB XFS filesystem for over 2 years now with no problems. Definately works better than ext3, etc at that size (especially when dealing with many large files and lots of I/O).

What you get with ZFS is device pooling, striping and other advanced features built into the filesystem itself. I can't speak to specifics (I'll let others comment), but from what I can tell, you'd want to use Solaris to get the most benefit here. It's also unclear to me how much ZFS helps if you're already using hardware RAID (as I am).

Solution 2:

ZFS will give you advantages beyond software RAID. The command structure is very thoughtfully laid out, and intuitive. It's also got compression, snapshots, cloning, filesystem send/receive, and cache devices (those fancy new SSD drives) to speed up indexing meta-data.

Compression:

#zfs set compression=on filesystem/home

It supports simple to create copy-on-write snapshots that can be live-mounted:

# zfs snapshot filesystem/home/user@tuesday
# cd filesystem/home/user/.zfs/snapshot/tuesday

Filesystem cloning:

# zfs clone filesystem/home/user@tuesday filesystem/home/user2

Filesystem send/receive:

# zfs send filesystem/home/user@tuesday | ssh otherserver "zfs receive -v filesystem/home/user"

Incremental send/receive:

# zfs send -i filesystem/home/user@tuesday | ssh otherserver "zfs receive -v filesystem/home/user"

Caching devices:

# zpool add filesystem cache ssddev

This is all just the tip of the iceberg, I would highly recommend getting your hands on an install of Open Solaris and trying this out.

http://www.opensolaris.org/os/TryOpenSolaris/

Edit: This is very old, Open Solaris has been discontinued, the best way to use ZFS is probably on Linux, or FreeBSD.

Full disclosure: I used to be a Sun storage architect, but I haven't worked for them in over a year, I'm just excited about this product.