best-practice for mid-sized shared storage
Our old 32TB RAID6 (net capacity: 26TB) system is showing its age and requires an upgrade, so we are currently thinking about building up a new 100TB system.
The storage is mostly used for our user's home directorie, for sharing project data (working with media, so projects can get up to 1TB) and for
The old storage is attached to a Debian host via iSCSI; the file server exports the data via NFS (for the un*x hosts) and SMB (for W32).
For practical reasons, we would like to keep that.
In the past we found that project data grows at unexpected ends, so we use a flat storage pool where all the projects (and home dirs) are located in the same filesystem (rather than pre-allocating a certain amount of storage capacity to a given project, and then having to re-allocate when hitting limits).
While this is a simple and flexible solution, I remember hitting some limitations even with the old setup. The biggest partition on the old RAID system is 20TB, because xfs wouldn't allow me to create a larger filesystem (back then; not to talk about ext4; and I didn't really consider zfs or btrfs - and still don't).
Since I haven't particularily followed recent progress on the storage front, I'm a bit lost what the current best-practices in creating such mid-sized storages are: JBOD, LVM, RAID, distributed filesystems; stacking which on which? Some blogs suggest that RAID6 is obsolete anyhow, but what are the alternatives?
My requirements are:
- ~100TB net capacity
- "flat" filesystem view
- redundant (survives multiple concurrent disk failures)
- white-box system
- runs on Debian (Debian/stretch)
iSCSI
scalable: ideally such a system would allow for growing (e.g. I start off with 70TB and later buy disks on the go until I reach full capacity, and the stack scales along: the filesystem grows and data is still stored redundantly, hopefully without having a handful of separate RAID6 systems).
Go ZFS. If you want Debian ZoL is your best friend.
https://github.com/zfsonlinux/zfs/wiki/Debian
RAID 6 is far from obsolete - RAID 5 is however. I'm not a fan of single-controller storage, single point of failure you see, but if you're happy with that then by all means carry on with something similar to what you have now. Of course what you may well choose to go for now is some form of flash-based front-end - either through caching or tiering - it depends where you want to do the actual RAIDing - I like hardware controllers, you may not. Either way if you can get to a point where 90% of your IO is coming from/to flash (SSD/NVME etc.) then your users will be very happy indeed.