Gluster: What's a Brick vs a Node
I'm having a small problem understanding Gluster Bricks and Nodes. In particular, on the 3.3.0 Administration Guide, eg, page 12 - 13, the discussion on "Striped Volumes" vs "Distributed Striped Volumes". See chapter 5, page 9, for the overview.
I'm looking to use it in a VMware ESXi 5.X environment for a HA NFS storage solution. As such the striped and distributed striped volumes are of interest as they relate to "high concurrency environments accessing very large files". This adequately describes Virtual Disks.
However, when reading the manual I have this:
- Striped volumes stripes data across bricks.
- Distributed Striped volumes stripe data across two or more nodes.
So I assumed a brick is (for example) a block device on a node (or a peer as they seem to refer to it sometimes). And a node is a distinct server (one of the peers).
However, the diagram on Striped Volumes shows server1 and server2 so that assumption seems wrong.
But then the distributed striped volume also shows the same thing, but differently. Now it shows server1 having two bricks (exp1 and exp2) which it calls 'striped volume 0' and then server2 also with two bricks (exp3 and exp4) called 'striped volume 1'.
The sample creation shows creating an 8 node 4 stripe volume which obviously would be different from the diagram having only 1 brick per node.
Also the example to create both types uses the same syntax.
Can somebody help with a clarification? What's a brick, and what's a node? Do I have it right and the diagram for 'striped' is in fact wrong, actually showing a distributed striped volume?
A brick is any directory on an underlying disk filesystem. For example, you could create 4 "bricks" as /var/export/brick0 /var/export/brick1... etc., even though those directories were all on one disk partition. Not that we recommend this, of course, but it's possible. At its most basic, a brick is simply any filesystem you can export as a GlusterFS mount point. In that way, it's similar to an NFS directory export.
A node is a box in a trusted group of peers.
-John Mark Gluster Community Lead
The way I understand these things is as 3 separate ideas. You then combine them depending on your needs. Therefore you need to ask yourself 3 questions:
- If one of the bricks goes down (hardware failure) do I need the file to still be accessible? Highly available? If yes, then you need replication. NOTE: this is not a substitute for backups. If someone deletes a file, it is deleted everywhere the data is replicated. Retrieving this file would need to be from a backup.
- Will I need more storage space in the future? If yes, then you need distribution. This makes it so that as you need more space, everything gets spread out evenly over the number of bricks you have. Adding a new brick gives you more space, and re-distributes the existing files over all the bricks.
- Are the files really big, and a significant performance gain will be had by allowing files to have parallel access? If yes, you need striping. Striping spreads the file over different bricks for faster access.
You then combine your answers to these questions and decide on the volume that will work the best for you.