Does the galera cluster have to have an odd number of nodes?

Solution 1:

No, 8 is OK. The "quorum" mechanism needs a majority. If you have at least 3 nodes (and only one fails), the remaining nodes can establish a quorum.

If your 8 nodes are split with 4 in each of two datacenters, then a datacenter outage would kill the entire cluster. To protect against datacenter failure, you should spread the nodes 'evenly' across 3 (or more) datacenters. For 8 nodes: 3+3+2 is the only safe split. 4+3+1 or 4+2+2 would kill the cluster if the '4' datacenter went down.

How well is the 8-node cluster working? (That's the biggest one I have heard of.)