How do I remove a replicated glusterfs peer?

I'm no longer in need of a replicated cluster of glusterfs storage servers, and can manage with just one. How do I get rid of the others?


SSH into the glusterfs machine you wish to keep and do:

[siddhartha@glusterfs-01-perf ~]$ sudo gluster peer status

Number of Peers: 1

Hostname: 10.240.0.123
Port: 24007
Uuid: 03747753-a2cc-47dc-8989-62203a7d31cd
State: Peer in Cluster (Connected)

This shows us our other peer which we wish to get rid of.

To detach this, try:

sudo gluster peer detach 10.240.0.123

You might fail with:

peer detach: failed: Brick(s) with the peer 10.240.0.123 exist in cluster

We need to be getting rid of the brick first:

[siddhartha@glusterfs-01-perf ~]$ sudo gluster volume info

Volume Name: glusterfs
Type: Replicate
Volume ID: 563f8593-4592-430f-9f0b-c9472c12570b
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: 10.240.0.122:/mnt/storage/glusterfs
Brick2: 10.240.0.123:/mnt/storage/glusterfs

To remove Brick2, do:

[siddhartha@glusterfs-01-perf ~]$ sudo gluster volume remove-brick glusterfs 10.240.0.123:/mnt/storage/glusterfs

This might fail with:

Removing brick(s) can result in data loss. Do you want to Continue? (y/n) y
volume remove-brick commit force: failed: Removing bricks from replicate configuration is not allowed without reducing replica count explicitly.

Our replication is set to 2, and needs to be explicitly be reduced to 1, so add a replica 1 flag to the previous command:

[siddhartha@glusterfs-01-perf ~]$ sudo gluster volume remove-brick glusterfs replica 1 10.240.0.123:/mnt/storage/glusterfs 
Removing brick(s) can result in data loss. Do you want to Continue? (y/n) y
volume remove-brick commit force: success

This should do the trick:

[siddhartha@glusterfs-01-perf ~]$ sudo gluster volume info glusterfs

Volume Name: glusterfs
Type: Distribute
Volume ID: 563f8593-4592-430f-9f0b-c9472c12570b
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: 10.240.0.122:/mnt/storage/glusterfs

You can probably go terminate the other machine.


Removing the bricks is well described in Gluster documentation, https://docs.gluster.org/en/latest/Administrator%20Guide/Managing%20Volumes/

The differences from the currently accepted answer from Siddhartha include

  • sudo gluster volume remove-brick glusterfs 10.240.0.123:/mnt/storage/glusterfs requires adding start at the end

Removing the peer is documented at https://docs.gluster.org/en/latest/Administrator%20Guide/Storage%20Pools/#removing-servers. That command does not appear changed.

The steps can be performed on either machine.


It worked for me on a 3 node cluster on Raspberry Pi 4 using Raspbian Buster with:

sudo gluster volume remove-brick rep_swarm_vol replica 2 picluster2:/gfs/bricks/2/brick force

It had to have that force command at the end to work.