Docker-Swarm 1.13 with 3 Nodes acting as Master/Worker

We are just trying out Docker-Swarm with 3 Nodes, each node is acting as Master and as Worker. This works fine, but i don't know if the following behavior is "normal".

If I shutdown one of those 3 nodes, the swarm handles the reallocation of the services fine.

If I shutdown two of those 3 nodes, the services are still working but if i connect to the "last man standing" and try to get my swarm overview with:

docker node ls

i get the following error message:

Error response from daemon: rpc error: code = 4 desc = context deadline exceeded

I am not sure if the swarm can operate when only one node is left? i would guess .. yes?


You have an explanation here : https://docs.docker.com/engine/swarm/raft/

Raft tolerates up to (N-1)/2 failures and requires a majority or quorum of (N/2)+1 members to agree on values proposed to the cluster.

When you are running 3 manager nodes, your cluster can accept to have one node down, because the remaining 2 can agree that the third one is dead.

If you have only one node remaining, maybe it's the one that have a problem. So it will not accept any changes, and you will not have an inconsistent state if your network split.