Elasticsearch read and write consistency
I think I can answer my own question now :)
Regarding the first question, by re-re-reading the documentation (this and this) a few times :) I realized that this statement should be right:
Index operation return when all live/active shards have finished indexing, regardless of consistency param. Consistency param may only prevent the operation to start if there are not enough available shards(nodes).
So for example, if there are 3 shards (one primary and two replicas), and all shards are available - the operation will be waiting for all 3 (considering that all 3 are live/available), regardless of consistency param (even when consistency=one
)
This makes the system consistent (at least the document-api part); unless there is a network-partition.
But, I didn't have a chance to test this yet.
UPDATE: by consistency here, I don't mean ACID-consistency, it is just the guarantee that all replicas are updated at the moment when request is returned.
Regarding the second question:
The obvious answer is - it is randomized
to spread the load; on the other hand, a client can pick a random node to talk to, but probably it is not 100% efficient as a single request may need multiple shards.
Write: I am not sure if above is true for ES 6.1 https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-wait-for-active-shards says that index operation returns if the primary shard is active and can be changed to other values.
Searches are random so setting wait for active shard field to all will ensure write is successful if it is performed on all shards.
Read: Preference can still be used but it is marked as deprecated.