High Availability Across AZs Configuration questions

Users Dgraph
1

First let me apologize for having posted this twice but, I could not figure out how to tag the first one as a question.
Continuing the discussion from Replicas across availability zones :

I am planning to implement the same thing, replication across AZs, and looking to prevent having the entire cluster go down when an AZ goes down. I am a little lost in the terminology, most specifically the seemingly interchangeable use of servers, nodes, alphas and zeros.

In the second diagram from @nickpoorman’s post if he lost say, AZ-1, would the cluster go down because there is not a majority of alphas to run Group1? Also, when it says the number of replicas is the number of nodes for a group I assume that refers to the number of Alphas and that the number of zeros does not come into play for a quorum? I also assume that the number of Alpha nodes must be a multiple of the number of replicas specified for the failover to work properly?

In the raft=‘group=…’ I am assuming that refers to the raft id provided in the zero command so given a zero command of…

dgraph zero --my=zero0:5080 --replicas 3 --raft=“idx=0”

then the alpha command would be

dgraph alpha --my=alpha1:7080 --zero=zero0:5080 --raft=‘group=0’

From which I infer that I need to have at least one zero for each group. May I have 2 zeros per group
that I could distribute across AZs equally? as in…
AZ-1
dgraph zero --my=zero0:5080 --replicas 3 --raft=“idx=0”

AZ-2
dgraph zero --my=zero0:5080 --replicas 3 --raft=“idx=0”

So, to keep the cluster up when an AZ goes down I would need replicas=5 and cross at least 3 AZs (ignoring latency issues for now)

AZ-1 Alphas
Group-1, Group-1, Group-2
Zero0

AZ-2 Alphas
Group-1, Group-1, Group-2, Group-2
Zero1

AZ-3 Alphas
Group-2, Group-2, Group-1
Zero2

And in that way losing one AZ would never take out a majority of the Alphas in a quorum.

Is that correct? And how many zeros should I have and how should I distribute them?

2

If the configuration in AZ-1 is Group1,Group1,Group2 and AZ-2 is Group1,Group2,Group2, then AZ-1 has two out of the three nodes in Group 1 (aka the majority) and if AZ-1 went down then group 1 would not be available for linearizable reads and writes (it would still be available for best-effort reads).

The --replicas configuration determines the max replica size per group for Alphas. Zeros are also part of a quorum (e.g., you can run three Zeros in a quorum and the Zero group is available when the majority aka two are up).

If --replicas=3, then you’d ideally want 3 Alphas per group to maintain a high availability configuration.

If you have 3 AZs, then there at least needs to be one replica per AZ to maintain availability in a single AZ going down. So, you only need replicas=3 at the minimum.

Three Zeros for a high availability quorum. One per AZ.

Zero comprises group 0 (hence the name). Alphas are part of group 1, 2, and so on. So, When configuring the Alpha and you want to specify the group (it’s optional—when a new Alpha connects to Zero it will get auto-assigned a group), you’d specify group 1 or higher (not group=0, that’s not valid).

3

Thank you

1 Like