Running multiple Dgraph alpha pod and zeros in single host

Users Dgraph
#1

Hi,

I am running Dgraph in minikube using dgraph-ha.yaml file having pod replicas 3 for alphas and zeros.

It’s a single node cluster setup, running 3 pods of alpha statefulsets and 3 pods of zeros statefulsets with persistent volume template pointing to different directories.

Is it possible to run this configuration on a single node cluster setup?

Will these 3 alpha pods create a alpha group and as well as 3 zeros create a zero group and where to mention the replica- settings?

Is container command section required for both zero and alpha spec in yaml file:

command:
- bash
- “-c”
- |
set -ex
[[ hostname =~ -([0-9]+)$ ]] || exit 1
ordinal=${BASH_REMATCH[1]}
idx=$(($ordinal + 1))
if [[ $ordinal -eq 0 ]]; then
exec dgraph zero --my=$(hostname -f):5080 --raft=“idx=$idx” --replicas 3
else
exec dgraph zero --my=$(hostname -f):5080 --peer dgraph-zero-0.dgraph-zero.${POD_NAMESPACE}.svc.cluster.local:5080 --raft=“idx=$idx” --replicas 3
fi

#2

@MichelDiz any inputs please or @mrjn is the above scenario works?

#3

What you mean by node? K8s node?
Well, yes? I’m confused with the question.

If your cluster is replica set to 3(which means it is HA). You gonna have a single Alpha group. Cuz you have only 3 Alpha nodes. Do you mean replica settings from Dgraph RAFT group or K8s replica settings? Please, make it clear. Or remove K8s from your example. As it is another scope.

Yes. This command will infer the SVC address created by K8s’s DNS service.

Your main question was in the title:

Running multiple Dgraph alpha pod and zeros in single host

If that is your question. Yes. You can run the whole cluster in a single machine whatever it be or where it be. What is your concern? In my opinion you shouldn’t do this. But you can. I’d prefer to run each Alpha or Alpha group in different hosts.

#4

Yes single k8s node like minikube

I am talking about - k8s replica settings - replicas:3 [in bold]
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: dgraph-zero
spec:
serviceName: “dgraph-zero”
replicas: 3

Yes I have verified it will create a single Alpha group. But my question is I have not mentioned the replica settings for Dgraph RAFT group while creating zero statefulset.

I have omitted the container whole command section while deploying zero statefulset in dgraph-ha.yaml file.
command:
- bash
- “-c”
- |
set -ex
[[ hostname =~ -([0-9]+)$ ]] || exit 1
ordinal=${BASH_REMATCH[1]}
idx=$(($ordinal + 1))
if [[ $ordinal -eq 0 ]]; then
exec dgraph zero --my=$(hostname -f):5080 --raft=“idx=$idx” --replicas 3
else
exec dgraph zero --my=$(hostname -f):5080 --peer dgraph-zero-0.dgraph-zero.${POD_NAMESPACE}.svc.cluster.local:5080 --raft=“idx=$idx” --replicas 3
fi

And while deploying alpha statefulset also, I have omitted the container command section:
command:
- bash
- “-c”
- |
set -ex
dgraph alpha --my=$(hostname -f):7080 --zero dgraph-zero-0.dgraph-zero.${POD_NAMESPACE}.svc.cluster.local:5080,dgraph-zero-1.dgraph-zero.${POD_NAMESPACE}.svc.cluster.local:5080,dgraph-zero-2.dgraph-zero.${POD_NAMESPACE}.svc.cluster.local:5080

Then how all these three alphas create a single alpha group?

#5

Hi @MichelDiz, I understood the best practice says to run each alpha or alpha group to run in dedicated hosts but this is for development purpose. Thanks for your reply.

But my confusion is, If, we don’t mention the replica settings for Dgraph RAFT group while deploying Dgraph zero, ideally every alpha node should creates it’s own alpha group right?

#6

How it creates a HA, because if we don’t mention replica settings of Dgraph RAFT group while deploying zero, then ideally every alpha creates it’s own group.

Surprisingly, I have noticed when we deploy dgraph-ha.yaml with k8s replica as 3 for both zero and alpha but omitting the container commands from zero and alpha statefulset which creates Dgraph RAFT group, still it forms a single dgraph alpha group with all those k8s replicas. How is that possible? Is it something I am missing or not understanding

#7

This is K8s replica. But in the context with Dgraph, means about the same thing. K8s will spin up 3 Alphas and 3 Zeros pods. In the RAFT they will become (the alphas) Dgraph replicas.

They will talk to the Zero Group leader and ask to join the cluster. The zero will ask them to create an election. They will elect a leader. Based on the Zero configs they will organize themselves as replicas. Done.

It is fine to run like this in a dev environment. Avoid doing this in prod.

Yes. If you don’t set the replication factor in the replica flag(for the zeros). Each Alpha will be a group. So you will endup with 3 groups. That’s the default behavior. Sharding all along.

In your copy paste commands you shown --replicas 3, so I assumed you are doing replication.

I mentioned above. The statefulset replication just replicates the pods. The “magic” relies in the replica flag. Dgraph does all the election based on the RAFT Algorithm.

#8

HI @MichelDiz , Thanks for your reply and making me understand the inner logic but we are neither executing any of these commands for zero and alpha deployment. Means, using the dgraph-ha.yaml file we are deploying 3 k8s replica for both zero and alpha but we are not executing any container commands where we mention RAFT group replica settings. Still we saw one alpha group created rather than 3 alpha groups.

#9

Based on the Zero configs – no zero configs we are providing

#10

If you wanna remove replication. Remove this flag from “–replicas 3” zero.
But do better, don’t use the HA template. Use the other one. HA is basically replication.

Yes you are. The flags in the yml configs.

I’m quite confused by your purpose here. One moment I think you want to understand how replication works, and now it seems like you don’t want replication. The other time it seems your concern is running all instances of Dgraph on the same host. lol