I am running Dgraph in minikube using dgraph-ha.yaml file having pod replicas 3 for alphas and zeros.
It’s a single node cluster setup, running 3 pods of alpha statefulsets and 3 pods of zeros statefulsets with persistent volume template pointing to different directories.
Is it possible to run this configuration on a single node cluster setup?
Will these 3 alpha pods create a alpha group and as well as 3 zeros create a zero group and where to mention the replica- settings?
Is container command section required for both zero and alpha spec in yaml file:
command:
- bash
- “-c”
- |
set -ex
[[ hostname =~ -([0-9]+)$ ]] || exit 1
ordinal=${BASH_REMATCH[1]}
idx=$(($ordinal + 1))
if [[ $ordinal -eq 0 ]]; then
exec dgraph zero --my=$(hostname -f):5080 --raft=“idx=$idx” --replicas 3
else
exec dgraph zero --my=$(hostname -f):5080 --peer dgraph-zero-0.dgraph-zero.${POD_NAMESPACE}.svc.cluster.local:5080 --raft=“idx=$idx” --replicas 3
fi
What you mean by node? K8s node?
Well, yes? I’m confused with the question.
If your cluster is replica set to 3(which means it is HA). You gonna have a single Alpha group. Cuz you have only 3 Alpha nodes. Do you mean replica settings from Dgraph RAFT group or K8s replica settings? Please, make it clear. Or remove K8s from your example. As it is another scope.
Yes. This command will infer the SVC address created by K8s’s DNS service.
Your main question was in the title:
Running multiple Dgraph alpha pod and zeros in single host
If that is your question. Yes. You can run the whole cluster in a single machine whatever it be or where it be. What is your concern? In my opinion you shouldn’t do this. But you can. I’d prefer to run each Alpha or Alpha group in different hosts.
I am talking about - k8s replica settings - replicas:3 [in bold]
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: dgraph-zero
spec:
serviceName: “dgraph-zero” replicas: 3
Yes I have verified it will create a single Alpha group. But my question is I have not mentioned the replica settings for Dgraph RAFT group while creating zero statefulset.
I have omitted the container whole command section while deploying zero statefulset in dgraph-ha.yaml file.
command:
- bash
- “-c”
- |
set -ex
[[ hostname =~ -([0-9]+)$ ]] || exit 1
ordinal=${BASH_REMATCH[1]}
idx=$(($ordinal + 1))
if [[ $ordinal -eq 0 ]]; then
exec dgraph zero --my=$(hostname -f):5080 --raft=“idx=$idx” --replicas 3
else
exec dgraph zero --my=$(hostname -f):5080 --peer dgraph-zero-0.dgraph-zero.${POD_NAMESPACE}.svc.cluster.local:5080 --raft=“idx=$idx” --replicas 3
fi
And while deploying alpha statefulset also, I have omitted the container command section:
command:
- bash
- “-c”
- |
set -ex
dgraph alpha --my=$(hostname -f):7080 --zero dgraph-zero-0.dgraph-zero.${POD_NAMESPACE}.svc.cluster.local:5080,dgraph-zero-1.dgraph-zero.${POD_NAMESPACE}.svc.cluster.local:5080,dgraph-zero-2.dgraph-zero.${POD_NAMESPACE}.svc.cluster.local:5080
Then how all these three alphas create a single alpha group?
Hi @MichelDiz, I understood the best practice says to run each alpha or alpha group to run in dedicated hosts but this is for development purpose. Thanks for your reply.
But my confusion is, If, we don’t mention the replica settings for Dgraph RAFT group while deploying Dgraph zero, ideally every alpha node should creates it’s own alpha group right?
How it creates a HA, because if we don’t mention replica settings of Dgraph RAFT group while deploying zero, then ideally every alpha creates it’s own group.
Surprisingly, I have noticed when we deploy dgraph-ha.yaml with k8s replica as 3 for both zero and alpha but omitting the container commands from zero and alpha statefulset which creates Dgraph RAFT group, still it forms a single dgraph alpha group with all those k8s replicas. How is that possible? Is it something I am missing or not understanding
This is K8s replica. But in the context with Dgraph, means about the same thing. K8s will spin up 3 Alphas and 3 Zeros pods. In the RAFT they will become (the alphas) Dgraph replicas.
They will talk to the Zero Group leader and ask to join the cluster. The zero will ask them to create an election. They will elect a leader. Based on the Zero configs they will organize themselves as replicas. Done.
It is fine to run like this in a dev environment. Avoid doing this in prod.
Yes. If you don’t set the replication factor in the replica flag(for the zeros). Each Alpha will be a group. So you will endup with 3 groups. That’s the default behavior. Sharding all along.
In your copy paste commands you shown --replicas 3, so I assumed you are doing replication.
I mentioned above. The statefulset replication just replicates the pods. The “magic” relies in the replica flag. Dgraph does all the election based on the RAFT Algorithm.
HI @MichelDiz , Thanks for your reply and making me understand the inner logic but we are neither executing any of these commands for zero and alpha deployment. Means, using the dgraph-ha.yaml file we are deploying 3 k8s replica for both zero and alpha but we are not executing any container commands where we mention RAFT group replica settings. Still we saw one alpha group created rather than 3 alpha groups.
If you wanna remove replication. Remove this flag from “–replicas 3” zero.
But do better, don’t use the HA template. Use the other one. HA is basically replication.
Yes you are. The flags in the yml configs.
I’m quite confused by your purpose here. One moment I think you want to understand how replication works, and now it seems like you don’t want replication. The other time it seems your concern is running all instances of Dgraph on the same host. lol
Hi @MichelDiz, I know it might sound funny but looks like I am not able to make you understand what I am trying to do. I am not sure whether it’s a Dgraph documentation issue or not.
I am running dgraph in minikube which is single k8s node kubernetes cluster with k8s replicaset : 3 for both zero and alpha statefulset mention below:
Till here everything is fine and hope we both are on the same page.
And as per your previous comments these 3 alpha k8s replica settings will create a single Alpha raft group and 3 zero k8s replica settings will create a single Zero raft group.
I am NOT EXECUTING any zero configs as mentioned below
No configs we are passing as configs are mentioned in the command section for both zero and alpha.
If alphas are not getting raft configs like --replica-settings from zero ideally they should create three single group but without running those zero configs also I have noticed they form a single group.
I hope it’s understandable now if not can we have a schedule call because it’s very urgent and we are stuck.
So you don’t have HA. You should let the yaml as it is. No modifications. The command sections are basically the Cluster configs.
They are using the default config. If you don’t explicitly change the configs via those flags it will run the default values. So you will have 3 groups.
Reasons for this can be two. Or you are confusing something at some point. Or you ran that same yaml once and modified it later as overwrite. But modifying the YAML does not modify the configuration that was passed in the first moment.
Solution. Export your data. Delete everything, and start from scratch with your desired config. Be it th default or HA(by not touching the yaml).
BTW, If you want to run a cluster without HA you should take the example without HA. Don’t try to modify a YAML. You can end up breaking everything unless you really know what you’re doing.
