Data corrupted, promote an Alpha

Issues Dgraph
1

We have a situation where one of the nodes has different data (only two nodes online), how do we force promote to the working Alpha so we can bring another node and it will sync data from the correct Alpha.

2

Hi,

Is the Alpha node with the inconsistency the current RAFT “leader” in the cluster? You can check the same using the Zero’s /state endpoint. From any Zero, run this:

curl -s localhost:6080/state | jq '{alphas: .groups."1".members, removed: .removed, zeros: .zeros}'

The Alpha leader would have the leader attribute set to true

"alphas": {
        "1": {
          "id": "1",
          "groupId": 1,
          "addr": "alpha-0.local:7080",
          "leader": true,
          "amDead": false,
          "lastUpdate": "xxxxxxxx",
          "learner": false,
          "clusterInfoOnly": false,
          "forceGroupId": false
        }

If yes, you can shutdown that node, following which one of the 2 working Alphas would get elected as the new 'leader". With quorum, the cluster would still be operational while you add a new node to the cluster. The steps to add a new node vary based on the mode of deployment i.e. bare-metal, VM, Kubernetes or VPC.

3

Thank you for your reply, this is good to know for the future. We ended up having to rebuild the cluster from a backup.