Issues initializing kubernetes cluster

Users
1

I’m trying to restore a data following the bulk loader instructions on the kubernetes config dgraph-ha

But nothing i’ve done works. It manages to recreate the schema but there aren’t any nodes to be found. When i created the rdf export my database had about 80k nodes.

The steps i took to restore the database:

Generate bulk loader output:

a. Copy the *.gz files to the zero leader pod:

kubectl cp g01.schema.gz dgraph-zero-2:/dgraph 
kubectl cp g01.rdf.gz dgraph-zero-2:/dgraph

b. Ssh into the zero leader pod:
kubectl exec -it dgraph-zero-2 -- /bin/bash
c. I run the command below

 dgraph bulk -f g01.rdf.gz -s g01.schema.gz --map_shards=2 --reduce_shards=1 --http localhost:8000 --zero=localhost:5080

After running the above command a folder “out” would be created

I then copy the out directory to my local computer

To restore the database i create a fresh new kubernetes cluster with initcontainers sections uncommented

kubectl apply -f config.yaml

On running
kubectl get pods
i find all the alpha pods are in status Init:0/1

then i copy the ./0/p directory from the bulk loader output
to each of the pods:

kubectl cp ./0/p dgraph-alpha-0:/dgraph/ -c init-alpha
kubectl cp ./0/p dgraph-alpha-1:/dgraph/ -c init-alpha
kubectl cp ./0/p dgraph-alpha-2:/dgraph/ -c init-alpha

and i finalize by creating a doneinit file in each of the pods:

kubectl exec dgraph-alpha-0 -c init-alpha touch /dgraph/doneinit
kubectl exec dgraph-alpha-1 -c init-alpha touch /dgraph/doneinit
kubectl exec dgraph-alpha-2 -c init-alpha touch /dgraph/doneinit
2

It’s been more than a day without any replies, can I get some help ?

3

The restored cluster should be with the same Zeros you used during the bulk loader. So, given the export data and schema from a previous cluster (g01.rdf.gz and g01.schema.gz), you’ll do the following:

  1. Run the Dgraph Zeros of the new cluster (no Alphas yet)
  2. Run bulk loader with the --zero option set to the Zero leader. This allocates the UID lease based on the dataset.
  3. Copy the p directory from public loader to each Alpha replica.
  4. Start the Alphas.
1 Like
4

Okay will try this, but your docs are super confusing, it doesn’t say tell me what you just said specifically.

5

How can i know which of the zero pods is the leader ?

6

You can check for the current leadership info per group by checking /state. The Dgraph Zero group info is under the zeros section.

You could also start off with a single Zero when running bulk loader. A single Zero means that is guaranteed to be the leader of the group (a group of itself only). Then, you can start the other Zeros to the cluster after finishing the bulk load.

1 Like
7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.