# Bulk loader in HA Kubernetes deployment

**URL:** <https://discuss.dgraph.io/t/bulk-loader-in-ha-kubernetes-deployment/12849>\
**Category:** Dgraph\
**Tags:** kind:question\
**Created:** [February 19, 2021, 7:20pm UTC](https://discuss.dgraph.io/t/bulk-loader-in-ha-kubernetes-deployment/12849 "2021-02-19T19:20:15Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Matthias\_Baetens](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/matthias_baetens/32/6041_2.png) [@Matthias\_Baetens](https://discuss.dgraph.io/u/Matthias_Baetens)\
**Post date:** [February 19, 2021, 7:20pm UTC](https://discuss.dgraph.io/t/bulk-loader-in-ha-kubernetes-deployment/12849/1 "2021-02-19T19:20:15Z")

</div>

## I Want to Do

I want to load a large amount of data into Dgraph using the Bulk Loader.

## What I Did

I have a 3 nodes Kubernetes cluster running the HA K8s deployment with the init containers on the `alphas` (and the pods are still pending as such). I have launched a Pod inside the cluster with a PV attached that contains the data and tried to launch the following command:

```auto
dgraph bulk -f filename.rdf.gz -s schema/dgraph.schema --reduce_shards=1 --zero=dgraph-zero.default.svc.cluster.local:5080

```

I get back the following errors: `Error communicating with dgraph zero, retrying: rpc error: code = Unknown desc = Assigning IDs is only allowed on leader.`

I tried scaling back the `zero` statefulset to 1 node, to no avail. The logs on that side show:  
`Got error: Assigning IDs is only allowed on leader. while leasing timestamps: val:1`

## Dgraph Metadata

```auto
Dgraph version : v20.11.1
Dgraph codename : tchalla-1
Dgraph SHA-256 : cefdcc880c0607a92a1d8d3ba0beb015459ebe216e79fdad613eb0d00d09f134
Commit SHA-1 : 7153d13fe
Commit timestamp : 2021-01-28 15:59:35 +0530
Branch : HEAD
Go version : go1.15.5
jemalloc enabled : true

```

---

<div class="post-metadata">

**Author:** ![MichelDiz](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/micheldiz/32/11873_2.png) [@MichelDiz](https://discuss.dgraph.io/u/MichelDiz)\
**Post date:** [February 19, 2021, 8:18pm UTC](https://discuss.dgraph.io/t/bulk-loader-in-ha-kubernetes-deployment/12849/2 "2021-02-19T20:18:39Z")

</div>

Can you share the Zero logs (from all if you have multiple)

---

<div class="post-metadata">

**Author:** ![Matthias\_Baetens](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/matthias_baetens/32/6041_2.png) [@Matthias\_Baetens](https://discuss.dgraph.io/u/Matthias_Baetens)\
**Post date:** [February 20, 2021, 9:11am UTC](https://discuss.dgraph.io/t/bulk-loader-in-ha-kubernetes-deployment/12849/3 "2021-02-20T09:11:32Z")

</div>

Sure. I reduced to just 1 Zero (reasoning that potentially, through the service in front of the Pods, it was hitting a different one than the leader and having just one would solve that problem).

Some of the logs:

```auto
2021-02-20 09:08:21.393 GMT "Unable to send message to peer: 0x3. Error: Unhealthy connection"
2021-02-20 09:08:22.392 GMT "[0x1] Read index context timed out"
2021-02-20 09:08:24.393 GMT "[0x1] Read index context timed out"
2021-02-20 09:08:25.393 GMT "Unable to send message to peer: 0x2. Error: Unhealthy connection"
2021-02-20 09:10:04.608 GMT Got error: Assigning IDs is only allowed on leader. while leasing timestamps: val:1 "
2021-02-20 09:10:05.400 GMT "Unable to send message to peer: 0x2. Error: Unhealthy connection"

```

---

<div class="post-metadata">

**Author:** ![MichelDiz](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/micheldiz/32/11873_2.png) [@MichelDiz](https://discuss.dgraph.io/u/MichelDiz)\
**Post date:** [February 20, 2021, 1:40pm UTC](https://discuss.dgraph.io/t/bulk-loader-in-ha-kubernetes-deployment/12849/4 "2021-02-20T13:40:24Z")

</div>

Don’t reduce it in that way. You have to remove it from the quorum or start with 1 and then add more nodes.

Use [https://dgraph.io/docs/deploy/dgraph-zero/#endpoints](https://dgraph.io/docs/deploy/dgraph-zero/#endpoints) to remove zeros.

---

<div class="post-metadata">

**Author:** ![Matthias\_Baetens](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/matthias_baetens/32/6041_2.png) [@Matthias\_Baetens](https://discuss.dgraph.io/u/Matthias_Baetens)\
**Post date:** [February 21, 2021, 11:05am UTC](https://discuss.dgraph.io/t/bulk-loader-in-ha-kubernetes-deployment/12849/5 "2021-02-21T11:05:03Z")

</div>

Thanks Michel - that’s a very helpful page. I queried the `/state` endpoint and found out who the leader is that way. Pointing the bulk loader to that `Pod` did the trick. Loading now - it looks promising 🕶