Load Data using Dgraph w/ Kubernetes

Users
, ,
9

I did these steps, however the zero pod seems to have a infinite tcp error:

  1. List my dgraph cluster pod in eks.
kubectl get pod -n myrelease

NAME                                            READY   STATUS    RESTARTS   AGE
dgraph-release-dgraph-alpha-0                   1/1     Running   0          89m
dgraph-release-dgraph-zero-0                    1/1     Running   0          54m

Cluster is healthy by checking path /health of Alpha pod listening on port 8080.

[{"instance":"alpha","address":"dgraph-release-dgraph-alpha-0.dgraph-release-dgraph-alpha-headless.neosight.svc.cluster.local:7080","status":"healthy","group":"1","version":"v23.1.0","uptime":5948,"lastEcho":1711246631,"ongoing":["opRollup"],"ee_features":["backup_restore","cdc"],"max_assigned":21840}]

Using curl command to apply schema works.

  1. Forward Alph and Zero port
kubectl port-forward dgraph-release-dgraph-alpha-0 9080:9080 -n myrelease
Forwarding from 127.0.0.1:9080 -> 9080
Forwarding from [::1]:9080 -> 9080

kubectl port-forward dgraph-release-dgraph-zero-0 5080:5080 -n myrelease
Forwarding from 127.0.0.1:5080 -> 5080
Forwarding from [::1]:5080 -> 5080
  1. Run dgraph standalone in docker container on my laptop
docker run --name dgraph-utility -d -p "8080:8080" dgraph/standalone:latest
  1. Copy schema and data files into container
docker cp /Users/Ann.Zhang/Downloads/g01.schema dgraph-utility:/g01.schema
docker cp /Users/Ann.Zhang/Downloads/g01.rdf.gz dgraph-utility:/g01.rdf.gz
  1. Shell into the container
docker exec -i -t dgraph-utility /bin/bash
  1. Start live loading data
dgraph live \
--alpha localhost:9080 \
--zero localhost:5080 \
--schema /g01.schema \
--files /g01.rdf.gz

Message print out in container:

Dgraph version   : v23.1.0
Dgraph codename  : dgraph
Dgraph SHA-256   : 2b0d2fb977807f9d681c3a8e5f67a6fb133c99c772009158107aa6b1ac4cbd10
Commit SHA-1     : 2b18d19
Commit timestamp : 2023-08-17 13:27:10 -0500
Branch           : HEAD
Go version       : go1.19.12
jemalloc enabled : true

For Dgraph official documentation, visit https://dgraph.io/docs.
For discussions about Dgraph     , visit https://discuss.dgraph.io.
For fully-managed Dgraph Cloud   , visit https://dgraph.io/cloud.

Licensed variously under the Apache Public License 2.0 and Dgraph Community License.
Copyright 2015-2023 Dgraph Labs, Inc.



Running transaction with dgraph endpoint: localhost:9080

Processing schema file "/g01.schema"
Processed schema file "/g01.schema"

Found 1 data file(s) to process
Processing data file "/g01.rdf.gz"
[02:21:30Z] Elapsed: 05s Txns: 71 N-Quads: 71000 N-Quads/s [last 5s]: 14200 Aborts: 0
[02:21:35Z] Elapsed: 10s Txns: 140 N-Quads: 140000 N-Quads/s [last 5s]: 13800 Aborts: 0
[02:21:40Z] Elapsed: 15s Txns: 186 N-Quads: 186000 N-Quads/s [last 5s]:  9200 Aborts: 0

Error logs in dgraph-release-dgraph-zero-0. 10.178.11.193 is the IP Address of dgraph-release-dgraph-zero-0.

kubectl logs -f dgraph-release-dgraph-zero-0 -n myrelease

E0324 02:21:37.198103       1 x.go:352] Could not send error msg=write tcp 10.178.11.193:6080->10.178.11.175:58134: write: broken pipe code=ErrorNoData due to http error write tcp 10.178.11.193:6080->10.178.11.175:58134: write: broken pipe
E0324 02:21:47.197544       1 x.go:352] Could not send error msg=write tcp 10.178.11.193:6080->10.178.11.175:45494: write: broken pipe code=ErrorNoData due to http error write tcp 10.178.11.193:6080->10.178.11.175:45494: write: broken pipe
E0324 02:21:57.197362       1 x.go:352] Could not send error msg=write tcp 10.178.11.193:6080->10.178.11.175:53224: write: broken pipe code=ErrorNoData due to http error write tcp 10.178.11.193:6080->10.178.11.175:53224: write: broken pipe
E0324 02:22:07.197324       1 x.go:352] Could not send error msg=write tcp 10.178.11.193:6080->10.178.11.175:52942: write: broken pipe code=ErrorNoData due to http error write tcp 10.178.11.193:6080->10.178.11.175:52942: write: broken pipe
E0324 02:22:17.197429       1 x.go:352] Could not send error msg=write tcp 10.178.11.193:6080->10.178.11.175:56626: write: broken pipe code=ErrorNoData due to http error write tcp 10.178.11.193:6080->10.178.11.175:56626: write: broken pipe
E0324 02:22:27.197550       1 x.go:352] Could not send error msg=write tcp 10.178.11.193:6080->10.178.11.175:41134: write: broken pipe code=ErrorNoData due to http error write tcp 10.178.11.193:6080->10.178.11.175:41134: write: broken pipe
E0324 02:22:37.197504       1 x.go:352] Could not send error msg=write tcp 10.178.11.193:6080->10.178.11.175:37462: write: broken pipe code=ErrorNoData due to http error write tcp 10.178.11.193:6080->10.178.11.175:37462: write: broken pipe