K8S version: 1.20.7
Alpha nodes: 64 GB, NVMe SSD 1.9 T (Standard_L8s_v2 SKU)
Cluster setup: one zero, 3 alpha
Alpha replica count: 3
Live Upload command snippet
dgraph live --files /root/dgraph/data/1642449041 --schema /root/dgraph/schema/my_schema.rdf --alpha host.docker.internal:9080 --zero host.docker.internal:5080 --format=rdf --upsertPredicate "xid" -b 1000
Simple (165MB of RDFs) test of Live Uploader stats
Number of TXs run : 55757
Number of N-Quads processed : 55670191
Time spent : 2h9m22.150593615s
N-Quads processed per second : 7172
Should this be considered slow?
Yea seems slow for that hardware.
host.docker.internal what is this? Doesn’t seem like how you might do this from kubernetes. Are you running all 4 of these pods all on the same node or something?
host.docker.internal this is required because, I’m using
kubectl port-forward from local machine.
Also, I tried to use Live uploader from zero. It errored out with below message
Error while processing schema file "my_scheme.rdf": rpc error: code = Unknown desc = unauthorized ip address: 10.155.2.4
rpc error: code = Unknown desc = unauthorized ip address: 10.155.2.4#
Per this [post]( How to live load data from a different server on alpha node hosted on EKS - Users / Dgraph - Discuss Dgraph), do whitelist when using live uploader from zero?
Yea I assume your slowness is because of the port-forward and WAN (most likely)
The whitelist is necessary to perform admin tasks on the alphas, so you can either add the whitelist for your pod/service subnet or run this from an alpha.