Alpha not responding to termination and gets stuck

Issues Dgraph
, ,
1

Report a Dgraph Bug

Dgraph alpha is not terminating when using kubectl delete. The operator has to exec into the container and kill 1 several times, otherwise, the alpha enters into infinite loops with the log output shown below.

What version of Dgraph are you using?

v2.0.0-rc1-788-gbae8e645b

Have you tried reproducing the issue with the latest release?

this is a branch off master.

What is the hardware spec (RAM, OS)?

Steps to reproduce the issue (command/config used to run Dgraph).

kubectl delete <release-name>-dgraph-alpha-0

Expected behavior and actual result.

The expected behavior is that the alpha would terminate.

The actual behavior is this (small % of logs, over 90K of these lines):

E1001 19:15:50.125941       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.125947       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.125952       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.125960       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.125968       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.125981       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.125989       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.125998       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126009       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126020       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126037       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126046       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126060       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126074       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126083       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126094       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126105       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126114       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126124       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126132       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126158       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126168       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126177       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126185       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126194       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126204       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126216       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126236       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126245       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126287       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126297       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126306       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126321       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126330       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126350       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126359       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126370       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126379       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126400       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126409       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126419       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126436       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126443       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126449       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126457       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126462       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126467       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126476       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126482       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126489       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126506       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126515       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126547       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126552       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126558       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126563       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126571       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
E1001 19:15:50.126589       1 groups.go:1027] While proposing delta with MaxAssigned: 75669192 and num txns: 4. Error=context canceled. Retrying...
3

This should be fixed by