We run the community edition of dgraph… 3 of our 6 servers it runs on were unexpectedly powered off for several hours. Upon restart of the servers we’re seeing alphas and zeros not start properly.
From an alpha we see this:
groupd.go:1181 Error during SubscribeForUpdates for prefix "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x15dgraph.graphql.schema\x00": Unable to find any servers for grou: 1. closer err: <nil> admin.go:857 namespace:0. Error reading GraphQL schema: Please retry again, server is not ready to accept requests. ... CONNECTING to 10.x.x.2:5081 (zero) CONNECTING to 10.x.x.3:5082 (zero) Connected to group zero. Assigned group: 2 Raft Id after connection to Zero 0x4 CONNECTING to 10.x.x.3:5082 (zero) CONNECTING to 10.x.x.1:7080 (alpha) ... draft.go:1208 Found Raft progress: 102204895 groups.go:834 Starting a new membership stream receive from 10.x.x.3:5082 Received frist state update from Zero: counter:93288678 groups:<key:1 value<..... Sending fatal event [hash] to ... proto: Proposal: illegal tag 0 (wire type 0) github.com/dgraph-io/dgraph/x.Check /ext-go/1/src/github.com/dgraph-io/dgraph/x/error.go:42 github.com/dgraph-io/dgraph/worker.(*node).processApplyCh.func1 /ext-go/1/src/github.com/dgraph-io/dgraph/worker/draft.go:754 github.com/dgraph-io/dgraph/worker.(*node).processApplyCh /ext-go/1/src/github.com/dgraph-io/dgraph/worker/draft.go:819 runtime.goexit /user/local/go/src/runtime/asm_amd64.s:1371
Does anyone recognize what the issue may be?