Error while rebuilding predicates

Users Dgraph
1

This error is occurring when I rebuild my schema:

I1212 10:03:04.158316    3235 index.go:48] Deleting reverse index for meta_extends
I1212 10:03:04.158716    3235 index.go:54] Rebuilding reverse index for meta_extends
I1212 10:03:04.159134    3235 index.go:61] Deleting count index for meta_extends
I1212 10:03:04.159646    3235 index.go:66] Rebuilding count index for meta_extends
I1212 10:03:04.160228    3235 mutation.go:148] Done schema update predicate:"meta_extends" value_type:UID directive:REVERSE count:true 
I1212 10:03:04.166898    3235 index.go:48] Deleting reverse index for meta_libs
I1212 10:03:04.167124    3235 index.go:54] Rebuilding reverse index for meta_libs
2018/12/12 10:03:04 Unexpected list of size zero: "\x00\x00\tmeta_libs\f\x00\x00\x00\x00"
github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/y.AssertTruef
        /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/y/error.go:62
github.com/dgraph-io/dgraph/posting.(*rebuild).Run
        /ext-go/1/src/github.com/dgraph-io/dgraph/posting/index.go:587
github.com/dgraph-io/dgraph/posting.RebuildReverseEdges
        /ext-go/1/src/github.com/dgraph-io/dgraph/posting/index.go:710
github.com/dgraph-io/dgraph/worker.(*node).rebuildOrDelRevEdge
        /ext-go/1/src/github.com/dgraph-io/dgraph/worker/index.go:55
github.com/dgraph-io/dgraph/worker.runSchemaMutationHelper
        /ext-go/1/src/github.com/dgraph-io/dgraph/worker/mutation.go:169
github.com/dgraph-io/dgraph/worker.runSchemaMutation
        /ext-go/1/src/github.com/dgraph-io/dgraph/worker/mutation.go:99
github.com/dgraph-io/dgraph/worker.(*node).applyMutations
        /ext-go/1/src/github.com/dgraph-io/dgraph/worker/draft.go:198
github.com/dgraph-io/dgraph/worker.(*node).applyCommitted
        /ext-go/1/src/github.com/dgraph-io/dgraph/worker/draft.go:280
github.com/dgraph-io/dgraph/worker.(*node).processApplyCh.func1
        /ext-go/1/src/github.com/dgraph-io/dgraph/worker/draft.go:401
github.com/dgraph-io/dgraph/worker.(*node).processApplyCh
        /ext-go/1/src/github.com/dgraph-io/dgraph/worker/draft.go:429
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1333
E1212 10:03:05.592430    3234 oracle.go:431] Error while fetching minTs from group 1, err: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: Error while dialing dial tcp [::1]:7080: connect: connection refused"
E1212 10:03:05.624986    3234 pool.go:178] Echo error from localhost:7080. Err: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: Error while dialing dial tcp [::1]:7080: connect: connection refused"
E1212 10:03:35.593352    3234 oracle.go:425] No healthy connection found to leader of group 1
E1212 10:04:05.592296    3234 oracle.go:425] No healthy connection found to leader of group 1

Please help, kind of a hurry.

2

Restarted clean, watching the logs:

I1212 10:14:04.933215    3568 node.go:84] 1 became leader at term 4
I1212 10:14:04.933356    3568 node.go:84] raft.node: 1 elected leader 1 at term 4
I1212 10:14:05.128352    3568 groups.go:718] Leader idx=1 of group=1 is connecting to Zero for txn updates
E1212 10:14:05.129303    3568 groups.go:758] Error in oracle delta stream. Error: rpc error: code = Unknown desc = Node is no longer leader.
W1212 10:14:05.173581    3567 node.go:551] [1] Read index context timed out
I1212 10:14:05.174044    3567 node.go:84] 1 no leader at term 3; dropping index reading msg
I1212 10:14:05.887477    3567 node.go:84] 1 is starting a new election at term 3
I1212 10:14:05.887550    3567 node.go:84] 1 became pre-candidate at term 3
I1212 10:14:05.887575    3567 node.go:84] 1 received MsgPreVoteResp from 1 at term 3
I1212 10:14:05.887680    3567 node.go:84] 1 became candidate at term 4
I1212 10:14:05.887698    3567 node.go:84] 1 received MsgVoteResp from 1 at term 4
I1212 10:14:05.887861    3567 node.go:84] 1 became leader at term 4
I1212 10:14:05.887892    3567 node.go:84] raft.node: 1 elected leader 1 at term 4
I1212 10:14:06.128367    3568 groups.go:718] Leader idx=1 of group=1 is connecting to Zero for txn updates
E1212 10:14:06.131251    3568 draft.go:467] Lastcommit 4153 > current 3377. This would cause some commits to be lost.
W1212 10:14:08.174235    3567 node.go:551] [1] Read index context timed out
I1212 10:14:32.096253    3567 oracle.go:100] Purged below ts:4154, len(o.commits):0, len(o.rowCommit):0
I1212 10:14:32.146843    3568 draft.go:317] Creating snapshot at index: 4237. ReadTs: 4153.
E1212 10:14:40.343019    3568 draft.go:467] Lastcommit 4153 > current 3377. This would cause some commits to be lost.

“This would cause some commits to be lost.” ???

Dgraph version : v1.0.10
Commit SHA-1 : 8b801bd7
Commit timestamp : 2018-11-05 17:52:33 -0800
Branch : HEAD

3

This same issue was reported a month ago and fixed in v1.0.11-rc3. Please try again with the latest rc (v1.0.11-rc4).

4

Ok, thank you. Any ETA for 1.0.11?

Thanks

5

v1.0.11 will be out soon. :wink:

1 Like