1.0.7 unable to start after forced shutdown


(James) #1

hi,

i think it occured 3 times to me,
when i ran it in docker-compose and shutdown my machine, sometimes ive to force it to shutdown, but it will causes it to be unable to start back up. this is the logs in docker-compose up

dnode1_1    | 2018/11/22 02:26:19 raft.go:749: INFO: 1 is starting a new election at term 22
dnode1_1    | 2018/11/22 02:26:19 raft.go:594: INFO: 1 became pre-candidate at term 22
dnode1_1    | 2018/11/22 02:26:19 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 22
dnode1_1    | 2018/11/22 02:26:19 raft.go:580: INFO: 1 became candidate at term 23
dnode1_1    | 2018/11/22 02:26:19 raft.go:664: INFO: 1 received MsgVoteResp from 1 at term 23
dnode1_1    | 2018/11/22 02:26:19 raft.go:621: INFO: 1 became leader at term 23
dnode1_1    | 2018/11/22 02:26:19 node.go:301: INFO: raft.node: 1 elected leader 1 at term 23
dzero_1     | 2018/11/22 02:26:19 raft.go:1070: INFO: 1 no leader at term 22; dropping index reading msg
dzero_1     | 2018/11/22 02:26:20 raft.go:749: INFO: 1 is starting a new election at term 22
dzero_1     | 2018/11/22 02:26:20 raft.go:594: INFO: 1 became pre-candidate at term 22
dzero_1     | 2018/11/22 02:26:20 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 22
dzero_1     | 2018/11/22 02:26:20 raft.go:580: INFO: 1 became candidate at term 23
dzero_1     | 2018/11/22 02:26:20 raft.go:664: INFO: 1 received MsgVoteResp from 1 at term 23
dzero_1     | 2018/11/22 02:26:20 raft.go:621: INFO: 1 became leader at term 23
dzero_1     | 2018/11/22 02:26:20 node.go:301: INFO: raft.node: 1 elected leader 1 at term 23
dzero_1     | 2018/11/22 02:26:45 oracle.go:92: Purged below ts:140414, len(o.commits):0, len(o.rowCommit):0
dzero_1     | 2018/11/22 02:34:14 tablet.go:179:
dzero_1     |
dzero_1     | Groups sorted by size: [{gid:1 size:27020} {gid:2 size:27894}]
dzero_1     |
dzero_1     | 2018/11/22 02:34:14 tablet.go:184: size_diff 874
dnode1_1    | 2018/11/22 02:36:16 index.go:927: Dropping predicate: [_type]
dnode1_1    | 2018/11/22 02:36:16 schema.go:63: Deleting schema for predicate: [_type]
dzero_1     | 2018/11/22 02:42:14 tablet.go:179:
dzero_1     |
dzero_1     | Groups sorted by size: [{gid:1 size:27020} {gid:2 size:27894}]
dzero_1     |
dzero_1     | 2018/11/22 02:42:14 tablet.go:184: size_diff 874
dzero_1     | 2018/11/22 02:50:13 tablet.go:179:
  dnode1:
dzero_1     |
dzero_1     | Groups sorted by size: [{gid:1 size:27020} {gid:2 size:27894}]
dzero_1     |
dzero_1     | 2018/11/22 02:50:13 tablet.go:184: size_diff 874
dzero_1     | 2018/11/22 02:58:13 tablet.go:179:
dzero_1     |
dzero_1     | Groups sorted by size: [{gid:1 size:27020} {gid:2 size:27894}]
dzero_1     |
dzero_1     | 2018/11/22 02:58:13 tablet.go:184: size_diff 874
dzero_1     | 2018/11/22 03:06:12 tablet.go:179:
dzero_1     |
dzero_1     | Groups sorted by size: [{gid:1 size:27020} {gid:2 size:27894}]
dzero_1     |
dzero_1     | 2018/11/22 03:06:12 tablet.go:184: size_diff 874
dzero_1     | 2018/11/22 03:14:12 tablet.go:179:
dzero_1     |
dzero_1     | Groups sorted by size: [{gid:1 size:27020} {gid:2 size:27894}]
dzero_1     |
dzero_1     | 2018/11/22 03:14:12 tablet.go:184: size_diff 874
dzero_1     | 2018/11/22 03:22:11 tablet.go:179:
dzero_1     |
dzero_1     | Groups sorted by size: [{gid:1 size:27020} {gid:2 size:27894}]
dzero_1     |
dzero_1     | 2018/11/22 03:22:11 tablet.go:184: size_diff 874
dzero_1     | 2018/11/22 03:30:10 tablet.go:179:
dzero_1     |
dzero_1     | Groups sorted by size: [{gid:1 size:27020} {gid:2 size:27894}]
dzero_1     |
dzero_1     | 2018/11/22 03:30:10 tablet.go:184: size_diff 874
dzero_1     | 2018/11/22 03:38:10 tablet.go:179:
dzero_1     |
dzero_1     | Groups sorted by size: [{gid:1 size:27020} {gid:2 size:27894}]
dzero_1     |
dzero_1     | 2018/11/22 03:38:10 tablet.go:184: size_diff 874
^CGracefully stopping... (press Ctrl+C again to force)

(Daniel Mai) #2

Try upgrading to the latest release.

In v1.0.8 this Raft election deadlock after start up has been fixed https://github.com/dgraph-io/dgraph/pull/2548. https://github.com/dgraph-io/dgraph/releases/tag/v1.0.8