Dgraph v1.0.7-rc3 release

Users Dgraph
1

Note: See post below for v1.0.7 release. The links are the same.

Hey folks,

Before cutting v1.0.7 release, I wanted to ensure that it’s working well, and catch any potential bugs upfront.
So, please find attached the binaries for v1.0.7-rc1 (release candidate 1). It would be great if you guys could test it out, and provide feedback. The v1.0.7 upcoming release contains a lot of bug fixes and design simplifications which improve the stability and robustness of Dgraph.

Link to binaries: Linux, Darwin, Windows.

Also, on Docker: Docker Hub
Tag: v1.0.7-rc1

Note: This is not the official release. You won’t be able to get these binaries using curl get.dgraph.io script.

Cheers,
Manish

2 Likes
3

i will test this release about flag replicas > 1, and see if it has the same problem in 1.0.6

1 Like
4

Hey @shanghai-Jerry,

Any updates on your tests?

5

Hey folks,

I’ve uploaded another release candidate, v1.0.7-rc2. The links are the same. Docker tag would be v1.0.7-rc2. So far no issues have been reported with this release, which means this would be converted to an official release.

Note that the --badger flag options are slightly changed to better reflect their purpose. If you’re on rotational hdd, you should use --badger.options=hdd (default is ssd).

Link to binaries: Linux, Darwin, Windows.

6

Not yet, today i will test my data.

i have a question:
–badger.vlog string [none, mmap, disk] Specifies how Badger Value log is stored. mmap consumes more RAM, but provides better performance in some cases. (default “none”)

what’s the default of this flag about “none”?

7

None would pick up the default setting based on Badger.options, I. E. SSD or HDD.

8

ok, i am importing data now;

here are three servers’ log:

server one:

02
022090×930 180 KB

server two:

23
231818×1294 552 KB

server three

51
512066×1300 569 KB

it seems everything is ok, what do you think?

i’ll keep focus on this.

later got a WARN

13
131182×716 120 KB

later again
— server one

42
421446×910 158 KB

---- server two
51
511450×1290 189 KB

— server three
58
581456×958 153 KB

9

I wonder if this is because your hard disk is slow. Can you try with the hdd option? Also, can you upload the entire logs from servers, instead of screenshots.

10

I’ve uploaded binaries and docker image for v1.0.7. Contains improvements upon some of the issues we saw in the release candidates.

Please try them out. I’ll make it an official release next week.

Link to binaries: Linux, Darwin, Windows. Pushed to Docker as well as v1.0.7.

13

i tried with badger.option=hdd，and the service is started with systemctl, i don’t know how to get the whole log.

14

Try again with the v1.0.7 binary. There’re some fixes here which should help.

15

ok，when i use bulk loader, it happened this error, will this miss any data?

52
522026×390 76 KB

16

i use bulk loader with --reduce_shards=3, and then i put these three shards into three different server,

then i restart dgraph server with a specific p dir, then it happened:

i used the latest release 1.0.7

server 1;

17
172762×1678 927 KB

server 2;

32
321760×1470 648 KB

server 3;

44
441942×1602 765 KB

server state;

{
    "counter": "5328",
    "groups": {
        "1": {
            "members": {
                "1": {
                    "id": "1",
                    "groupId": 1,
                    "addr": "172.20.0.8:7080",
                    "lastUpdate": "1532070607"
                }
            },
            "tablets": {
                "_predicate_": {
                    "groupId": 1,
                    "predicate": "_predicate_",
                    "space": "31"
                },
                "_share_hash_": {
                    "groupId": 1,
                    "predicate": "_share_hash_",
                    "space": "40"
                },
                "age_node": {
                    "groupId": 1,
                    "predicate": "age_node"
                },
                "annual_salary": {
                    "groupId": 1,
                    "predicate": "annual_salary",
                    "space": "654352948"
                },
                "birthday": {
                    "groupId": 1,
                    "predicate": "birthday",
                    "space": "2442347945"
                },
                "candidate_school": {
                    "groupId": 1,
                    "predicate": "candidate_school",
                    "space": "4077043703"
                },
                "company_dept": {
                    "groupId": 1,
                    "predicate": "company_dept",
                    "space": "130080582"
                },
                "industry": {
                    "groupId": 1,
                    "predicate": "industry"
                },
                "monthly_salary": {
                    "groupId": 1,
                    "predicate": "monthly_salary"
                },
                "seniorty_node": {
                    "groupId": 1,
                    "predicate": "seniorty_node",
                    "space": "1058046435"
                },
                "type": {
                    "groupId": 1,
                    "predicate": "type"
                }
            }
        },
        "2": {
            "members": {
                "3": {
                    "id": "3",
                    "groupId": 2,
                    "addr": "172.20.0.10:7080",
                    "leader": true,
                    "lastUpdate": "1532075489"
                }
            }
        },
        "3": {
            "members": {
                "2": {
                    "id": "2",
                    "groupId": 3,
                    "addr": "172.20.0.9:7080",
                    "leader": true,
                    "lastUpdate": "1532075613"
                }
            },
            "tablets": {
                "candidate_dept": {
                    "groupId": 3,
                    "predicate": "candidate_dept",
                    "space": "2272621839"
                },
                "code": {
                    "groupId": 3,
                    "predicate": "code",
                    "space": "757469"
                },
                "current_location_code": {
                    "groupId": 3,
                    "predicate": "current_location_code",
                    "space": "14722570"
                },
                "degree": {
                    "groupId": 3,
                    "predicate": "degree",
                    "space": "129061069"
                },
                "degree_node": {
                    "groupId": 3,
                    "predicate": "degree_node",
                    "space": "1016203678"
                },
                "gender": {
                    "groupId": 3,
                    "predicate": "gender",
                    "space": "1876206049"
                },
                "gender_node": {
                    "groupId": 3,
                    "predicate": "gender_node",
                    "space": "1016203678"
                },
                "name": {
                    "groupId": 3,
                    "predicate": "name",
                    "space": "5728457773"
                },
                "parent_industry": {
                    "groupId": 3,
                    "predicate": "parent_industry",
                    "space": "2907"
                },
                "started_work_at": {
                    "groupId": 3,
                    "predicate": "started_work_at",
                    "space": "2512663489"
                },
                "unique_id": {
                    "groupId": 3,
                    "predicate": "unique_id",
                    "space": "6783842509"
                }
            }
        }
    },
    "zeros": {
        "1": {
            "id": "1",
            "addr": "172.20.0.8:5080"
        },
        "2": {
            "id": "2",
            "addr": "172.20.0.9:5080",
            "leader": true
        },
        "3": {
            "id": "3",
            "addr": "172.20.0.10:5080"
        }
    },
    "maxLeaseId": "51580000",
    "maxTxnTs": "30000",
    "maxRaftId": "3"
}
17

Can you share the commands you’re using? And copy paste as much of the logs from the beginning as possible. It’s hard to determine anything without having this information.

Just by looking at the INFO logs, that doesn’t tell anything about what’s going on. I need to understand what you ran and how it reached where it reached. You should be able to gather the logs from systemctl, or just run it manually so you can then gather the logs.

FWIW, it seems like you have some network issues. Raft is constantly changing its leaders, unable to maintain the quorum. That only happens if your servers keep falling behind, and are unable to respond to heartbeats.

18

You can use pipes to save logs:

try e.g:

dgraph zero command |& tee -a Zero_output.txt
# or
dgraph zero command &>> Zero_output.txt

dgraph server command &>> Server_output.txt

take a look on these links:

Best for test e.g:

For Single Dgraph

dgraph zero --my=localhost:5080 -w zeroSingle |& tee -a Zero_output.txt
dgraph server --my=localhost:7080 --lru_mb=3512 --zero=localhost:5080 -p serverS/p -w serverS/w --badger.options=lsmonly --badger.options=hdd -o 0 |& tee -a Server_output.txt

Remove “–badger.options=hdd” if you’re using SSD

For Cluster Dgraph

dgraph zero --my=localhost:5080 --replicas 3 -w zeroCluster |& tee -a Zero_Cluster_output.txt

dgraph server --my=localhost:7080 --lru_mb=3512 --zero=localhost:5080 -o 0 -p server0/p -w server0/w --badger.options=lsmonly --badger.options=hdd |& tee -a server0/Server_output.txt

dgraph server --my=localhost:7081 --lru_mb=3512 --zero=localhost:5080 -o 1 -p server1/p -w server1/w --badger.options=lsmonly --badger.options=hdd |& tee -a server1/Server_output.txt

dgraph server --my=localhost:7082 --lru_mb=3512 --zero=localhost:5080 -o 2 -p server2/p -w server2/w --badger.options=lsmonly --badger.options=hdd |& tee -a server2/Server_output.txt
19

Here are configs and total logs:

  1. 08 server
    {
    “my”: “172.20.0.8:7080”,
    “zero”: “172.20.0.8:5080”,
    “lru_mb”: 21440,
    “badger.options”:“hdd”,
    “badger.vlog”: “disk”
    }
  2. 09 server
    {
    “my”: “172.20.0.9:7080”,
    “zero”: “172.20.0.8:5080”,
    “lru_mb”: 21440,
    “badger.options”:“hdd”,
    “badger.vlog”: “disk”
    }
  3. 10 server
    {
    “my”: “172.20.0.10:7080”,
    “zero”: “172.20.0.8:5080”,
    “lru_mb”: 21440,
    “badger.options”:“hdd”,
    “badger.vlog”: “disk”
    }

logs of these servers:
08 server:

-- Logs begin at Sat 2018-06-23 23:36:44 CST. --
Jul 20 16:31:24 host-00-08.wgq.higgs.com systemd[1]: Started A Distributed, Fast Graph Database.
Jul 20 16:31:24 host-00-08.wgq.higgs.com dgraph[46352]: Dgraph version   : v1.0.7
Jul 20 16:31:24 host-00-08.wgq.higgs.com dgraph[46352]: Commit SHA-1     : d9c3671f
Jul 20 16:31:24 host-00-08.wgq.higgs.com dgraph[46352]: Commit timestamp : 2018-07-19 13:28:43 -0700
Jul 20 16:31:24 host-00-08.wgq.higgs.com dgraph[46352]: Branch           : HEAD
Jul 20 16:31:24 host-00-08.wgq.higgs.com dgraph[46352]: For Dgraph official documentation, visit https://docs.dgraph.io.
Jul 20 16:31:24 host-00-08.wgq.higgs.com dgraph[46352]: For discussions about Dgraph     , visit http://discuss.dgraph.io.
Jul 20 16:31:24 host-00-08.wgq.higgs.com dgraph[46352]: To say hi to the community       , visit https://dgraph.slack.com.
Jul 20 16:31:24 host-00-08.wgq.higgs.com dgraph[46352]: Licensed under Apache 2.0 + Commons Clause. Copyright 2015-2018 Dgraph Labs, Inc.
Jul 20 16:31:24 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:24 server.go:116: Setting Badger option: hdd
Jul 20 16:31:24 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:24 server.go:131: Setting Badger table load option: mmap
Jul 20 16:31:24 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:24 server.go:144: Setting Badger value log load option: disk
Jul 20 16:31:24 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:24 server.go:155: Opening postings Badger DB with options: {Dir:p ValueDir:p SyncWrites:true TableLoadingMode:2 ValueLogLoadingMode:0 NumVersionsToKeep:2147483647 MaxTableSize:67108864 LevelSizeMultiplier:10 MaxLevels:7 ValueThreshold:65500 NumMemtables:5 NumLevelZeroTables:5 NumLevelZeroTablesStall:10 LevelOneSize:268435456 ValueLogFileSize:67108864 ValueLogMaxEntries:1000000 NumCompactors:3 managedTxns:false DoNotCompact:false maxBatchCount:0 maxBatchSize:0 ReadOnly:false Truncate:false}
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 groups.go:78: Current Raft Id: 1
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 gRPC server started.  Listening on port 9080
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 HTTP server started.  Listening on port 8080
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 worker.go:89: Worker listening at address: [::]:7080
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 pool.go:108: == CONNECTED ==> Setting 172.20.0.8:5080
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 groups.go:105: Connected to group zero. Assigned group: 0
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 pool.go:108: == CONNECTED ==> Setting 172.20.0.10:7080
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 pool.go:108: == CONNECTED ==> Setting 172.20.0.10:5080
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 pool.go:108: == CONNECTED ==> Setting 172.20.0.9:5080
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 draft.go:74: Node ID: 1 with GroupID: 1
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 node.go:229: Found hardstate: {Term:4 Vote:1 Commit:125 XXX_unrecognized:[]}
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 node.go:238: Group 1 found 126 entries
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 draft.go:941: Restarting node for group: 1
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 raft.go:567: INFO: 1 became follower at term 4
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 raft.go:315: INFO: newRaft 1 [peers: [], term: 4, commit: 125, applied: 0, lastindex: 125, lastterm: 4]
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 node.go:129: Setting conf state to nodes:1
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 node.go:129: Setting conf state to nodes:1 nodes:3
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 mutation.go:174: Done schema update predicate:"_predicate_" value_type:STRING list:true
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 pool.go:162: Echo error from 172.20.0.9:7080. Err: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: Error while dialing dial tcp 172.20.0.9:7080: connect: connection refused"
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 pool.go:108: == CONNECTED ==> Setting 172.20.0.9:7080
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 node.go:129: Setting conf state to nodes:1 nodes:2 nodes:3
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 mutation.go:174: Done schema update predicate:"_share_hash_" value_type:STRING directive:INDEX tokenizer:"exact"
Jul 20 16:31:32 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:32 groups.go:358: Serving tablet for: type
Jul 20 16:31:34 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:34 raft.go:749: INFO: 1 is starting a new election at term 4
Jul 20 16:31:34 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:34 raft.go:594: INFO: 1 became pre-candidate at term 4
Jul 20 16:31:34 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:34 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 4
Jul 20 16:31:34 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:34 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 3 at term 4
Jul 20 16:31:34 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:34 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 2 at term 4
Jul 20 16:31:34 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:34 node.go:328: Error while sending message to node with addr: 172.20.0.9:7080, err: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: Error while dialing dial tcp 172.20.0.9:7080: connect: connection refused"
Jul 20 16:31:34 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:34 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 3 [term: 2]
Jul 20 16:31:37 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:37 raft.go:749: INFO: 1 is starting a new election at term 4
Jul 20 16:31:37 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:37 raft.go:594: INFO: 1 became pre-candidate at term 4
Jul 20 16:31:37 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:37 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 4
Jul 20 16:31:37 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:37 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 3 at term 4
Jul 20 16:31:37 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:37 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 2 at term 4
Jul 20 16:31:37 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:37 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 3 [term: 2]
Jul 20 16:31:37 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:37 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 2 [term: 2]
Jul 20 16:31:39 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:39 raft.go:749: INFO: 1 is starting a new election at term 4
Jul 20 16:31:39 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:39 raft.go:594: INFO: 1 became pre-candidate at term 4
Jul 20 16:31:39 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:39 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 4
Jul 20 16:31:39 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:39 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 3 at term 4
Jul 20 16:31:39 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:39 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 2 at term 4
Jul 20 16:31:39 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:39 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 3 [term: 2]
Jul 20 16:31:39 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:39 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 2 [term: 2]
Jul 20 16:31:41 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:41 raft.go:749: INFO: 1 is starting a new election at term 4
Jul 20 16:31:41 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:41 raft.go:594: INFO: 1 became pre-candidate at term 4
Jul 20 16:31:41 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:41 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 4
Jul 20 16:31:41 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:41 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 3 at term 4
Jul 20 16:31:41 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:41 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 2 at term 4
Jul 20 16:31:41 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:41 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 3 [term: 2]
Jul 20 16:31:41 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:41 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 2 [term: 2]
Jul 20 16:31:43 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:43 raft.go:749: INFO: 1 is starting a new election at term 4
Jul 20 16:31:43 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:43 raft.go:594: INFO: 1 became pre-candidate at term 4
Jul 20 16:31:43 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:43 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 4
Jul 20 16:31:43 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:43 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 3 at term 4
Jul 20 16:31:43 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:43 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 2 at term 4
Jul 20 16:31:43 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:43 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 3 [term: 2]
Jul 20 16:31:43 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:43 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 2 [term: 2]
Jul 20 16:31:46 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:46 raft.go:749: INFO: 1 is starting a new election at term 4
Jul 20 16:31:46 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:46 raft.go:594: INFO: 1 became pre-candidate at term 4
Jul 20 16:31:46 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:46 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 4
Jul 20 16:31:46 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:46 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 3 at term 4
Jul 20 16:31:46 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:46 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 2 at term 4
Jul 20 16:31:46 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:46 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 3 [term: 2]
Jul 20 16:31:46 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:46 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 2 [term: 2]
Jul 20 16:31:48 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:48 raft.go:749: INFO: 1 is starting a new election at term 4
Jul 20 16:31:48 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:48 raft.go:594: INFO: 1 became pre-candidate at term 4
Jul 20 16:31:48 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:48 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 4
Jul 20 16:31:48 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:48 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 3 at term 4
Jul 20 16:31:48 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:48 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 2 at term 4
Jul 20 16:31:48 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:48 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 3 [term: 2]
Jul 20 16:31:48 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:48 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 2 [term: 2]
Jul 20 16:31:50 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:50 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 3 [term: 2]
Jul 20 16:31:50 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:50 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 2 [term: 2]
Jul 20 16:31:53 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:53 raft.go:749: INFO: 1 is starting a new election at term 4
Jul 20 16:31:53 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:53 raft.go:594: INFO: 1 became pre-candidate at term 4
Jul 20 16:31:53 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:53 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 4
Jul 20 16:31:53 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:53 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 3 at term 4
Jul 20 16:31:53 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:53 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 2 at term 4
Jul 20 16:31:53 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:53 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 3 [term: 2]
Jul 20 16:31:53 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:53 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 2 [term: 2]
Jul 20 16:31:55 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:55 raft.go:749: INFO: 1 is starting a new election at term 4
Jul 20 16:31:55 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:55 raft.go:594: INFO: 1 became pre-candidate at term 4
Jul 20 16:31:55 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:55 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 4
Jul 20 16:31:55 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:55 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 3 at term 4
Jul 20 16:31:55 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:55 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 2 at term 4
Jul 20 16:31:55 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:55 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 3 [term: 2]
Jul 20 16:31:55 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:55 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 2 [term: 2]
Jul 20 16:31:57 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:57 raft.go:749: INFO: 1 is starting a new election at term 4
Jul 20 16:31:57 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:57 raft.go:594: INFO: 1 became pre-candidate at term 4
Jul 20 16:31:57 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:57 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 4
Jul 20 16:31:57 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:57 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 3 at term 4
Jul 20 16:31:57 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:57 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 2 at term 4
Jul 20 16:31:57 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:57 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 3 [term: 2]
Jul 20 16:31:57 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:57 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 2 [term: 2]
Jul 20 16:31:59 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:59 groups.go:358: Serving tablet for: age_node
Jul 20 16:31:59 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:59 raft.go:749: INFO: 1 is starting a new election at term 4
Jul 20 16:31:59 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:59 raft.go:594: INFO: 1 became pre-candidate at term 4
Jul 20 16:31:59 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:59 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 4
Jul 20 16:31:59 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:59 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 3 at term 4
Jul 20 16:31:59 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:59 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 2 at term 4
Jul 20 16:31:59 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:59 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 3 [term: 2]
Jul 20 16:31:59 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:31:59 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 2 [term: 2]
Jul 20 16:32:02 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:02 raft.go:749: INFO: 1 is starting a new election at term 4
Jul 20 16:32:02 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:02 raft.go:594: INFO: 1 became pre-candidate at term 4
Jul 20 16:32:02 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:02 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 4
Jul 20 16:32:02 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:02 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 3 at term 4
Jul 20 16:32:02 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:02 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 2 at term 4
Jul 20 16:32:02 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:02 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 3 [term: 2]
Jul 20 16:32:02 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:02 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 2 [term: 2]
Jul 20 16:32:04 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:04 raft.go:749: INFO: 1 is starting a new election at term 4
Jul 20 16:32:04 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:04 raft.go:594: INFO: 1 became pre-candidate at term 4
Jul 20 16:32:04 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:04 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 4
Jul 20 16:32:04 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:04 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 3 at term 4
Jul 20 16:32:04 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:04 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 2 at term 4
Jul 20 16:32:04 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:04 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 3 [term: 2]
Jul 20 16:32:04 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:04 raft.go:731: INFO: 1 [term: 4] ignored a MsgPreVoteResp message with lower term from 2 [term: 2]
Jul 20 16:32:06 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:06 raft.go:749: INFO: 1 is starting a new election at term 4
Jul 20 16:32:06 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:06 raft.go:594: INFO: 1 became pre-candidate at term 4
Jul 20 16:32:06 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:06 raft.go:664: INFO: 1 received MsgPreVoteResp from 1 at term 4
Jul 20 16:32:06 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:06 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 3 at term 4
Jul 20 16:32:06 host-00-08.wgq.higgs.com dgraph[46352]: 2018/07/20 16:32:06 raft.go:651: INFO: 1 [logterm: 4, index: 125] sent MsgPreVote request to 2 at term 4
20

continue …


09 server :slight_smile:

– Logs begin at Wed 2018-01-31 14:35:14 CST. –
Jul 20 16:31:24 host-00-09.wgq.higgs.com systemd[1]: Started A Distributed, Fast Graph Database.
Jul 20 16:31:24 host-00-09.wgq.higgs.com dgraph[66033]: Dgraph version : v1.0.7
Jul 20 16:31:25 host-00-09.wgq.higgs.com dgraph[66033]: Commit SHA-1 : d9c3671f
Jul 20 16:31:25 host-00-09.wgq.higgs.com dgraph[66033]: Commit timestamp : 2018-07-19 13:28:43 -0700
Jul 20 16:31:25 host-00-09.wgq.higgs.com dgraph[66033]: Branch : HEAD
Jul 20 16:31:25 host-00-09.wgq.higgs.com dgraph[66033]: For Dgraph official documentation, visit https://docs.dgraph.io.
Jul 20 16:31:25 host-00-09.wgq.higgs.com dgraph[66033]: For discussions about Dgraph , visit http://discuss.dgraph.io.
Jul 20 16:31:25 host-00-09.wgq.higgs.com dgraph[66033]: To say hi to the community , visit https://dgraph.slack.com.
Jul 20 16:31:25 host-00-09.wgq.higgs.com dgraph[66033]: Licensed under Apache 2.0 + Commons Clause. Copyright 2015-2018 Dgraph Labs, Inc.
Jul 20 16:31:25 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:24 server.go:116: Setting Badger option: hdd
Jul 20 16:31:25 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:24 server.go:131: Setting Badger table load option: mmap
Jul 20 16:31:25 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:24 server.go:144: Setting Badger value log load option: disk
Jul 20 16:31:25 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:24 server.go:155: Opening postings Badger DB with options: {Dir:p ValueDir:p SyncWrites:true TableLoadingMode:2 ValueLogLoadingMode:0 NumVersionsToKeep:2147483647 MaxTableSize:67108864 LevelSizeMultiplier:10 MaxLevels:7 ValueThreshold:65500 NumMemtables:5 NumLevelZeroTables:5 NumLevelZeroTablesStall:10 LevelOneSize:268435456 ValueLogFileSize:67108864 ValueLogMaxEntries:1000000 NumCompactors:3 managedTxns:false DoNotCompact:false maxBatchCount:0 maxBatchSize:0 ReadOnly:false Truncate:false}
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 groups.go:78: Current Raft Id: 2
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 gRPC server started. Listening on port 9080
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 HTTP server started. Listening on port 8080
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 worker.go:89: Worker listening at address: [::]:7080
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 pool.go:108: == CONNECTED ==> Setting 172.20.0.8:5080
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 groups.go:105: Connected to group zero. Assigned group: 3
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 pool.go:108: == CONNECTED ==> Setting 172.20.0.8:7080
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 pool.go:108: == CONNECTED ==> Setting 172.20.0.10:7080
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 pool.go:108: == CONNECTED ==> Setting 172.20.0.10:5080
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 pool.go:108: == CONNECTED ==> Setting 172.20.0.9:5080
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 draft.go:74: Node ID: 2 with GroupID: 3
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 node.go:238: Group 3 found 1 entries
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 draft.go:955: New Node for group: 3
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 raft.go:567: INFO: 2 became follower at term 0
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 raft.go:315: INFO: newRaft 2 [peers: , term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 raft.go:567: INFO: 2 became follower at term 1
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 node.go:129: Setting conf state to nodes:2
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 raft.go:749: INFO: 2 is starting a new election at term 1
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 raft.go:594: INFO: 2 became pre-candidate at term 1
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 raft.go:664: INFO: 2 received MsgPreVoteResp from 2 at term 1
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 raft.go:580: INFO: 2 became candidate at term 2
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 raft.go:664: INFO: 2 received MsgVoteResp from 2 at term 2
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 raft.go:621: INFO: 2 became leader at term 2
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 node.go:301: INFO: raft.node: 2 elected leader 2 at term 2
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 groups.go:358: Serving tablet for: code
Jul 20 16:31:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:36 groups.go:358: Serving tablet for: name
Jul 20 16:31:37 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:37 raft.go:763: INFO: 2 [logterm: 2, index: 2, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:31:39 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:39 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:31:41 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:41 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:31:43 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:43 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:31:46 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:46 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:31:48 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:48 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:31:50 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:50 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:31:53 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:53 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:31:55 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:55 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:31:57 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:57 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:31:59 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:31:59 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:02 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:02 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:04 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:04 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:06 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:06 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:07 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:07 groups.go:358: Serving tablet for: degree
Jul 20 16:32:07 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:07 groups.go:358: Serving tablet for: gender
Jul 20 16:32:09 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:09 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:11 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:11 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:13 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:13 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:15 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:15 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:18 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:18 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:19 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:19 groups.go:358: Serving tablet for: unique_id
Jul 20 16:32:20 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:20 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:22 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:22 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:24 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:24 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:27 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:27 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:29 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:29 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:31 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:31 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:34 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:34 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:36 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:36 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:38 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:38 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:40 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:40 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:43 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:43 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:45 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:45 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:47 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:47 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:49 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:49 groups.go:358: Serving tablet for: degree_node
Jul 20 16:32:50 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:50 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:52 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:52 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:54 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:54 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:56 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:56 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:32:59 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:32:59 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:01 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:01 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:02 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:02 groups.go:358: Serving tablet for: gender_node
Jul 20 16:33:03 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:03 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:06 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:06 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:08 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:08 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:10 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:10 raft.go:763: INFO: 2 [logterm: 2, index: 3, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:12 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:12 raft.go:763: INFO: 2 [logterm: 2, index: 4, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:15 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:15 groups.go:358: Serving tablet for: candidate_dept
Jul 20 16:33:15 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:15 raft.go:763: INFO: 2 [logterm: 2, index: 5, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:17 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:17 raft.go:763: INFO: 2 [logterm: 2, index: 5, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:19 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:19 raft.go:763: INFO: 2 [logterm: 2, index: 5, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:21 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:21 groups.go:358: Serving tablet for: parent_industry
Jul 20 16:33:21 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:21 groups.go:358: Serving tablet for: started_work_at
Jul 20 16:33:21 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:21 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:24 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:24 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:26 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:26 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:28 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:28 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:31 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:31 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:33 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:33 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:33 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:33 groups.go:358: Serving tablet for: current_location_code
Jul 20 16:33:33 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:33 groups.go:496: Got address of a Zero server: 172.20.0.8:5080
Jul 20 16:33:35 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:35 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:37 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:37 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:40 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:40 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:42 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:42 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:44 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:44 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:47 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:47 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:49 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:49 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:51 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:51 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:53 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:53 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:56 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:56 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:33:58 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:33:58 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:00 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:00 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:03 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:03 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:05 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:05 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:07 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:07 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:09 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:09 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:12 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:12 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:14 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:14 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:16 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:16 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:18 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:18 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:21 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:21 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:23 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:23 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:25 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:25 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:28 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:28 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:30 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:30 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:32 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:32 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:34 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:34 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:37 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:37 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:39 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:39 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:41 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:41 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:44 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:44 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:46 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:46 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:48 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:48 raft.go:763: INFO: 2 [logterm: 2, index: 7, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:50 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:50 raft.go:763: INFO: 2 [logterm: 2, index: 8, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:53 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:53 raft.go:763: INFO: 2 [logterm: 2, index: 15, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:55 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:55 raft.go:763: INFO: 2 [logterm: 2, index: 15, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:34:57 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:34:57 raft.go:763: INFO: 2 [logterm: 2, index: 15, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:35:00 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:35:00 raft.go:763: INFO: 2 [logterm: 2, index: 15, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:35:02 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:35:02 raft.go:763: INFO: 2 [logterm: 2, index: 15, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:35:04 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:35:04 raft.go:763: INFO: 2 [logterm: 2, index: 15, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:35:06 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:35:06 raft.go:763: INFO: 2 [logterm: 2, index: 15, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:35:09 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:35:09 raft.go:763: INFO: 2 [logterm: 2, index: 15, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:35:11 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:35:11 raft.go:763: INFO: 2 [logterm: 2, index: 15, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2
Jul 20 16:35:13 host-00-09.wgq.higgs.com dgraph[66033]: 2018/07/20 16:35:13 raft.go:763: INFO: 2 [logterm: 2, index: 15, vote: 2] cast MsgPreVote for 1 [logterm: 4, index: 125] at term 2