Dgraph v1.0.6 is out

Dgraph v1.0.6 is out. It contains many bug fixes since v1.0.5. It is highly recommended to upgrade asap. Let us know your experiences with v1.0.6 in this thread.

Quickest way to upgrade is:

curl https://get.dgraph.io -sSf | bash

Link: https://github.com/dgraph-io/dgraph/releases/tag/v1.0.6

Changelog

Added

  • Support GraphQL vars as args for Regexp function. #2353
  • Support GraphQL vars with filters. #2359
  • Add JSON mutations to raw HTTP. #2396

Fixed

  • Fix math >= evaluation. #2365
  • Avoid race condition between mutation commit and predicate move. #2392
  • Ability to correctly distinguish float from int in JSON. #2398
  • Remove dummy data key. #2401
  • Serialize applying of Raft proposals. Concurrent application was complex and
    cause of multiple bugs. #2428.
  • Improve Zero connections.
  • Fix bugs in snapshot move, refactor code and improve performance significantly. #2440, #2442
  • Add error handling to GetNoStore. Fixes #2373.
  • Fix bugs in Bulk loader. #2449
  • Posting List and Raft bug fixes. #2457

Changed

  • Pull in Badger v1.5.2.
  • Raft storage is now done entirely via Badger. This reduces RAM
    consumption by previously used MemoryStorage. #2433
  • Trace how node.Run loop performs.
  • Allow tweaking Badger options.

Note: This change modifies some flag names. In particular, Badger options
are now exposed via flags named with --badger. prefix.

3 Likes

i will test the new version1.0.6

after trying to use --badger.vlog disk, i found the latency is much longer than before

maybe i should set --badger.vlog mmap, and then compare those

Push to docker hub please?

Expect that by end of day today.

Update: Docker v1.0.6 is out. Latest now points to v1.0.6.

See https://docs.dgraph.io/deploy for more information on how to get the latest Docker image.

2 Likes

Warning: Due to a change in Raft storage format, this upgrade would require you to export and re-import your data, clearing out all the WAL (zw in Zero and w in Server) directories.

Please take note of this. If you try to restart your servers with this upgrade with data in WAL from v1.0.5, they would fail with this stack trace:

github.com/dgraph-io/dgraph/raftwal.(*DiskStorage).parseIndex
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/raftwal/storage.go:160
github.com/dgraph-io/dgraph/raftwal.(*DiskStorage).seekEntry.func1
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/raftwal/storage.go:221
github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger.(*DB).View
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/badger/transaction.go:567
github.com/dgraph-io/dgraph/raftwal.(*DiskStorage).seekEntry
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/raftwal/storage.go:210
github.com/dgraph-io/dgraph/raftwal.(*DiskStorage).FirstIndex
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/raftwal/storage.go:243
github.com/dgraph-io/dgraph/vendor/github.com/coreos/etcd/raft.newLog
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/vendor/github.com/coreos/etcd/raft/log.go:53
github.com/dgraph-io/dgraph/vendor/github.com/coreos/etcd/raft.newRaft
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/vendor/github.com/coreos/etcd/raft/raft.go:294
github.com/dgraph-io/dgraph/vendor/github.com/coreos/etcd/raft.RestartNode
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/vendor/github.com/coreos/etcd/raft/node.go:219
github.com/dgraph-io/dgraph/dgraph/cmd/zero.(*node).initAndStartNode
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/raft.go:474
github.com/dgraph-io/dgraph/dgraph/cmd/zero.run
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/run.go:191
github.com/dgraph-io/dgraph/dgraph/cmd/zero.init.0.func1
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/run.go:60
github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra.(*Command).execute
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra/command.go:702
github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra.(*Command).ExecuteC
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra/command.go:783
github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra.(*Command).Execute
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/vendor/github.com/spf13/cobra/command.go:736
github.com/dgraph-io/dgraph/dgraph/cmd.Execute
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/dgraph/cmd/root.go:43
main.main
	/home/mrjn/go/src/github.com/dgraph-io/dgraph/dgraph/main.go:24

If an export and reimport is not possible for some specific reason, reach out to us.

ok.

@mrjn i found a situation:
i set replica=3, badger.vlog:disk
i have three dgraph server, i insert data to one server, and i can get results from this server, but i cannot get any results and i found p dir has not data in another two server,
12
04
19

why?

it seems that group members (id=2, id=3) dont have lastUpdate.
might be i just need to wait.

it seems you are to gradually migrate to json mutations?
Would you recommend users to gradually switch to json mutations and drop rdf n-quads?

that’s not a good idea to upgrade to v1.0.6.

server’s log



Are you using Docker, @shanghai-Jerry? I can’t replicate this issue at my end, but I can give you an image for you to try, and see if that solves this issue at your end.

@shanghai-Jerry and others, I’ve downgraded a library that we are using to the same version as we had before and updated the binaries for v1.0.6. Can you retry and check if you still see this issue?

I am not using docker.

at the end of this topic, i posted my server configs.

and

i used this query:

{
  query(func:has(name)) {
    count(uid)
  }
}

all three servers can return the same result:

{
  "data": {
    "query": [
      {
        "count": 6903
      }
    ]
  },
  "extensions": {
    "server_latency": {
      "parsing_ns": 6480,
      "processing_ns": 10197032,
      "encoding_ns": 2797792
    },
    "txn": {
      "start_ts": 69,
      "lin_read": {
        "ids": {
          "1": 14
        }
      }
    }
  }
}

but when i used a query with specific uid:

{
  query(func:uid(0x9b6)) {
    expand(_all_) {
      
    }
  }
}

only one server can get result

ok, i will retry. but when i use p directory created by bulk loader out directory to restart server, i got too many WARNs like:

2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!
2018/06/30 13:07:06 node.go:402: WARN: A tick missed to fire. Node blocks too long!

can you tell me what happened?
when i used v1.0.5, it’s normal, no WARNs

i’ve tried with v1.0.6, more details

but it still happened:

and

and

and when i restarted dgraph-server:

How did you load your data into the cluster? Did you start with empty directories?

How long did the cluster run before you saw these MsgVote logs?

i used mutate operation declared in api.proto, connected server with port 9080.

first, i started with empty directories. and then when i saw these MsgVote logs, i restarted with already exist directories. i found one server was getting snapshot from another server from p directory.

final question, after about one hour , i saw these MsgVote logs.

and now i found getting snapshot again

Added this warning in v1.0.6. I’m seeing some issues with replication. Working on solving those, and would be part of the next release.

Due to some issues with application of replication logs, we recommend only using --replicas=1 for this release. We’re working on solving these issues for the next release.

2 Likes

hi, @mrjn

i used to alter schema, but i found something in server’s log:

it seems that this log "Done schema update xxx " cannot stop, i only did alter schema operation once.

Any ETA for next release? Replication is critical for our production.

Well, if you do not need any of the features of version 1.0.6 or anything else, you can stick to version 1.0.5 until there is a final RC to next release.