Become unable to execute mutation suddenly in write-heavy workload

Users
1

Hi,

I ran into an issue which can’t execute mutation while inserting a lot of data into Dgraph.
Each operation contains 5,000 triples, and the request is sent serially.
Also, Dgraph already stored 0.4 billion RDFs.

The result of an investigation, I found below.

  • A query is executable and performance isn’t decrease
  • I tried to restart zero and server but didn’t recover
  • zero and server don’t be output logs anything

Now I try to create a small example to reproduce this issue.
I use Dgraph in production so can you help me to resolve it.

my environment

  • version: v1.0.5 (commit SHA-1 82787414)
  • OS: CoreOS beta 1632.1.0
  • host: AWS EC2 r4.4xlarge

How do I run Dgraph

Use official docker image and serve zero and server in the same instance.

$ docker run -d \
  --name dgraph-zero \
  --net host \
  -v /home/core/dgraph:/dgraph \
  dgraph/dgraph:v1.0.5 \
  dgraph zero --my 10.1.3.106:5080 --idx 1

$ docker run -d \
  --name dgraph-server \
  --net host \
  -v /home/core/dgraph:/dgraph \
  dgraph/dgraph:v1.0.5 \
  dgraph server --my 10.1.3.106:7080 --zero 10.1.3.106:5080 --posting_tables memorymap --lru_mb 40960

logs

server

2018/06/08 09:50:11 oracle.go:75: purging below ts:1295243, len(o.commits):310, len(o.aborts):6, len(o.rowCommit):4
2018/06/08 09:51:11 oracle.go:75: purging below ts:1295250, len(o.commits):309, len(o.aborts):5, len(o.rowCommit):4
2018/06/08 09:52:01 oracle.go:75: purging below ts:1295257, len(o.commits):308, len(o.aborts):4, len(o.rowCommit):4
2018/06/08 09:53:11 oracle.go:75: purging below ts:1295264, len(o.commits):307, len(o.aborts):3, len(o.rowCommit):4
2018/06/08 09:54:21 oracle.go:75: purging below ts:1295271, len(o.commits):306, len(o.aborts):2, len(o.rowCommit):4

zero

2018/06/08 09:50:11 oracle.go:75: purging below ts:1295243, len(o.commits):310, len(o.aborts):6, len(o.rowCommit):4
2018/06/08 09:51:11 oracle.go:75: purging below ts:1295250, len(o.commits):309, len(o.aborts):5, len(o.rowCommit):4
2018/06/08 09:52:01 oracle.go:75: purging below ts:1295257, len(o.commits):308, len(o.aborts):4, len(o.rowCommit):4
2018/06/08 09:53:11 oracle.go:75: purging below ts:1295264, len(o.commits):307, len(o.aborts):3, len(o.rowCommit):4
2018/06/08 09:54:21 oracle.go:75: purging below ts:1295271, len(o.commits):306, len(o.aborts):2, len(o.rowCommit):4
2

i can not figure out what happened to dgraph, maybe i need more details about drgraph

3

What exactly is it that you want to know?

4

Version 1.0.6 will be out soon, so can you test with it? as soon as the binaries come out.
It would be interesting to reproduce this with public RDF. So that we can imitate the process.

5

What’s the issue here? It’s not clear.

1 Like
6

i need your help @mrjn

7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.