Online write and query too slow

Issues Dgraph
, ,
1

Moved from GitHub dgraph/5590

Posted by JimWen:

What version of Dgraph are you using?

Dgraph version : v20.03.3
Dgraph SHA-256 : 08424035910be6b6720570427948bab8352a0b5a6d59a0d20c3ec5ed29533121
Commit SHA-1 : fa3c19120
Commit timestamp : 2020-06-02 16:47:25 -0700
Branch : HEAD
Go version : go1.14.1

Have you tried reproducing the issue with the latest release?

yes

What is the hardware spec (RAM, OS)?

128G mem & 1.8T SSD

Linux version 3.10.0-1062.9.1.el7.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Fri Dec 6 15:49:49 UTC 2019

Steps to reproduce the issue (command/config used to run Dgraph).

Situation 1
Bulkload large dataset and then online write data through java api or liveload, it’s hard to write any data, the import is very slow.

Even i just init the cluster with bulkload res and do not online write, the quey is much slower than v20.03.1.

Situation 2
liveload a large dataset, everything works fine at begin, but the import speed became slower and slower until it’s hard to write any thing.

Expected behaviour and actual result.

There is nothing exception in the log, and v20.3.1 and before is fine with the same env and dataset

2

Willem520 commented :

yes, I had meet the same problem.update is ok, but insert or upsert will make query more slower .Query throughput

3

OmarAyo commented :

Hi @JimWen

We can start analyzing the issue with the followings:

  • for the slow query please enable Jaeger tracing, this will help to understand which part of the query is taking more time
  • when live loading get a memory profile so we can see how much memory is being used by dgraph it self

Best,
Omar Ayoubi

4

JimWen commented :

The log is here Active Mutations stuck forever - #6 by JimWen.