Alpha Crashes When Inserting Lots of Data Multithreaded

Issues Dgraph Clients
,
1

Moved from GitHub pydgraph/64

Posted by MeteHanC:

So I am inserting thousands of data with multi threading, so this caused the alpha to crash. Here is the error :

     

goroutine 56822524 [chan receive, 4 minutes]:
github.com/dgraph-io/dgraph/worker.MutateOverNetwork(0x15a8ae0, 0xcb3ff6c6c0, 0xd1bd8ef810, 0x0, 0x0, 0x0)
        /ext-go/1/src/github.com/dgraph-io/dgraph/worker/mutation.go:467 +0x41e
github.com/dgraph-io/dgraph/query.ApplyMutations(0x15a8ae0, 0xc31577d590, 0xd1bd8ef810, 0x0, 0x147dc9a, 0x17)
        /ext-go/1/src/github.com/dgraph-io/dgraph/query/mutation.go:51 +0x118
github.com/dgraph-io/dgraph/edgraph.(*Server).Mutate(0x1fbc410, 0x15a8ae0, 0xc31577d590, 0xd16d2a8f00, 0xd1c68f0240, 0x0, 0x0)
        /ext-go/1/src/github.com/dgraph-io/dgraph/edgraph/server.go:398 +0x517
github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api._Dgraph_Mutate_Handler(0x13d8f80, 0x1fbc410, 0x15a8ae0, 0xc31577c210, 0xd1bd8ef650, 0x0, 0x0, 0x0, 0xd1be38ac00, 0x97745f)
        /ext-go/1/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api/api.pb.go:1775 +0x23e
google.golang.org/grpc.(*Server).processUnaryRPC(0xc00019a000, 0x15ae3a0, 0xc161f00000, 0xd1be38ac00, 0xc0004d0390, 0x1ef2630, 0x0, 0x0, 0x0)
        /ext-go/1/src/google.golang.org/grpc/server.go:1011 +0x4cd
google.golang.org/grpc.(*Server).handleStream(0xc00019a000, 0x15ae3a0, 0xc161f00000, 0xd1be38ac00, 0x0)
        /ext-go/1/src/google.golang.org/grpc/server.go:1249 +0x1311
google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc161f2bc60, 0xc00019a000, 0x15ae3a0, 0xc161f00000, 0xd1be38ac00)
        /ext-go/1/src/google.golang.org/grpc/server.go:680 +0x9f
created by google.golang.org/grpc.(*Server).serveStreams.func1
        /ext-go/1/src/google.golang.org/grpc/server.go:678 +0xa1

It was working fine when I was inserting with only main thread. And memory usage is around %98. Maybe inserting with multi thread makes the insertions slow and when there are too many insertions stacked (so RAM usage is very high) and it makes the alpha crash?

I have 64 GBs of memory the usage was %98 and it suddenly dropped to %40, very odd. And I just realized that this is issue is related core Dgraph not the Python client. Should I re-open it on the main repo?

Thank you in the advance

2

martinmr commented :

Yes, this should be reopened in the dgraph repo. Also, it’d be helpful to know which version of dgraph you are using so that we can look at the right lines of code.
Also paste the complete stack trace is possible. If dgraph crash I doubt this snippet is the entire stack trace.