Hey @keks51 , bulk loader exposes an http port to debug. You can take heap profile with a command that should look like:
go tool pprof localhost:8080/debug/pprof/heap
It will print out the destination:
Fetching profile over HTTP from http://localhost:8080/debug/pprof/heap
Saved profile in /home/dg/pprof/pprof.dgraph.alloc_objects.alloc_space.inuse_objects.inuse_space.003.pb.gz
File: dgraph
Build ID: a05419818fad7662cc49992bb9b39ae4426199ad
Type: inuse_space
Time: Oct 6, 2021 at 5:39am (PDT)
Entering interactive mode (type "help" for commands, "o" for options)
Heap profiles will help us debug the issue.
Also, please collect the jemalloc profile as well.
dg@machine:~$ curl localhost:8080/jemalloc
Num Allocated Bytes: 6.7 GiB [7235175716]
Allocators:
Tag: Reducer.ToList Num: 1 Size: 4.0 GiB . Tag: Builder Num: 1 Size: 384 MiB .
Allocations:
4.0 GiB at file: Reducer.ToList
260 MiB at file: Reducer.Buffer.ToList
2.1 GiB at file: Reducer.GetBuf
128 MiB at file: TableBuilder
256 MiB at file: Builder