How to solve mutation conflict

Issues Dgraph
,
23

Thanks. Can you also provide the periodic memory profiles, please? And the bulk loader and zero logs.
Note:
Bulk loader exposes HTTP port (default 8080 see --http) for profiling.
go tool pprof localhost:8080/debug/pprof/heap

24

sorry, man. we haven’t save logs or runtime status etc when dgraph bulk running.
i’m not good at pprof tool, i can’t save memory profiles when bulk failed, and i also don’t know when will bulk failed, but i remember it failed at REDUCE stage completion between 28%~29% many time (more accurately, it’s about 28.7x%).

if i use --skip_map_phase flag to skip the map phase, will the data lost which has already REDUCEed and saved in the out directory ?

25

We won’t be able to resolve the issue you are facing without the logs and profiles. Can you collect them when you run it again (even if you are skipping the map phase)? Do let us know if we can help somehow collecting them.

Yes, the out directory will be cleaned up before starting the reduce phase.

26

thanks for care about our problem.
i wanner do that so. but
em… the bulk process was killed by OS with -9 signal, so the alpha and zero has no chance to record error log. and we don’t know when will the process be killed, how to collect the memery profiles which is effective for you, the most detailed exception log i can find is from /var/log/messages. and we can’t watch this whole time, it’s too long.

27

You can write the logs to some file as they are generated? Like running

dgraph bulk -s schema -f rdffile |& tee bulk.log

Also, for profiling, it can be obtained by a single command: (now that 8080 is the default port opened by bulk loader for profiling)

go tool pprof localhost:8080/debug/pprof/heap

You can take it periodically by something like:

while ; do go tool pprof --text "localhost:8080/debug/pprof/heap" |& head -n 10 | tee -a pprof.log  && sleep 900 ; done;
28

i hav done that what u say. but the OOM doesn’t appeared.
maybe that because of i expand memery and processor, i have doubled resource, and used -j 3 option, the memery use is below 22% at whole runtime.

and maybe there is another problem. seems like when there is many rdf file, if the resource defined in one file and other resource in other file has edge point to that resource, that edge will lost after bulk loaded…

i’ll try again

29

Hey @Naman

I’m facing the same issue while doing bulk loader

2023/02/14 10:56:30 RDF doesn’t match schema: Input for predicate “Country” of type scalar is uid. Edge: entity:966178 attr:"\000\000\000\000\000\000\000\000Country" value_type:UID value_id:6338232

How to fix this? please suggest.

30

He doesn’t work at Dgraph anymore and if he still login in this community. I doubt he answer fast. His last login was Feb 28, '22. About a year ago.