To try to speed up loader, I turned on the bulkload option from RocksDB. However, ti seems to me that there is little gain here.
Here is my PR:
Here are the CPU and memory profiles.
I didn’t touch my machine as it does all the profiling. I did four runs in total (mem/cpu) x (bulk load or not).
It seems to me that the differences are within noise levels and I don’t think we should try to conclude anything from it, except that RocksDB’s bulkload has little impact on dgraphloader. In fact, runtime.cgocall seems to take less time without bulkload than with bulkload…
Previously, I also did some runs with assigner and loader, and it seems that bulkload actually slows down everything significantly. I’m not sure why that didn’t show up in the profiling of only the loader.
Unless anyone objects @minions, my take is not to use bulk load and there is no need to submit / review the PR.