If I’m wrong please tell me. The Title is Reduce memory usage of bulkloader.
Thanks for raising the PR. I was looking at the PR yesterday and plan to try it locally on my system today. You will see more activity on your PR soon. Really appreciate you taking the time to contribute.
Thank you for your reply! It’s my first PR, so I need some responses. I myself will continue testing.
Hi, I have finish my test. And I removed the use of a mapEntry pool, swiched back to new(mapEntry), because new(mapEntry) doesn’t affect the memory usage, but faster.
Now, my method not only reduce the memory usage of the reduce stage, but also achieves speedup.
Do you need I to provide you my dataset?