I’ve built more around my project thinking that the “if windows, add delay” on writes was sufficient, since Windows is not officially supported by the Badger team. It turns out the issue may be a bit deeper than that. My target environment will be to host this in a Linux container, and so I set up a project with a Dockerfile to build and test my solution. It turns out that the problem still occurs on a Linux kernel.
The problem is specifically related to writing transactions to the Badger KV store in rapid succession. My repository is in Github: GitHub - D-Haven/fact-totem: Fact Totem is an event store designed to allow authenticated access to different aggregates. and the familiar code from the last time I raised the issue is in the
Potentially relevant stats:
- CPU 4 core i5-4690K
- Go 1.16
- Badger v3.2103.0
- SSD reads: ~810 MB/s 1MB sequential, 670 MB/s random 4K
- SSD writes: ~740 MB/s 1MB sequential, 520 MB/s random 4K
- Run inside docker container
This is a data loss on write issue, as once the data is written, the reads work perfectly. Additionally for my tests, I’m using memory rather than disk. Not sure if that is relevant as well.