BadgerDB control/fine-tune disk writes

Greetings everyone,

I just started using badger and was wondering if there is a way to control when the vlog is synced to disk. It looks like that immediately after a transaction is committed the sync starts.

I set SyncWrites to false, and MaxTableSize is set to 64 MB, but still after every second Set operation, approx. 170 bytes written to the DB, I got another disk write.

Can I fine tune when a disk write happens?

This is the place where I see sync happening, irrespective of the value of SyncWrites. This would happen if the value log file gets filled up, which if you’re using default options is set to 1GB. This indicates that your values might be big.

If you’re seeing syncs happening more often (170 bytes), can you create a runnable Go code, and create a Github issue, so we can investigate.

Syncs happen only as they’re needed on SyncWrites=false. Any more fine tuning than that would lead to data corruption issues; so it’s better to investigate why syncs are happening as often as you say they’re.

Update: Note that disk writes != file syncs. Disk writes need to happen, but file syncs are what you control with SyncWrites option.

Thanks @mrjn for your reply.

I observed y.NumWrites and this happens for every write to the file descriptor. The doneWriting operation correctly happens if the ValueLogFileSize is reached. Sorry, I misunderstood y.NumWrites. Hence, the actual number of writes depends on the mmap configuration of the file descriptor, correct?

We don’t do writes to the mmap area. We write directly to the underlying file. So, it all depends on the OS when it wants to flush this out to the disk.

I see, thank @mrjn

So if I want to fine-tune this, I have to go to the OS/driver configuration. Thanks a lot for the support. Nice DB BTW :smiley:

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.