Disable Write Ahead Log

We are using badger and raft together to build a distributed system. We have performance issue because we writing a write ahead log for raft and badger has its own WAL.

Is there anyway to disable WAL for badger because they are redundant in this case?

Hey @ehsannm, we recently added a way to directly hand over the skiplist to badger. see feat(Skiplist): Introduce a way to hand over skiplists to Badger by manishrjain · Pull Request #1696 · dgraph-io/badger · GitHub
Note: This option can only be used in managed mode.

Dgraph has integrated this change in master opt(txn commits): Optimize txns by passing Skiplists to Badger by manishrjain · Pull Request #7777 · dgraph-io/dgraph · GitHub :rocket:

Are you using Badger as a Raft WAL? If so, you should look into the Raft WAL implementation we use. It’s much faster.

@mrjn yes we are using badger for Raft WAL too. and also we use the same badger instance as our key/value store. I will check your raft WAL implementation for sure.

However our problem is that we are govern by Raft WAL and Raft Snapshot. Hence we don’t need badger WAL for our writes. I think we could safely ignore the second WAL and still be sure that our data
are safe. It I’m wrong please correct me.

@mrjn I just checked your raftwal package in dgraph and realized you are using etcd raft but we are using hashicorp raft. I reckon these two are not fully compatible. But I will try to understand what tricks you have used to make it fast. :wink: Thanks

If you’re using Badger as Raft WAL, then that Badger instance needs a WAL. At least in Etcd Raft, you are supposed to persist to disk before advancing. Without a WAL, you can lose data.

We used to have 2 instances of Badger, one for WAL, one for state. We replaced the WAL one with raftwal. The state keeping Badger is the one running without personal WAL.