# Compaction uses a lot of memory and is frequent

**URL:** <https://discuss.dgraph.io/t/compaction-uses-a-lot-of-memory-and-is-frequent/15903>\
**Category:** Badger\
**Tags:** badger\
**Created:** [October 28, 2021, 7:53am UTC](https://discuss.dgraph.io/t/compaction-uses-a-lot-of-memory-and-is-frequent/15903 "2021-10-28T07:53:04Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Leo\_Chen](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/leo_chen/32/9037_2.png) [@Leo\_Chen](https://discuss.dgraph.io/u/Leo_Chen)\
**Post date:** [October 28, 2021, 7:53am UTC](https://discuss.dgraph.io/t/compaction-uses-a-lot-of-memory-and-is-frequent/15903/1 "2021-10-28T07:53:05Z")

</div>

### What version of Go are you using (`go version`)?

```
$ go version
go version go1.17 linux/amd64
```

### What operating system are you using?

linux/amd64

### What version of Badger are you using?

badgerv3

### Does this issue reproduce with the latest master?

yes

### Steps to Reproduce the issue

I did a test and used bdagerdb as the storage engine of geth (replacing leveldb). The test results found that badgerdb has a serious memory jitter problem, and as time goes by, the performance on geth will gradually slow down. I All guesses have a big relationship with compaction, because I grabbed pprof’s heap analysis and compaction will occupy 3G-7G of memory at the peak

 ![image](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/6/67298a9695366c645cdb1ab2ec1b227bad0b8e90.png)

### What Badger options were set?

```
    opts.BlockCacheSize = 512 << 20
    opts.IndexCacheSize = 128 << 20
    opts.MaxLevels = 7
    opts.MemTableSize = 67108864
    opts.SyncWrites = false
    opts.NumCompactors = 10
    opts.NumLevelZeroTables = 20
    opts.NumLevelZeroTablesStall = 40
    opts.ValueThreshold = 128
    opts.WithCompression(options.Snappy)

```

### What did you do?

### What did you expect to see?

control compaction nagtive effect

### What did you see instead?

---

<div class="post-metadata">

**Author:** ![iluminae](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/iluminae/32/7367_2.png) [@iluminae](https://discuss.dgraph.io/u/iluminae)\
**Post date:** [October 28, 2021, 11:47am UTC](https://discuss.dgraph.io/t/compaction-uses-a-lot-of-memory-and-is-frequent/15903/2 "2021-10-28T11:47:04Z")

</div>

Have you tried something closer to the default settings? Dgraph uses it with a default of 2 compactors for example. Any difference?

(by the way, each compactor thread runs with a ticker on 50ms _possibly_ running doCompact() on every tick (though it will probably miss ticks if its busy). Check out` levels.go:levelsController.runCompactor()` - that is the goroutine that is spun up one per NumCompactor option.

---

<div class="post-metadata">

**Author:** ![Leo\_Chen](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/leo_chen/32/9037_2.png) [@Leo\_Chen](https://discuss.dgraph.io/u/Leo_Chen)\
**Post date:** [October 29, 2021, 9:19am UTC](https://discuss.dgraph.io/t/compaction-uses-a-lot-of-memory-and-is-frequent/15903/3 "2021-10-29T09:19:26Z")

</div>

At the beginning I used the default NumCompactors parameter, which has serious memory jitter. I tried to increase the number of this compactor to reduce the impact of compact, but it had no effect.

---

<div class="post-metadata">

**Author:** ![jarifibrahim](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/jarifibrahim/32/9077_2.png) [@jarifibrahim](https://discuss.dgraph.io/u/jarifibrahim)\
**Post date:** [November 7, 2021, 9:19am UTC](https://discuss.dgraph.io/t/compaction-uses-a-lot-of-memory-and-is-frequent/15903/4 "2021-11-07T09:19:01Z")

</div>

More number of compactors == more memory usage.

What are you trying to accomplish @Leo_Chen ?