On Badger 1.6, I got an error on Close() that I’ve never seen before. Nothing else was going on at this time (no other reads/writes). There was only one Close() call on this database.
I looked through the code and am wondering if this is the problem, in value.go:updateDiscardStats:1385, the error occurs in the flushDiscardStats, which is inside a conditional that maybe typically isn’t triggered (i.e., the updates is less than the threshold of 100)?
if vlog.lfDiscardStats.updatesSinceFlush > discardStatsFlushThreshold {
if err := vlog.flushDiscardStats(); err != nil {
return err
}
vlog.lfDiscardStats.updatesSinceFlush = 0
}
The flushDiscardStats() method calls vlog.db.sendToWriteCh(), which checks the blockWrites flag, which close() itself sets. Meaning that this would never work inside of a close().
badger 2020/07/17 13:21:13 WARNING: LOG Compact FAILED with error: Writes are blocked, possibly due to DropAll or Close
github.com/dgraph-io/badger.init
/Users/donohoe/go/pkg/mod/github.com/dgraph-io/badger@v1.6.0/errors.go:113
runtime.doInit
/usr/local/Cellar/go/1.14.3/libexec/src/runtime/proc.go:5414
runtime.doInit
/usr/local/Cellar/go/1.14.3/libexec/src/runtime/proc.go:5409
runtime.doInit
/usr/local/Cellar/go/1.14.3/libexec/src/runtime/proc.go:5409
runtime.doInit
/usr/local/Cellar/go/1.14.3/libexec/src/runtime/proc.go:5409
runtime.doInit
/usr/local/Cellar/go/1.14.3/libexec/src/runtime/proc.go:5409
runtime.main
/usr/local/Cellar/go/1.14.3/libexec/src/runtime/proc.go:190
runtime.goexit
/usr/local/Cellar/go/1.14.3/libexec/src/runtime/asm_amd64.s:1373
failed to push discard stats to write channel
github.com/dgraph-io/badger.(*valueLog).flushDiscardStats
/Users/donohoe/go/pkg/mod/github.com/dgraph-io/badger@v1.6.0/value.go:1405
github.com/dgraph-io/badger.(*valueLog).updateDiscardStats
/Users/donohoe/go/pkg/mod/github.com/dgraph-io/badger@v1.6.0/value.go:1386
github.com/dgraph-io/badger.(*levelsController).compactBuildTables
/Users/donohoe/go/pkg/mod/github.com/dgraph-io/badger@v1.6.0/levels.go:623
github.com/dgraph-io/badger.(*levelsController).runCompactDef
/Users/donohoe/go/pkg/mod/github.com/dgraph-io/badger@v1.6.0/levels.go:764
github.com/dgraph-io/badger.(*levelsController).doCompact
/Users/donohoe/go/pkg/mod/github.com/dgraph-io/badger@v1.6.0/levels.go:833
github.com/dgraph-io/badger.(*DB).close
/Users/donohoe/go/pkg/mod/github.com/dgraph-io/badger@v1.6.0/db.go:414
github.com/dgraph-io/badger.(*DB).Close.func1
/Users/donohoe/go/pkg/mod/github.com/dgraph-io/badger@v1.6.0/db.go:351
sync.(*Once).doSlow
/usr/local/Cellar/go/1.14.3/libexec/src/sync/once.go:66
sync.(*Once).Do
/usr/local/Cellar/go/1.14.3/libexec/src/sync/once.go:57
github.com/dgraph-io/badger.(*DB).Close
/Users/donohoe/go/pkg/mod/github.com/dgraph-io/badger@v1.6.0/db.go:350
github.com/nytimes/pubp-monolog-sort/pkg/db.(*KeyValueDB).Close
/Users/donohoe/work/pubp-monolog-sort/pkg/db/db.go:83
github.com/nytimes/pubp-monolog-sort/pkg/db.(*BatchDB).Close
/Users/donohoe/work/pubp-monolog-sort/pkg/db/batch_db.go:196
github.com/nytimes/pubp-monolog-sort/pkg/info.(*BodegaBinaryInfo).Close
/Users/donohoe/work/pubp-monolog-sort/pkg/info/bodega_binaryinfo.go:59
main.main
/Users/donohoe/work/pubp-monolog-sort/cmd/bodega/bodega.go:104
runtime.main
/usr/local/Cellar/go/1.14.3/libexec/src/runtime/proc.go:203
runtime.goexit
/usr/local/Cellar/go/1.14.3/libexec/src/runtime/asm_amd64.s:1373
failed to update discard stats
github.com/dgraph-io/badger.(*levelsController).compactBuildTables
/Users/donohoe/go/pkg/mod/github.com/dgraph-io/badger@v1.6.0/levels.go:624
github.com/dgraph-io/badger.(*levelsController).runCompactDef
/Users/donohoe/go/pkg/mod/github.com/dgraph-io/badger@v1.6.0/levels.go:764
github.com/dgraph-io/badger.(*levelsController).doCompact
/Users/donohoe/go/pkg/mod/github.com/dgraph-io/badger@v1.6.0/levels.go:833
github.com/dgraph-io/badger.(*DB).close
/Users/donohoe/go/pkg/mod/github.com/dgraph-io/badger@v1.6.0/db.go:414
github.com/dgraph-io/badger.(*DB).Close.func1
/Users/donohoe/go/pkg/mod/github.com/dgraph-io/badger@v1.6.0/db.go:351
sync.(*Once).doSlow
/usr/local/Cellar/go/1.14.3/libexec/src/sync/once.go:66
sync.(*Once).Do
/usr/local/Cellar/go/1.14.3/libexec/src/sync/once.go:57
github.com/dgraph-io/badger.(*DB).Close
/Users/donohoe/go/pkg/mod/github.com/dgraph-io/badger@v1.6.0/db.go:350
github.com/nytimes/pubp-monolog-sort/pkg/db.(*KeyValueDB).Close
/Users/donohoe/work/pubp-monolog-sort/pkg/db/db.go:83
github.com/nytimes/pubp-monolog-sort/pkg/db.(*BatchDB).Close
/Users/donohoe/work/pubp-monolog-sort/pkg/db/batch_db.go:196
github.com/nytimes/pubp-monolog-sort/pkg/info.(*BodegaBinaryInfo).Close
/Users/donohoe/work/pubp-monolog-sort/pkg/info/bodega_binaryinfo.go:59