Question: more resilient to panic while db is closed but there may be some ongoing write transactions

hu13 · December 1, 2020, 1:13am

What version of Go are you using (`go version`)?

$ go version
go version go1.13.15 darwin/amd64

What operating system are you using?

macOS big sur 11.0.1

What version of Badger are you using?

1.6.2

Does this issue reproduce with the latest master?

Not sure

What Badger options were set?

default option.

Hi there, I’ve been trying badgerDB in one of my projects.
Basically I have some object A that contains a db reference and will create write transactions, etc.

However, at any time, there could be a user who will trigger an event to get rid of object A. This in turns should trigger a db’s close and terminates any on going db work.
However, some of the db work is done in the background and won’t be tracked.

If such background work tries to use a write transaction while the db is already closed, we will get a panic in the application.
Therefore, I am wondering if there is a way to say detect that connection is closed, don’t panic in those background work, but just clean up and exit gracefully?

As a comparison, If I use sqlite3 as my db store instead, I will just get connection closed error in the background work logic and can exit cleanly.

Thanks

ibrahim · December 7, 2020, 10:35am

Hey @hu13, you can use the db.IsClosed() API in your background routines.

github.com

dgraph-io/badger/blob/0f77c00a34d49d7e4b2e417c5f0899c6341712bb/db.go#L476


func (db *DB) Close() error {
	var err error
	db.closeOnce.Do(func() {
		err = db.close()
	})
	return err
}

// IsClosed denotes if the badger DB is closed or not. A DB instance should not
// be used after closing it.
func (db *DB) IsClosed() bool {
	return atomic.LoadUint32(&db.isClosed) == 1
}

func (db *DB) close() (err error) {
	defer db.allocPool.Release()

	db.opt.Debugf("Closing database")
	db.opt.Infof("Lifetime L0 stalled for: %s\n", time.Duration(atomic.LoadInt64(&db.lc.l0stallsMs)))

	atomic.StoreInt32(&db.blockWrites, 1)

There is still a chance that the transaction could panic and you should use Defer, Panic, and Recover - The Go Blog to recover from it. It’s hard to ensure that the transaction remains valid when the DB is being closed.

Topic		Replies	Views
Panic when concurrently closing DB while some goroutines doing reads Badger status:accepted , kind:bug , ticket:created	0	631	May 20, 2021
Best practices for closing badger db Badger discussion , best-practice	2	1674	January 29, 2019
Has a Badger Lock file? Badger	2	1485	October 22, 2018
How often should we open and close a db? On each request? Or each time a server comes up and goes down? Badger	0	457	June 26, 2022
WARNING: LOG Compact FAILED with error: Writes are blocked, possibly due to DropAll or Close Badger	3	809	July 17, 2020