Badger crashes on full disk

Issues Badger
1

What version of Go are you using (go version)?

$ go version
go version go1.16 linux/amd64

What operating system are you using?

Linux kernel 5.11.7 amd64

What version of Badger are you using?

v3.2011.1

Does this issue reproduce with the latest master?

Don’t know yet

Steps to Reproduce the issue

What Badger options were set?

What did you do?

What did you expect to see?

An error from badger

What did you see instead?

unexpected fault address 0x7f15b6c75000
fatal error: fault
[signal SIGBUS: bus error code=0x2 addr=0x7f15b6c75000 pc=0x472db2]

goroutine 610217 [running]:
runtime.throw(0x10f6662, 0x5)
	/usr/local/go/src/runtime/panic.go:1117 +0x72 fp=0xc01fc208d8 sp=0xc01fc208a8 pc=0x4381b2
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:731 +0x2c8 fp=0xc01fc20910 sp=0xc01fc208d8 pc=0x44f968
runtime.memmove(0x7f15b6c74f71, 0xc0001721c0, 0xe9)
	/usr/local/go/src/runtime/memmove_amd64.s:241 +0x272 fp=0xc01fc20918 sp=0xc01fc20910 pc=0x472db2
github.com/dgraph-io/badger/v3.(*logFile).writeEntry(0xc0000ac340, 0xc0004640c0, 0xc01fc20ad0, 0xc00013cdb0, 0xb, 0xc00013cdb0, 0xb, 0x0, 0x1, 0x0, ...)
	/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2011.1/memtable.go:344 +0x16b fp=0xc01fc20970 sp=0xc01fc20918 pc=0xe04d8b
github.com/dgraph-io/badger/v3.(*memTable).Put(0xc00023c6e0, 0xc8670adb40, 0x37, 0x37, 0x40, 0x0, 0xc8496716b0, 0xa8, 0xa8, 0x0, ...)
	/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2011.1/memtable.go:198 +0x25b fp=0xc01fc20c68 sp=0xc01fc20970 pc=0xe03e1b
github.com/dgraph-io/badger/v3.(*DB).writeToLSM(0xc000071c00, 0xc24ea02960, 0x0, 0x0)
	/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2011.1/db.go:715 +0x345 fp=0xc01fc20eb0 sp=0xc01fc20c68 pc=0xde1085
github.com/dgraph-io/badger/v3.(*DB).writeRequests(0xc000071c00, 0xc9bedcf7c0, 0x1, 0xa, 0x0, 0x0)
	/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2011.1/db.go:789 +0x314 fp=0xc01fc20f68 sp=0xc01fc20eb0 pc=0xde14b4
github.com/dgraph-io/badger/v3.(*DB).doWrites.func1(0xc9bedcf7c0, 0x1, 0xa)
	/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2011.1/db.go:830 +0x5a fp=0xc01fc20fc8 sp=0xc01fc20f68 pc=0xe1c87a
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc01fc20fd0 sp=0xc01fc20fc8 pc=0x471d61
created by github.com/dgraph-io/badger/v3.(*DB).doWrites
	/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2011.1/db.go:883 +0x46c
2

Hi @anya
Could you tell me how you got the error?

3

In my application, I just use Set, Get and Delete operations though transactions. If the disk where the badger database files are located gets full, instead of an error from I get a crash as in the stack dump above. The application does not use iterators.

4

I added a comment, with details on how to reproduce it, and a suggestion, on a similar issue on Github:

Basically, it is caused by writing mmap, and the idea of the solution is not to crash the runtime with SIGBUS, but recover it, and return an error.

1 Like
5

Hi @CristianCurteanu , thanks a lot for the details! We will look into this.