[badger v3] updateDiscardStats unexpected fault address

Issues Badger
1

What version of Go are you using (go version)?

$ go version
1.16

What operating system are you using?

linux/windows

What version of Badger are you using?

v3

Does this issue reproduce with the latest master?

Yes

Steps to Reproduce the issue

func TestRawDB(t *testing.T) {
	dir, err := ioutil.TempDir("", "badger-test")
	if err != nil {
		panic("failed to create temp dir")
	}
	options := badger.DefaultOptions(dir)
	options.Dir = dir
	options.CompactL0OnClose = true
	options.ValueThreshold = 1 << 9
	options.NumVersionsToKeep = 1

	db, err := badger.Open(options)
	if err != nil {
		fmt.Println(err)
		panic("failed db open")
	}

	var keys [][]byte
	randValue := make([]byte, 1<<12)
	for i := 0; i < 1<<17; i++ {
		randKey := make([]byte, 40)
		_, err := rand.Read(randKey)
		if err != nil {
			panic("failed to read rand")
		}
		keys = append(keys, randKey)
		err = db.Update(func(txn *badger.Txn) error {
			e := badger.NewEntry(randKey, randValue)
			err := txn.SetEntry(e)
			return err
		})
	}

	for j := 0; j < 4; j++ {
		for i := 0; i < len(keys); i++ {
			err = db.Update(func(txn *badger.Txn) error {
				e := badger.NewEntry(keys[i], randValue)
				err := txn.SetEntry(e)
				return err
			})
		}
	}

	time.Sleep(2 * time.Minute) // https://github.com/sahib/brig/pull/96#discussion_r576542931
	db.Flatten(4)
	db.Close()
	db, err = badger.Open(options)
	err = db.RunValueLogGC(0.5)
	if err != nil {
		fmt.Println(err)
	}
	db.Close()
}

What Badger options were set?

options := badger.DefaultOptions(dir)
options.Dir = dir
options.CompactL0OnClose = true
options.ValueThreshold = 1 << 9
options.NumVersionsToKeep = 1

What did you do?

We have an application deployed that sees duplicate writes to the same value. We are trying to observe/recreate how/when GC kicks in. Creating a test to rewrite many key value pairs multiple times then run GC.

What did you expect to see?

GC to kick in and reap the old versions of values

What did you see instead?

Exception has occurred: fatal error
Throw reason unavailable
Stack:
2 0x0000000000b1fcbb in encoding/binary.bigEndian.PutUint64
at C:/Program Files/Go/src/encoding/binary/binary.go:132
3 0x00000000010dac85 in /dgraph-io/badger/v3.(*discardStats).set
at C:/Users/jkuki/go/pkg/mod/_/dgraph-io/badger/v3@v3.2103.0/discard.go:90
4 0x00000000010db105 in ____/dgraph-io/badger/v3.(*discardStats).Update
at C:/Users/jkuki/go/pkg/mod/github.com/dgraph-io/badger/v3@v3.2103.0/discard.go:134
5 0x0000000001122371 in /dgraph-io/badger/v3.(*valueLog).updateDiscardStats
at C:/Users/jkuki/go/pkg/mod//dgraph-io/badger/v3@v3.2103.0/value.go:1109
6 0x00000000010ee41e in /dgraph-io/badger/v3.(*levelsController).subcompact
at C:/Users/jkuki/go/pkg/mod//dgraph-io/badger/v3@v3.2103.0/levels.go:861
7 0x000000000112cec8 in /dgraph-io/badger/v3.(*levelsController).compactBuildTables.func3
at C:/Users/jkuki/go/pkg/mod//dgraph-io/badger/v3@v3.2103.0/levels.go:924

2

Thanks, @jkuki for reporting the issue. This will be resolved by https://github.com/dgraph-io/badger/pull/1752

3

Thanks @Naman!

1 Like