Keeping key version when adding same key on the same transaction

Is it possible to keep the key version when adding the same key from the same transaction (or write batch)?

For example:

If I add a key a in the same transaction twice, only the last one is commited:

txn := db.NewTransaction(true)
defer txn.Discard()

err = txn.Set([]byte("a"), []byte("41"))
if err != nil {
	log.Fatal(err)
}

err = txn.Set([]byte("a"), []byte("42"))
if err != nil {
	log.Fatal(err)
}

if err := txn.Commit(); err != nil {
	log.Fatal(err)
}

This will return only 42 when I iterate through key a.

But, if I split the transaction into 2 transactions:

{
	txn := db.NewTransaction(true)
	defer txn.Discard()

	err = txn.Set([]byte("a"), []byte("41"))
	if err != nil {
		log.Fatal(err)
	}

	if err := txn.Commit(); err != nil {
		log.Fatal(err)
	}
}

{
	txn := db.NewTransaction(true)
	defer txn.Discard()

	err = txn.Set([]byte("a"), []byte("42"))
	if err != nil {
		log.Fatal(err)
	}

	if err := txn.Commit(); err != nil {
		log.Fatal(err)
	}
}

Now I get the desired 41 and 42 values during iteration.

My reason to want that is that my WriteBatch is created from another system, and it is possible that a repeated key is inside that batch.

Best regards,

Eduardo

I think I found out a way to do this, if I open the database as a managed DB, I can specify the version timestamp for each key and then my values will be added correctly.

The only issue now is that with this approach, seems like NumVersionsToKeep is ignored, since I always get all versions for that key during iteration instead of only the max number by above option.

This is the code:

options := badger.DefaultOptions("./badger_db").WithNumVersionsToKeep(2)

db, err := badger.OpenManaged(options)
if err != nil {
	log.Fatal(err)
}
defer db.Close()

batch := db.NewManagedWriteBatch()
defer batch.Cancel()

err = batch.SetEntryAt(&badger.Entry{Key: []byte("a"), Value: []byte("41")}, uint64(time.Now().UnixNano()))
if err != nil {
	log.Fatal(err)
}

err = batch.SetEntryAt(&badger.Entry{Key: []byte("a"), Value: []byte("42")}, uint64(time.Now().UnixNano()))
if err != nil {
	log.Fatal(err)
}

err = batch.Flush()
if err != nil {
	log.Fatal(err)
}

Hmm… No one knows how to fix this? There is not much documentation about ManagedDB

Hey @sezaru, apologies for the delay in the reply.
Badger keeps the transactions in-memory before they’re committed.

In normal mode, if you write the same key twice, we will overwrite the first value (which is fine because you want to update the value).

You’re correct that you will need managed mode. The setEntryAt will commit entry at a specific timestamp. However, please be aware that managed mode is used when the application can deal with transaction conflicts and provide correct timestamps (dgraph used managed mode and deals with this).

If you commit things at an incorrect timestamp, you might not be able to read it correctly.

The NumVersionsToKeep flag is not used in the iterators. The NumVersionsToKeep flag is used by compaction to determine if the value should be kept or removed. So if the data you inserted is compacted (which happens when you have enough data), we will remove more than 2 versions of the key.
Iterators do not respect the NumVersionsToKeep flag.