Txn is too big to fit into one request

Users Badger
1

I got error while running RunValueLogGC. It says Iteration function. Path=/db/000002.vlog. Error=Txn is too big to fit into one request. The vlog file size is 220MB. The DB has MaxTableSize set to 8MB and ValueLogFileSize is 1<<30 -1 which more than 1G, I think.

I checked the places where this error is in the code and found that the error is returned when this condition is true: count >= db.opt.maxBatchCount || size >= db.opt.maxBatchSize. Well maxBatchCount and maxBatchSize are derived from MaxTableSize. So is there any dependancy condition between MaxTableSize and ValueLogFileSize for GC?

2

Hey @rohanil, are you using the latest version of the badger? We fixed a similar issue some days ago Ensure rewrite in vlog is within transactional limits (#911) · dgraph-io/badger@740b849 · GitHub .

So is there any dependancy condition between MaxTableSize and ValueLogFileSize for GC?

There is a dependency between the size of the table and the maximum size of the batch of entries we can push into badger. When we perform GC, we batch the valid entries together and push then into badger. In you case the size of batch is more than what badger can deal with.

For future reference, you can file issues on Issues · dgraph-io/badger · GitHub

3

My badger version is behind the above commit. I will update it and test with it. Thanks :slight_smile:

1 Like