Txn is too big to fit into one request


(Rohanil) #1

I got error while running RunValueLogGC. It says Iteration function. Path=/db/000002.vlog. Error=Txn is too big to fit into one request. The vlog file size is 220MB. The DB has MaxTableSize set to 8MB and ValueLogFileSize is 1<<30 -1 which more than 1G, I think.

I checked the places where this error is in the code and found that the error is returned when this condition is true: count >= db.opt.maxBatchCount || size >= db.opt.maxBatchSize. Well maxBatchCount and maxBatchSize are derived from MaxTableSize. So is there any dependancy condition between MaxTableSize and ValueLogFileSize for GC?


(Ibrahim Jarif) #2

Hey @rohanil, are you using the latest version of the badger? We fixed a similar issue some days ago https://github.com/dgraph-io/badger/commit/740b849d305e62012c5454bbe7348851c8b69b26 .

So is there any dependancy condition between MaxTableSize and ValueLogFileSize for GC?

There is a dependency between the size of the table and the maximum size of the batch of entries we can push into badger. When we perform GC, we batch the valid entries together and push then into badger. In you case the size of batch is more than what badger can deal with.

For future reference, you can file issues on https://github.com/dgraph-io/badger/issues


(Rohanil) #3

My badger version is behind the above commit. I will update it and test with it. Thanks :slight_smile: