Exception: file size: 380081 greater than 4294967295

Issues Badger
1

What version of Go are you using (go version)?

v1.16.3

$ go version

What operating system are you using?

Mac

What version of Badger are you using?

v2.0.0-20200630163423-09dfa663bbc3

When I run my program, it will exit abnormally after running for a while. The error is from the badger, said that " file size: 380081 greater than 4294967295"

I thought my disk space was not enough, but I found that my disk still has a lot of free space.

image
image856×545 154 KB

this error always happens, I don’t know what to do, can you give me some advice

2

I think you have ran into an issue where you are storing more than the allocated space (maxuint32 = 4294967295)

@ibrahim any clues?

3

thanks , is there any suggestions to solve it ?

4

Hey @chanders_bing it looks like we cannot create a big file because we would overflow maxUint32. This would happen only if you have too many duplicate keys in a single file.

If you can share the sst file that is bigger than 4.2GB in your data directory, I can help you figure out which is the key that’s causing the issue.

Also, you are using v2.0.0 which is the old release of Badger. We have v3 released which has a lot of improvements compared to v2.