Error in building indexes (Request bigger than maximum offset)

When I add an int index to a predicate, I get this error.

error in building indexes, aborting :: Request size offset 4789371283 is bigger than maximum offset 4294967295

The version of dgraph I use is v20.07.
In the cluster information, this predicate has 106GB space.

The location of the error has been found.

@ibrahim - what happens when the index is too big for the maxVlogFileSize ? How does one get there?

@Valdanito @chewxy Badger uses pointers to record the offset of the value in the vlog file. Since @Valdanito has a very big predicate, we’re trying to write at an offset bigger than 4 GB but the offset type in badger is uint32 which cannot point to more than 4 GB of data. The error means that we’re writing at an offset that we can never read.

@Valdanito what kind of index does this predicate have? I could try to reproduce this locally to better understand why are we generating so much data.

1 Like

It is an int type predicate, I tried to add an int index to it.