I’m currently on dgraph 21.12 unable to export my data to transition to a more stable dgraph version due to following error:
Dec 14 11:31:51 dm-dgraph-04 dgraph[224355]: I1214 11:31:51.840130 224355 log.go:34] Export [01h26m22s] Scan (12): ~2.1 TiB/2.5 TiB at 177 MiB/sec. Sent: 801.5 GiB at 231 MiB/sec. jemalloc: 7.5 GiB
Dec 14 11:31:55 dm-dgraph-04 dgraph[224355]: W1214 11:31:55.408201 224355 log.go:36] Error while sending: unexpected EOF
What I did
After a rather long investigation i came to the root of the issue. In short, I have a bloated reverse relation to the one of the most common nodes in the DB. It has around 72kk related nodes and size of badger KV structure with its RDF exceeds uint32 limit (with around 4.5Gb data). This uint32 limit is introduced inside ristretto buffer slice methods. I have fixed it and did a pull request to the ristretto repo, but it seems that those pull requests are not really investigated, as there are no responses from developers since the summer.
Maybe, if the devs see this here, they would check the issue out and introduce the fix to some of the dgraph versions to help people migrate (though, i have not seen any other mentions of this issue earlier).