I filed this issue on friday to no answer that was (unknowingly) a duplicate of this.
I completely rebuilt my production system on friday night to get around this (I have 3 shards so only one in that group is borked.) This just happened again on the new system. No restarts, brand new system as of saturday night - built from bulk loader, if that is interesting.
One of the peers just gives up with this MANIFEST removes non-existing table X and a restart will make it crashloop with the panic in the linked issue.
Here is the raft applied index of that group, you can see when the one encountered this error:
note: unlike OP I have 12 alphas, but it seems this could happen at any time. Each alpha is the only pod on a GKE VM. I have been running dgraph for over a year and have never encountered this until upgrade to v21.03.
Also see here, the memory usage of the node with this error increases where the others are low.
@ibrahim, please - there is some critical issue in badger it seems. If another peer in the same group corrupts thats probably full data loss without any further guidance.