This happens when we’re producing more ticks than what we can process. All operations in raft are dependent on the ticker.
@Valdanito How long has the cluster been running?
We’ve seen these messages in zero nodes. Can you confirm you’re seeing it in alpha nodes?
Each node has its own state. You shouldn’t copy over the data directory. You could add a new node to the group and the leader will stream the data to the new node (which is much better than copy-pasting the directory since the leader will stream only the valid data and it will clean up your disk too). This operation is generally safe but you should take a backup.
Also see
https://github.com/etcd-io/etcd/issues/9939
https://github.com/dgraph-io/dgraph/issues/2541