V1.0.7 - Inconsistent query results between dgraph servers in same group


(coolguru) #1

We’re seeing inconsistent results being returned for the same query from all dgraph server in a particular group. We’re using K8s HA for our setup on AWS.

Issue went away when we rebooted all 3 dgraph server pods (in a particular group) separately. Let us know what other information would be helpful to debug this issue. Thanks!


(Manish R Jain) #2

Can you try without k8s and run Dgraph directly on host, so we know that this is not a k8s config issue?


(coolguru) #3

Do you guys have a cloudformation template for HA dgraph setup? That will make it easy to deploy to AWS.

Thanks!


(Michel Conrado (Support Engineer)) #4

this one serves? https://docs.dgraph.io/deploy#multi-host-setup

https://docs.dgraph.io/deploy#run-using-docker-compose-on-single-aws-instance


(Manish R Jain) #5

@coolguru: Can you put together steps to reproduce this, and file a Github issue? I’ll look into it.


(coolguru) #6

@mrjn https://github.com/dgraph-io/dgraph/issues/2503 Please let me know if more info is needed. thanks again!


(coolguru) #7

@mrjn wondering if you were able to repro the issue. Please let me know if there is anything I can do to help out, thanks!


(Manish R Jain) #8

I’m out on vacation. Will be back next week and look into this.


(Manish R Jain) #9

Btw, could you try with v1.0.4-rc4? Someone else in another topic said they didn’t see the inconsistency with that version.


(coolguru) #10

Come on Manish, stop slacking off dude! You’ve a bigger responsibility here! Just kidding…I appreciate you taking time during your vacation to respond to this thread. I’ll try out rc4 and update back.


(Paul Korzhyk) #11

I saw something similar and increasing RAM/CPU allocation seemed to improve things a lot in that case.
I can see you are already using m5.xlarge, but depending on your use case and budget it may be worth a try