I am trying to get to the bottom of severe performance issues on our dgraph system. I have traces indicating that
posting.Oracle().WaitForTs() (here) is taking several seconds on what seems to be most of my queries:
166.75ms: funcId=847459680 funcName=processTaskhca.xid message=Start. 166.75ms: message=Waiting for startTs: 16052571 at node: 3, gid: 1 4.85s: message=Done waiting for maxAssigned. Attr: "\x00\x00\x00\x00\x00\x00\x00\x00hca.xid" ReadTs: 16052571 Max: 16052571 4.87s: message=Done waiting for checksum match
What does this indicate?
dgraph version v21.03, 1group, 3alphas, 20 cores 27GiB ram each, 2TiB disk in gke.