Certain GraphQL Queries Hang


Report a GraphQL Bug

What edition and version of Dgraph are you using?

Edition:

  • SlashGraphQL
  • Dgraph (community edition/Dgraph Cloud)

If you are using the community edition or enterprise edition of Dgraph, please list the version:

Dgraph Version
$ dgraph version
 
v21.03.0

Have you tried reproducing the issue with the latest release?

Yes, this issue is exhibited in the latest release (v21.03.0).

Steps to reproduce the issue (paste the query/schema if possible)

I apologize I cannot paste the schema. The only data in the graph via ratel (v20.11.03) looks like:

"data": {
    "q": [
      {
        "uid": "0x3",
        "Common.name": "Name of Node",
        "NodeTypeA.edge1": [
          {
            "uid": "0x2",
            "Common.name": "Name1 of NodeTypeB"
          },
          {
            "uid": "0x4",
            "Common.name": "Name2 of NodeTypeB"
          }
        ],
        "NodeTypeA.edge2": [
          {
            "uid": "0x4",
            "Common.name": "Name3 of NodeTypeB"
          },
          {
            "uid": "0x5",
            "Common.name": "Name4 of NodeTypeB"
          }
        ]
      }
    ]
  }

And the query is something like:

query {
  getNodeTypeA(id: "0x3") {
     id
     name
     edge1 {
       id
       name
     }
     edge2 {
       id
       name
     }
     [all other edges from the schema even though no data populates]
  }
}

Expected behaviour and actual result.

I expect the GraphQL query to return almost instantly like the DQL equivalent query, but the above query hangs and the alpha node spins up a process that uses 100% CPU. The alpha’s 100% CPU usage doesn’t stop after the request is canceled. The only way to stop it is reset the alpha node. However, the following two queries will return instantly:

query {
  getNodeTypeA(id: "0x3") {
     id
     name
     [all other edges from the schema even though no data populates]
  }
}
query {
  getNodeTypeB(id: "0x4") {
     id
     name
     reverseEdge1 {
       id
       name
     }
     reverseEdge2 {
       id
       name
     }
  }
}

This is the output of the /debug/pprof/profile endpoint of the alpha:

File: dgraph
Build ID: 1885c8e72a106d80bcb295c63a32ea894168d40e
Type: cpu
Time: May 7, 2021 at 4:10pm (EDT)
Duration: 30.19s, Total samples = 3.27mins (650.08%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 110.87s, 56.48% of 196.29s total
Dropped 285 nodes (cum <= 0.98s)
Showing top 10 nodes out of 74
      flat  flat%   sum%        cum   cum%
    25.42s 12.95% 12.95%     65.02s 33.12%  runtime.mallocgc
    18.21s  9.28% 22.23%     57.22s 29.15%  runtime.concatstrings
    14.60s  7.44% 29.67%    183.31s 93.39%  github.com/dgraph-io/dgraph/query.(*graphQLEncoder).encode
    13.47s  6.86% 36.53%     13.47s  6.86%  runtime.memmove
     8.86s  4.51% 41.04%      8.86s  4.51%  runtime.nextFreeFast
     8.52s  4.34% 45.38%     12.54s  6.39%  runtime.heapBitsSetType
     7.54s  3.84% 49.22%      7.95s  4.05%  runtime.mapaccess1_faststr
     6.01s  3.06% 52.28%     49.23s 25.08%  github.com/dgraph-io/dgraph/graphql/schema.(*astType).ListType
     4.55s  2.32% 54.60%      4.55s  2.32%  github.com/dgraph-io/dgraph/graphql/schema.(*field).Name
     3.69s  1.88% 56.48%      9.08s  4.63%  bytes.(*Buffer).Write

And the /debug/pprof/heap output:

File: dgraph
Build ID: 1885c8e72a106d80bcb295c63a32ea894168d40e
Type: inuse_space
Time: May 7, 2021 at 4:10pm (EDT)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 91.37MB, 98.38% of 92.87MB total
Showing top 10 nodes out of 120
      flat  flat%   sum%        cum   cum%
   83.20MB 89.59% 89.59%    83.20MB 89.59%  github.com/dgraph-io/badger/v3/skl.newArena
    2.31MB  2.49% 92.08%     2.31MB  2.49%  github.com/dgraph-io/ristretto.newCmRow
    1.51MB  1.63% 93.71%     1.51MB  1.63%  runtime.procresize
    1.27MB  1.37% 95.08%     1.27MB  1.37%  bufio.NewWriterSize
    0.54MB  0.58% 95.66%     0.54MB  0.58%  github.com/blevesearch/bleve/geo.init.0
    0.53MB  0.57% 96.23%     0.53MB  0.57%  google.golang.org/grpc/internal/transport.newBufWriter
    0.50MB  0.54% 96.77%     0.50MB  0.54%  go.opencensus.io/trace.(*Span).interfaceArrayToAnnotationArray
    0.50MB  0.54% 97.31%     0.50MB  0.54%  google.golang.org/protobuf/internal/filedesc.(*Message).unmarshalFull
    0.50MB  0.54% 97.85%     1.50MB  1.62%  github.com/dgraph-io/gqlparser/v2/parser.(*parser).parseObjectTypeDefinition
    0.50MB  0.54% 98.38%     0.50MB  0.54%  google.golang.org/protobuf/internal/impl.fieldInfoForScalar
1 Like

I had a very similar issue to this. The schema was similar - a Node with two edges and some other fields. If I query only one of the edges, it works instantly, but if I query for both of them simultaneously, like your example - it just hangs.
The odd thing that fixed it was I rearranged the links in the query.
For example, this doesn’t work:

query {
  getNodeTypeA(id: "0x3") {
     id
     name
     edge1 {
       id
       name
     }
     edge2 {
       id
       name
     }
  }
}

This does

query {
  getNodeTypeA(id: "0x3") {
     id
     name
     edge2 {
       id
       name
     }
  }
}

This does

query {
  getNodeTypeA(id: "0x3") {
     id
     name
     edge1 {
       id
       name
     }
  }
}

This does

query {
  getNodeTypeA(id: "0x3") {
     id
     name
     edge2 {
       id
       name
     }
     edge1 {
       id
       name
     }
  }
}
1 Like

Hi @StellarAnt and @testak ,

Welcome to Dgraph Community !!

Thanks for posting this bug. We had made changes to how graphql results are encoded in Dgraph in January and the new encoding has this issue.

This issue was reported earlier and has been fixed in master and upcoming 21.03.1 release with the following PR, Fix(GraphQL): Fix GraphQL encoding in case of empty list by vmrajas · Pull Request #7726 · dgraph-io/dgraph · GitHub .

This happens when edge1 is an empty list and edge2 has more than one elements.

1 Like

Thank you for the welcome!

The issue you described fits perfectly with what I have been experiencing. I will keep an eye out for the next release. Thanks for your help and quick response!