Retrieve large schema through GraphQL±

I have a huge schema of 235k predicates and my application has to retrieve that schema from dgraph first. I am using

schema { predicate type }

Which becomes a 17 MB gRPC response message, so I get the following exception:

Caused by: java.util.concurrent.ExecutionException: io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: gRPC message exceeds maximum size 4194304: 17476444
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
	at io.dgraph.DgraphAsyncClient.lambda$runWithRetries$2(DgraphAsyncClient.java:181)
	... 6 more

I can set the max inbound message size via maxInboundMessageSize to a higher value, e.g.

NettyChannelBuilder.forTarget("localhost:9080").usePlaintext().maxInboundMessageSize(24 * 1024 * 1024).build()

but no matter what value limit I set, there might be a schema that breaks this setting. I’d rather like a solution that 1) works for any schema and 2) has a small limited memory footprint. Is there something like pagination available for the schema { … } query, like first, offset and after?

No, I don’t think we have pagination for schema queries yet.
Your use-case seems legit. I feel like we should support it.

@pawan any thoughts?

1 Like

@EnricoMi Maybe this does not fits your needs but you can query for particular predicates or particular types. https://dgraph.io/docs/query-language/#querying-schema

We don’t support pagination on the schema query right now but we could support it. Could you please open an issue on GitHub and we can look into supporting it?

Sure!

A related issue is when I query the schema like schema { predicate type } I get predicates and types, and there is no way to get types or predicates only:

{
  "data": {
    "schema": [
      {
        "predicate": "dgraph.graphql.schema",
        "type": "string"
      },
      {
        "predicate": "dgraph.type",
        "type": "string"
      },
      …
    ],
    "types": [
      …
    ]
  },
  "extensions": {
    …
  }
}

There is a lot redundant or for me irrelevant information in that response, so I would like to be able to query more specifically and retrieve only "types" or "schema".

Should I raise a separate ticket for that too?

Sure, please raise a separate ticket for that.

For reference, raised feature requests #5810 and #5811.

2 Likes