Retrieve large schema through GraphQL±

Users Dgraph
1

I have a huge schema of 235k predicates and my application has to retrieve that schema from dgraph first. I am using

schema { predicate type }

Which becomes a 17 MB gRPC response message, so I get the following exception:

Caused by: java.util.concurrent.ExecutionException: io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: gRPC message exceeds maximum size 4194304: 17476444
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
	at io.dgraph.DgraphAsyncClient.lambda$runWithRetries$2(DgraphAsyncClient.java:181)
	... 6 more

I can set the max inbound message size via maxInboundMessageSize to a higher value, e.g.

NettyChannelBuilder.forTarget("localhost:9080").usePlaintext().maxInboundMessageSize(24 * 1024 * 1024).build()

but no matter what value limit I set, there might be a schema that breaks this setting. I’d rather like a solution that 1) works for any schema and 2) has a small limited memory footprint. Is there something like pagination available for the schema { … } query, like first, offset and after?

2

No, I don’t think we have pagination for schema queries yet.
Your use-case seems legit. I feel like we should support it.

@pawan any thoughts?

1 Like
3

@EnricoMi Maybe this does not fits your needs but you can query for particular predicates or particular types. https://dgraph.io/docs/query-language/#querying-schema

4

We don’t support pagination on the schema query right now but we could support it. Could you please open an issue on GitHub and we can look into supporting it?

5

Sure!

A related issue is when I query the schema like schema { predicate type } I get predicates and types, and there is no way to get types or predicates only:

{
  "data": {
    "schema": [
      {
        "predicate": "dgraph.graphql.schema",
        "type": "string"
      },
      {
        "predicate": "dgraph.type",
        "type": "string"
      },
      …
    ],
    "types": [
      …
    ]
  },
  "extensions": {
    …
  }
}

There is a lot redundant or for me irrelevant information in that response, so I would like to be able to query more specifically and retrieve only "types" or "schema".

Should I raise a separate ticket for that too?

6

Sure, please raise a separate ticket for that.

7

For reference, raised feature requests #5810 and #5811.

2 Likes