New to Dgraph? DQL vs. GraphQL, Endpoints, Headers, and Schema Translation

I would like to better understand the relationship between the DQL schemas and the GraphQL schemas

This statement and a multitude of related questions (see References) have prompted me to write this to help new users. I remember when I started on the Dgraph road. Why were there so many endpoints? Which one should I use when? What is graphql±? (Meaning to me “GraphQL more or less.” Now it is commonly called DQL to avoid confusion and easier to say/write) Should I write my schema in GraphQL or DQL? What headers should I use? And a lot more of other questions along the way…

Dgraph started out as its own proprietary graph database without the support for the native GraphQL endpoint it has today. During its conception they wanted to use graphql but it could not support everything that was needed at the database level because no one has ever used it as a database level query language before. So the Dgraph team formed their own language based off from the foundation of graphql and termed it graphql± (Which is now commonly referred to as DQL). Building a Native GraphQL Database: Challenges, Learnings and Future

Why the history lesson? Because knowing its origin helps understand its structure and setup. For a video from the founder @mrjn check this out:

DQL is not within the spec compliance of GraphQL because of the needs of the database itself. Dgraph is a database and can be considered a single layer in a stack but inside the inner workings of Dgraph it consists of separate entities:

  • Badger - a Key-Value store
  • Ristretto - a cache
  • Dgraph (github link) - the methods and algorithms used to parse DQL (and now GraphQL) and act accordingly
  • Ratel - an optional GUI layer to work directly with DQL (Ratel does not work with the graphql endpoint at this time)
  • Dgraph Cloud(Formerly named Slash GraphQL) - a full featured layer that hosts Backends As A Service and builds implementations of Dgraph on a hosted environment such as AWS.

So in the beginning there was only a single schema, now known as a DQL Schema. As Dgraph developed, it drew much attention from the GraphQL world, but many developers faced the challenge that has almost always been there with using GraphQL in a stack: building the layer of resolvers from the queries, mutation, and subscriptions. This still required to developers to translate GraphQL to DQL to be able to use Dgraph natively with GraphQL clients.

There was one bright individual @michaelcompton who theorized a way to do much of this resolver building automatically. Dgraph brought Michael onto their team and began the work on a true GraphQL endpoint.

To be able to generate these resolvers, a user would have to tell the script what resolvers it wanted. This is done through something that we are very familiar with, a GraphQL schema. By defining types with properties, the automatic generation of these resolvers could be obtained. All that would be missing would be one part, the attached DQL underlying schema to actual store the data correctly. The resolver generation script (for lack of a better name right now) also handled this by translating a GraphQL schema to a DQL schema.

This whole process could have been a separate layer and optional for all of Dgraph users, but Dgraph took it a step further and brought the resolver and schema generation functions and put it right into the core. Now it works as a seamless one stop solution to a true native GraphQL database. To make it work as a single unit the resolvers layer is not actually a layer, but it functions that are used on the fly to translate the generated queries, mutations, and subscriptions into DQL to continue to request.

Dgraph could have pulled out the DQL schema and the DQL endpoints, but they found value in leaving it all usable by developers. Some developers will use only the GraphQL endpoint, some may only use the DQL endpoints, and some may use both.

Here is my understanding of the endpoints:

  • DQL Endpoints:
    • /query - used to make query requests in DQL
    • /mutate - used to make mutations in DQL, which can be mutated without committing the mutation
  • GraphQL Endpoint:
    • /graphql - used to host the GraphQL functionality, otherwise known in other Stacks as a separate layer.
  • Management Endpoints:
    • /alter - used to alter the DQL schema.
    • /health - used to do a health check on the server [Deprecated] Docs
    • /admin - a GraphQL API endpoint used to administer the GraphQL generated endpoint. You can read, the health, schema, and update the schema.
    • /admin/shutdown - intiates a proper shutdown of the Alpha [Deprecated] Docs
    • /admin/export - initiates a data export. [Deprecated] Docs
    • /admin/schema - used to alter the GraphQL schema via curl
    • /state - endpoint of Dgraph Zero returns a JSON document of the current group membership info. Docs

Note: I think that is all of them, but I might be missing something.

The question about the headers…

  • Content-Type - this header is needed for most endpoints.
    • For the /query endpoint you will need it set to graphql+-
    • For the /mutate endpoint you will need it set to application/rdf
    • For the /graphql and /admin endpoints you will need it set to application/json
    • I am not sure for the /alter endpoint as I never alter the DQL schema directly You can find this in the docs though most likely.
  • X-Dgraph-AuthToken - this is used when altering the DQL schema using the /alter endpoint.
  • X-Auth-Token - this is used exclusively by Dgraph Cloud I believe for the /admin endpoint. I believe the only way to secure the admin endpoint without using Dgraph Cloud is to do IP whitelisting. This might warrant its own discussion for expert advice if you are not using Dgraph Cloud.
  • Content-length - I do not think this is a required header on any endpoint. I have never used it as of yet.
  • X-CSRF-Token, X-Requested-With, and Access-Control-Allow-Origin - I believe all of these are specific to Slash’s implementation of CORS: https://dgraph.io/docs/slash-graphql/security/#restricting-cors-from-allowlisted-domains

I know this is not the case. I added all of my data with DQL and can access it with all of my GraphQL queries and mutations. The trick is to line up the DQL exactly with how GraphQL generates it.

Note: If you are going to do an import with DQL, you will need to build out your GraphQL schema first and then import the data.

Here is a very quick example of how a GraphQL schema aligns to a DQL schema. (I do not know all of the syntax for a DQL schema so I will show it with a query example on the DQL side.)

type Person {
  id: ID
  name: String!
  friends: [Person]
}

GraphQL query

query {
  queryPerson {
    id
    name
    friends {
      id
      name
    }
  }
}

DQL query:

{
  queryPerson(func: type(Person)) {
    id: uid
    name: Person.name
    friends: Person.friends {
      id: uid
      name: Person.name
    }
  }
}

Note that this would also get all of the same data but the property names of the data would not be the same:

{
  node(func: type(Person)) {
    uid
    Person.name
    Person.friends {
      uid
      Person.name
    }
  }
}

The main thing to learn is that any predicate or edge created in the GraphQL schema is created in a dotted notation for the DQL schema. (ie: how name evolved into Person.name).

Another note of importance in GraphQL to DQL schema is that DQL does not have the concept of interfaces. Here is an interface in GraphQL schema:

interface Location {
  lat: String
  lon: String
  acres: Float
}

type Property implements Location {
  id: ID
  price: Float
  owner: Person
}

In a GraphQL query, we can just request the predicates without needing to know the interface where the type came from:

{
  queryProperty {
    id
    price
    lat
    lon
    acres
    owner {
      id
      name
    }
  }
}

But in DQL, we need to know the parent type|interface of the predicate because of the transformation into DQL adds either the type or interface as the predicate’s name prefix.

{
  queryProperty(func: type(Property)) {
    id:uid
    price:Property.price
    lat:Location.lat
    lon:Location.lon
    acres:Location.acres
    owner:Property.owner {
      id:uid
      name:Person.name
    }
  }
}

References

38 Likes

This should be the first post shown to anyone new to Dgraph. Thanks :100:

12 Likes

Extremely helpful, was very confused about this. Thanks!

2 Likes

very helpful! I’ve been exploring dgraph for almost a week now and still didn’t really get it. I haven’t found anything in the documentation about the endpoints and when to use them.
I don’t know why this post is not in the documentation

4 Likes

This really should be folded into the documentation. It would’ve saved me hours of banging my head against the wall.

This statement and a multitude of related questions (see References) have prompted me to write this to help new users.

This should be ample justification! And you can add my thread wondering about this to the list of references.

2 Likes

@elhil

We just created on our unofficial discord channel, a collection of useful links (like this post here), and gotchas you really need to keep in mind while developing with dgraph! you can join the dgraph family https://discord.gg/W6GHYxzFeG

1 Like

:+1:

1 Like

This should be part of the official docs (and a YouTube video) under the broader category of: “How to extend GraphQL to do graph database stuff that GraphQL alone can’t do” and include:

  • The relationship between GraphQL schemas and DQL schemas above
  • How to integrate/call DQL queries/mutations from within GraphQL
  • Use cases, like recursive queries (that GraphQL can’t do, but which graph dbs are built for) such as “Start with node X and return the full sequence of nodes of an unknown depth”

Another title for a document within this section: “How to extend GraphQL to enable recursive queries and mutations using Dgraph DQL”

1 Like

I really believe we need two things here:

  1. A compiled list in a more official place (not discord) of common gotchas when working with Dgraph. This should include side by side between GraphQL and Dgraph and links to any possible work arounds. I began compiling some notes for this idea.

I would like to put this also beside SQL, Gremlin, Cypher, others(?) with links to how it is done in each language sort of building out more from GraphQL Syntax - Courses which does some quick side by sides with SQL/GraphQL

  1. A followup on: GraphQL vs DQL - Dgraph Blog to go more in depth with the differences like expand, normalize, var blocks, missing functions, upserts, interfaces, unions, lambdas, reverse vs. hasInverse, and maybe others not on the top of my head right now, oh like the upcoming new feature of defaults.

Then #1 can link to #2

This would also go a long way to help newcomers with “I know how to do in , but how do I do that in Dgraph?” And then also lead people quickly to you have to use X work around, or this is pending Y feature under development/RFC, or you can only do this with Z limitations/conditions.

2 Likes

Would love to see a @matt_tanner esque blog post + YouTube video of this use case as one way to dive into the topic: a use case that seems like a common graph db-related need (and related to some questions on this forum that do not seem to be completely answered): How to use GraohQL + DQL to deliver:

“Start with node X and return the full sequence of related nodes of an unknown depth”

In a format similar to Matt’s:

1 Like