[Doc] JSON DQL mutation for GraphQL Schema requires clarification

Users Dgraph Cloud
, ,
1

Since I was always using full predicate names on my DQL mutations in JSON format, I have never realised that the format which is presented in your docs (v21.03 and v22) is not valid. I did not yet have time to test it for v23 but from the changeling I can’t see anything which addresses this issue.

However, assume the following schema

type User {
  id: ID!
  name: String!
}

According to you docs, we should be able to add/update without typing out the full predicate

{
  "set": {
    "uid": "_:New",
    "dgraph.type": "User",
    "name": "John Doe"
  }
}

The mutation will succeed but when running a GraphQL query we get the error:

Non-nullable field ‘name’ (type String!) was not present in result from Dgraph. GraphQL error propagation triggered.

When updating a correctly defined User node (with User.name), name won’t be changed.

Working

{
  "set": {
    "uid": "_:New",
    "dgraph.type": "User",
    "User.name": "John Doe"
  }
}

Also, when checking out the predicate list in Ratel, we have a new name predicate of type string which obviously should not exist.

2

You are mixing GraphQL and DQL. The GraphQL type definition is translated to a DQL type definition. The DQL mutation then uses the generated DQL type definition, not the GraphQL type definition.

See this link: GraphQL on Existing Dgraph - GraphQL

3

@vnium did you click join the link to the documentation?

This has nothing to do with what gets translated, it’s a matter how the documentation is written.

Unless I got this completely wrong.

4

Hey Pool, @vnium is right. You are mixing the models here. The docs are correct. And you are too in the case of GraphQL. Which is not the case of that doc you mentioned.

5

@MichelDiz then it is just really confusing. Why is JSON Mutation Format listed directly under DQL then?

Screenshot 2023-03-23 at 10.12.33
Screenshot 2023-03-23 at 10.12.331856×993 187 KB

However, thanks for clarifying.

6

Did you find this specific example in the doc ?
We are revamping the doc at the moment. the JSON format is in DQL exactly for the reason your mention. By default it’s working as explained in DQL.
We are working on a specific page of the doc about GraphQL and DQL interoperability.
Your point is very valid and we will make sure we explain the naming convention when you are using a DQL mutation to set data for a GraphQL schema. This is a very common use case and we certainly have to clarify the doc on that subject. Thanks for the feedback.

7

DQL can be written in JSON format or RDF format. Both are valid DQL. GraphQL is something different.

J

8

Yep,

Although JSON is not literally DQL. It’s there not because it’s part of the language, but because it’s part of the mutation operation.

Also, RDF is not DQL, but it is part of this documentation topic for the same reason.

Cheers

9

Check out these learning resources:

1 Like
10

Will use a part of this post to update the documentation, it’s in the list…hopefully next week !

There are already some info at
https://dgraph.io/docs/graphql/dgraph/

11

First of all, thanks for all the answers. :raised_hands:

I guess I’m well aware about the difference of DQL and GraphQL. What I’m on about here is that it might be a bit confusing for new users when the documentation shows DQL mutations, written in JSON, which are based on a DQL schema only. I believe that most people (including me) are choosing Dgraph because of its GraphQL abilities, not the certainly powerful DQL query language.

I have to admit that, even though I was aware of a DQL schema, and that one gets auto-generated when deploying a GraphQL schema, I somehow did not see it as two separate things. This might have to do with how Dgraph blurs the difference between a Query language and a server which understands the query language and reacts according to it. Therefore, I have to slightly disagree with @amaster507 when he states that GraphQL is an API - there is a GraphQL API but GraphQL should always be understood as the layer between the requesting and the responding service. Dgraph kind of breaks this doctrine by adding a certain set of business logic to the schema itself where it usually does not belong (@auth directive).

I think Dgraph does a pretty good job here and it clearly shows that their approach is sophisticated enough to cover most of the business logic without writing tons of of code, setting up services, etc - this at least is true for most of our use cases. However, the missing, let’s say 10%, must usually be covered by writing custom resolvers or webhooks. And in my opinion, this is where you can’t really avoid DQL anymore. It’s just so much more powerful and lets you write safer and cleaner resolvers.

This is where it gets a bit complicated. You making a GraphQL request, process data with DQL in a custom resolver and (in most cases) must return valid GraphQL again. In our case this led to the development of quite a bit of tooling to deal with eg. the translation from GraphQL to DQL and vice versa, the parsing of the actual type from the dgraph.type array, identifying predicates which are from interfaces and thus require the correct predicate definition, etc.
Obviously, it could be the case that I simply do not know enough DQL or misinterpreted concepts and best practises.

Nonetheless, for these use-cases it would be nice if there would be eg:

  • a draph.interfaces array, which shows the interfaces which are implemented into a node
  • a flag for the DQL endpoint or a parameter for dql.mutate/dql.query (inside resolvers) to get a valid GraphQL response from a DQL request. I assume something like this already exists since DQL is running under the hood.
  • a DQL translation of the request body would also be nice :sweat_smile:

To summarise: I do not think Dgraph users should have to decide between using GraphQL and DQL. Most users will most certainly run into problems which they cannot tackle or getting overwhelmingly complicated when done using GraphQL only.

Replies

@Raphael the docs do not show this specific example but one which is quite similar:

   {
   "set": [
     {
      "uid": "_:diggy",
      "name": "diggy",
      "dgraph.type": "Mascot"
     },
     {
      "uid": "_:diggy",
      "specie": "badger"
     }
   ]
 }

Thanks for being on one page with me when it comes to GraphQL and DQL interoperability! :raised_hands: As already mentioned, we do have quite a bit of tooling in place which, if of any help, we’d be happy to share.

@MichelDiz thanks for clarifying that not JSON nor RDF is DQL. :pray:

@amaster507 kudos for writing this article! :pray: However, as already mentioned above, I’m not 100% agreeing with you here. GraphQL is not an API and nor should potential users decide if they want to use DQL or GraphQL. I think GraphQL should always be the main focus and I see DQL more in the realms of “backend operations”, meaning custom resolvers, etc.

2 Likes
12

@Poolshark thanks for the feedback, I’ll include you in PR reviews for the doc when we have the “DQL for GraphQL users” done.
That said, you are right, documenting how to do it now is a first step, the second will be to simplify things. I like your idea of tagging a DQL request “in the context of a GraphQL resolver” so we can do the translation work for you. Let us give it some thoughts … I recognize that we can improve things in that domain.

1 Like
13

The @auth directive is used in many GraphQL interfaces (neo4j, hasura, grafbase, etc, etc). Directives are not breaking any GraphQL specs.

It is also not true that everyone comes to Dgraph just for the GraphQL. The more and more I get into Dgraph, the more I appreciate DQL over GraphQL.

I do agree with you that GraphQL, literally short for Graph Query Language, is not just an API. Then again, is React a framework or a library?

Should Hasura users have to choose between GraphQL and SQL? Should neo4j users have to choose between GraphQL and Cypher?

They are two completely different things. The problem is that in this case DQL and GraphQL happened to be similar, so therefore more confusing. There are historical reasons for this I won’t get into, but it just happens to be similar for all intensive purposes. That doesn’t mean they should be considered the same thing.

As far as the schema, you should never touch the DQL Schema. Always let the GraphQL Schema generate this for you. You have one place. This would be the same for Prisma. You wouldn’t manually modify the SQL tables.

There is not direct way to translate DQL to GraphQL (not written) or GraphQL to DQL (generated, and unique to different @auth rules – could also be changed with a query planner one day).

What you’re really seeing is some holes in GraphQL. When nested filters, complex sorts, and more auth rules are added, you won’t need to worry about 95% of your custom dql problems.

J

1 Like
14

Incorrect assumption. I have a client I am working with currently that is not interested at all in the GraphQL API and builds their own (even better IMHO) GraphQL server on top of DQL directly.

Lol, but this is exactly what an API is by definition.

15

I’m not saying that they break GraphQL specs but @auth contains quite a bit of business logic which should be handled by the server and not be applied to the schema directly. However, I’ve also said that I do not mind this approach as long as it does not go crazy.

I did not say that every user prioritises GraphQL but I’m almost certain that most users start using Dgraph because there is a schema only approach. Moreover, as this resembles basically Dgraph’s USP. The very first headline on digraph.io literally claims

Dgraph is the only native GraphQL database with a graph backend.

Although, I also basically only use DQL in my custom resolvers.

No one should choose between one or the other and thus interoperability between, in Dgraph’s case, DQL and GraphQL is even more important! In my opinion Dgraph is always GraphQL first and DQL should only be used when you are forced to exend the core business logic. However, I guess there could be cases where you build your own layer on top of DQL and use something else than GraphQL as query language.

I do not agree. It has nothing to do with holes in the GraphQL layer. I would even advocate NOT to extend the complexity of the current GraphQL since you would put more and more business logic in the schema. At one point you might find yourself having an @execute directive, where you run command blocks - again, I don’t think that this should be part of the GraphQL schema. I already have such complex @auth rules which I would love to tackle properly with a programming language rather than query logic.

Again, I’m not saying that it is everyone but certainly most people are hooked by the GraphQl part. However, since your client builds his own GraphQL layer, he is also somehow interested in the GraphQL part and the fact that DQL is similar, will certainly help. I think this is actually a perfect example for the necessity of a better DQL ←→ GraphQL interoperability!

No, this is simply wrong. An API is much more than a query language like GraphQL. GraphQL by itself cannot and never will be able to provide (programmable) mechanisms connect to services. But read for yourself, graphql.org has it as it’s H1 header

A query language for your API

16

I think you’re misunderstanding how @auth directive works in Dgraph. Keep in mind I can’t speak for how it is implemented under-the-hood in other databases.

The @auth directive is actually handled on the server. It is not middleware, as a long time ago, the Dgraph team, for better or worse, decided to put the core code right next to the core dql code. I can confirm this, as I have dug deep into the core graphql code.

The @auth does not get triggered, and then your final query gets triggered separately. In fact, it compiles with your query, as an extra filter. This means it is faster, and you ultimately just have one filter. If I look for todos, my @auth is going to filter for todos I have access to form the beginning, hence never over-fetching, and then filtering those results.

It is actually pretty powerful and remarkable. There are definitely issues with implementing complex @auth directives in large schemas, there are some missing features etc, but it ultimately doesn’t slow down your code any more than necessary.

FWI: This is exactly how postgres policies work and compile.

I used to think this way, until I realized how big the Graph Database community is. A lot of people want something different than neo4j, and they don’t care about GraphQL. Dgraph is ultimately both a Firebase (Hasura) competitor and a neo4j competitor, and there sometimes isn’t a lot of overlap. I also suspect the Graph Database users (analytics etc) are the ones giving Dgraph the most money. There are also people in between who like and use both of course. Either way, I can tell you from many conversations on here, Dgraph isn’t quite sure the percentages, but they are certain who gives them money. This could have changed, however, in the last two years, but I doubt it. I could be wrong, but this is what I have gathered from the old active community.

Most of the features I believe Dgraph needs, won’t add any more logic to the Schema itself. @custom dql queries are the equivalent to @execute btw.

There have been many posts about separating the @auth rules into another file. I think this would be a good idea. First thing is first and get a new version up and running knocking out some of these huge bugs (as well as things like pre-hooks).

J

2 Likes
17

Love how this turns into a proper discussion here! :smile:

Although I haven’t dug deep into the the source code of Dgraph, I am pretty confident in my understanding of @auth. Of course @auth is handled on the server, there is no other way since GraphQL is nothing else than a query language. Therefore, your comment that there is no middleware is not really accurate - I’d say that you don’t have access to the middleware directly but, as you have stated yourself, Dgraph runs these @auth queries before the actual request. That’s basically the definition of a middleware.

Again, I have no problem with @auth but it is certainly an extension of business logic which GraphQL (as a query language) is simply not made for. Furthermore, in my opinion @auth for add mutations is completely useless since you do not have access to a valid UID yet. The only thing you can test for is key-value pairs in the user claim. But again - I’m not against @auth, I’m only saying that abusing a query language by implementing business logic might become. a dangerous path. Especially considering that the GraphQL team most certainly will not work towards something like that.

So just to clarify: Since GraphQL is a query language it should not be responsible for business logic other than the representation of it in the schema itself (my opinion). It is a great tool for filtering and that’s certainly something very powerful in this context. However, at least in my use-cases, it causes more problems (eg. no field protection and thus compromising the GraühQL schema, etc.) than having an additional “middleware” which lets me do auth properly.

As of performance, I don’t see a reason why a separate auth middleware (or an auth middleware where you have access to) should slow down the process. I think it only depends on how you implement things and how you write your code. I’m not sure but I could imagine that if someone goes crazy with queries inside @auth you could also slow down the process.

It certainly is! But that has nothing to do with Dgraph as a company. Dgraph’s approach is schema first - you yourself stated that “you should never touch the DQL schema”. So if this is the case, then even if people are interested in the graph database only, they still need to deal with somehow with Dgraph’s “GraphQL” part. If Dgraph should decide to focus more on the powerful graph DB, then even more so, logic needs to be separated from the actual query language. I 100% agree that there probably are quite a few people who want to build their own API around Dgraph’s graph DB - eg. @amaster507 client.

That’s what we do already because otherwise @auth rules are not maintainable in a larger projects - at least not in ours but that of course could also mean that your schema design has flaws. However, just separating into another file to have some of the stuff available for re-use, does not mean “it is not part of the schema file anymore”. You just made it a bit better looking.

Anyway, thanks for the discussion! :raised_hands: I’m thrilled to see what Dgraph will bring for us in the future!

18

Yes, it is middleware, just not separate logic. Semantics.

Extremely strong statement to say useless. Lots of databases don’t give you the UID beforehand, it just depends on your setup. Without getting into it on this posts, this particular issue may be solvable by using @id instead, create a new post if you have questions about that.

That being said, I personally won’t use it as a full security level for mutations until some of the bugs and @auth feature requests are fulfilled.

Again, very strong opinion. GraphQL is meant to be consumed on the frontend just like Rest API (although not limited to frontend). There is a reason you pass Authorization headers. The spec just doesn’t care how you handle them (Constraints, Authentication, Authorization, use RBAC, GBAC…etc)

If you use GraphQL. You can use Dgraph without GraphQL. Just use the DQL schema directly and ignore the GraphQL schema. I suspect more than half its users don’t use GraphQL.

I have taken part in many internal and external discussions. The new Dgraph team is focusing on both: the GraphQL Layer and DQL Layer as a Graph Database. Ultimately the foundation needs to be good at the bottom layer-up regardless.

This is again another strong statement from you, as completely separating logic would slow down your app. Perhaps we can agree that from a DX perspective, it should be separate, but should ultimately compile down to give use the fastest queries possible without over-fetching.

Well everyone definitely agrees with this that has built a medium sized or larger project.

The GraphQL spec is simply not opinionated on how you handle business logic; I personally wish it were. The @auth directive is just one pattern, and definitely has its problems. It is also not meant to handle constraints either.

I believe the Dgraph team is aware of these sufficiencies, and will eventually find a better way to separate this logic from a DX perspective. There is just so much work to be done first. I personally like the way Hasura does it.

J

2 Likes
19

You stopped the quote too early… It is also a runtime.

To say GraphQL is not an API is like saying REST is not an API or FHIR or HL7 or SOAP or CCDA or any other API flavor is not an API. It is the spec that the API must conform to, to be classified as a GraphQL server. But whatever you want to believe you may. You probably know GraphQL better than me anyways.

No they don’t… And this is not what Jonathan said either.

There are glaring holes in auth rules currently so be careful with them.

Auth will also currently slow down your queries if not done correctly which is close to impossible to do in the current state. I would almost say that there are NO secure Dgraph natiive GraphQL APIs. Anybody who thinks their auth rules are secure, please send me your graphql api endpoint and I will help you understand the issues.

1 Like
20

That’s exactly what I’m saying. Neither REST nor any other query language is an API. A query language simply can’t provide any form of resolver, it just tells you which data is available and what form this data has. This has nothing to do with believe - at least in my opinion. If GraphQL (or REST, …) would be an API, you could also say that Python, PHP, Node, … is a server because you can have a Python server, PHP server, etc.

And this is where (at least for me) the problem lies if you try to put too much logic into a (in this case GraphQL) query language. That’s simply not the idea of GraphQL.

However, I think I understand where you coming from and I guess we mean the same thing just have different semantics. :stuck_out_tongue_winking_eye:

True, he did not say that and I have to admit, that I haven’t dug into the code deep enough to argue against it. However, since we are basically writing queries in @auth and since everything gets translated into DQL on the Dgraph side, I could imagine that the auth queries are part of a query block. I guess when protecting a query, it can definitely be parsed and added as an additional filter, for mutations it will probably go in the query block of an upsert mutation. In both cases it’s something the server needs to evaluate before it returns the main query result - however the server does it in the end. But as said, there might be some very sophisticated things going on I simply don’t understand.

Have a nice week!

1 Like