Copying most of this from Discord Conversation with @MichelDiz
Has anyone in this community tried to use GraphQL Shield along with Dgraph’s GraphQL?
I was interested if anyone used it along with Dgraph’s GraphQL and Apollo Federation.
And then compare whether it would be better to use this instead of the built-in auth.
This could decrease the complexity of the Dgraph for some. And less code to maintain.
To do auth and pagination you would end up over-fetching. So the question is over’fetch in the db or over-fetch in the layer above meaning more network transportation, etc. At least from my understanding
Not sure what you mean. Why it would lead to over-fetching?
Let’s limit ourselves to the current Dgraph GraphQL API without any @auth rules. And then look at a somewhat simplified scenario to have a working schema to discuss.
So for the example, Let’s architect a simplified social media platform where we have just 3 data types, User, Post, Group and the simplified model for redundancy and somewhat full example would be modeled like:
#GraphQL Dgraph Schema
type User {
id: ID!
username: String! @id
posts: [Post!] @hasInverse(field: "author")
friends: [User!]
groups: [Group!] @hasInverse(field: "members")
}
type Post {
id: ID
posted: Datetime! @search
author: User!
group: Group @hasInverse(field: "posts")
title: String! @search
content: String! @search
}
type Group {
id: ID!
name: String! @search
members: [User!]
posts: [Post!]
}
Now let’s try to apply a logical OR condition with 3 rules to the Post type:
- You can query posts that you authored
- You can query posts that your friends authored
- You can query posts of groups that you are a member of
These rules would take the GraphQL shape of:
# posts you authored
query ($USERNAME: String!) {
queryPost {
author(filter: { username: { eq: $USERNAME } }) {
id
}
}
}
# posts that your friends authored
query ($USERNAME: String!) {
queryPost {
author {
friends(filter: { username: { eq: $USERNAME } }) {
id
}
}
}
}
# posts of groups that you are a member of
query ($USERNAME: String!) {
qureyPost {
group {
members(filter: { username: { eq: $USERNAME } }) {
id
}
}
}
}
Now let’s consider a query that we actually want to run. Let’s get the 5 most recent posts. This should obey the auth rules we have defined above.
query RecentFivePosts {
queryPost(first: 5, order: { posted: DESC }) {
id
posted
title
content
author {
username
}
}
}
So again limiting ourselves to the current capabilities of Dgraph (<= v21.12) how would you effectively query this with the rule logic without overfetching if the auth rules were in a layer above the database. Of course, you want to have that layer as close to the DB as possible, but even considering network lag and traffic between the database and your GraphQL server layer of abstraction, you will want to be very careful to only query what you need when you need it. Of course you could build a caching layer inside of your layer of abstraction, but then you have even that much more complexity to deal with and make sure it works right 100% of the time.
Seeing that @cascade is the logic that the @auth rules apply, you would have to factor that into the mixture here as well. Right now (IIRC) you cannot use cascade and pagination at the same time, because you may have incomplete results even then.
If you had nested filtering you might be able to get close to what you wanted by taking your rules and making them added on filtering in your GraphQL server layer. But that is not possible yet still.
You would basically have no choice but to run these queries to suffice even this very simplistic query:
# get my last 5 posts
query MyLastFivePosts($USERNAME: String!) {
getUser(username: { eq: $USERNAME }) {
posts(order: { posted: DESC, first: 5 }) {
id
}
}
}
# get all of my friends last 5 posts
query MyFriendsLastFivePosts($USERNAME: String!) {
getUser(username: { eq: $USERNAME }) {
friends {
posts(order: { posted: DESC, first: 5 }) {
id
}
}
}
}
# get all of my groups last 5 posts
query MyGroupsLastFivePosts($USERNAME: STRING!) {
getUser(username: { eq: $USERNAME }) {
groups {
posts(order: { posted: DESC, first: 5 }) {
id
}
}
}
}
Then you would have to flatten this graph to a unique list of ids, by flattening:
MyLastFivePosts.getUser.posts.id
MyFriendsLastFivePosts.getUser.friends.posts.id
MyGroupsLastFivePosts.getUser.groups.posts.id
And then after you flatten that to a list of ids, you can query again, to get the actual fields wanted in the original query:
query RecentFivePosts($IDS: [ID!]!) {
queryPost(filter: { id: $IDS }, order: { posted: DESC }, first: 5) {
id
posted
title
content
author {
username
}
}
}
I understand your worries. But graphql-shield would just do a single extra query. To get permissions data/context. The rest would be passed straight through. There would be no over-fetching in my opinion. And the pagination part didn’t make much sense either. Since graphql-shield will check the user permission once and keep it cached. From the little I understand about the shield and Dgraph junction, it would have this behavior and not over-fetching.
And this single extra query would get done how often? For every query done on the client? It has to get the current context at that point in time. It cannot work off from a cache and believe it has it all, unless the database is streaming updates to the cache and if it is doing that, then why even use the database, and not use persist the cache in the GraphQL server layer as the source of truth.
And it is that step alone of “a single extra query” that IS OVERFETCHING by definition.
And what if that extra query returns back hundreds, thousands, millions+ nodes of data even if just uid alone. How much time would that waste just transporting that data for context from the database to the GraphQL Server
From my point of view, if you remove @auth from the core of GraphQL, then you might as well remove GraphQL too. I would then build my entire GraphQL server using DQL resolvers (if I thought there was still value in doing so)
Here is how this would resolve to a single DQL roundtrip query:
# DQL (forgive syntax imperfections)
var(func: eq(User.username,"foo")) {
var1 as User.posts(orderdesc: "Post.posted", first: 5)
User.friends {
var2 as User.posts(orderdesc: "Post.posted", first: 5)
}
User.groups {
var3 as Group.posts(orderdesc: "Post.posted", first: 5)
}
}
RecentFivePosts(func: uid(var1,var2,var3), first: 5){
id: uid
posted: Post.posted
title: Post.title
content: Post.content
author: Post.author {
username: User.username
}
}
Nope, overfetching is when you get more then you need. If you need to check permissions, that’s not overfetching. Overfetching would be get a lot of non-needed info. In general that happens in REST API. Cuz a single endpoint can give you information that you don’t need.
Dgraph does permission checking all the time. The only difference here is that it is happening in a layer above. That would be similar a microservice logic. When you have the checking permission step in a separated service.
When using the GQL-Shield and Apollo Federation. It is talking Apples to Apples. There’s no chance from happening overfetching. There is a task delegation. To other layer.
In my opinion, GQL-Shield should work perfectly in conjunction with Dgraph. Because of Apollo Federation. And as far I remember, Michael [@michaelcompton] have added support for Apollo Federation.
I’m just interested in improving Dgraph and making it more friendly with other open-source projects. Dgraph Integrating with other projects. It’s a friendly strategy. Instead of being dependent on ourselves. Better to adopt than reinvent the wheel sometimes.
And I think this is where I differ from you. I believe this extra context checking IS OVERFETCHING because it brings extra stuff out of the Database layer into the network that never should have been fetched out of the database network.
And this is exactly where I was before I found Dgraph. I build a GraphQL layer just like you are talking about. But I was using a MySQL database instead of a GraphQL lower API or Dgraph DQL. I was doing all of this in my GraphQL API Server layer and things started grinding to a hault. The code complexity here is no joke. Maybe tools like GraphQL Shield might help that a little, but looking at just this example, GraphQL shield would IMHO have trouble knowing how to best optimize the rules and the context for this query. What if you had 5,000 friends and were in 50 groups, and on average every usre and group had posted 10,000 posts. Without knowing how to add that pagination and ordering correctly to the context query, you might have to fetch 50,500,000 just to have the context to get 5 posts!
It can be overfetching by your opinion. But from my understanding. It is something else. I really can’t see why a single extra step would do such harm. Really. (edited)
SQL you have overfetching for sure when doing the translation to GraphQL. That’s why there are cache techniques and dataloaders.
But that doesn’t happen when it is GraphQL to GraphQL in the case of Apollo Federation.
I’m all for integration and there are some key areas where I think (not knowing 100% about) GraphQL Shield could really add value instead of reinventing it on the API side, and that is auth based limitations. There is nothing in Dgraph right now that says a user can only query (to continue with our example) 1,000 posts a day. They could if they wanted to make a query up to the limitations of the database itself scraping it for all it’s worth. What if you had this power if Dgraph was the BE for Facebook. You don’t really want someone to write a GraphQL query to get 1 million posts in a single query. That would probably overload the http transport layer itself. And what if there were bots doing this on regular intervals… talk about a DDOS attack without even needing to think down to the DNS layer.
Also remember that GraphQL by specification has to be stateless too 
So now you are okay fetching 50,500,000 posts every time a new user wants to get their live feed. 
I don’t think the problem goes away when you are talking about GraphQL to GraphQL, because one could if very carefully craft SQL join statements that do all of the auth rule filtering and joining to not need to over-fetch data. Overfetching is not specific to SQL or any other database resolver query language. It is the act of transporting more data than necessary. But we can agree to disagree here and wait for someone anyone here that might actually have some real world experience with GraphQL Shield.
Do you use Dgraph’s pagination? or some recomended aproach in GraphQL like cursor?
I tried to, I don’t really use Dgraph at all right now.
You problem with pagination might be solve with Cursors.
But does Dgraph support cursors in the GraphQL layer… no. 
Cursors only really work for 2+ pages when you already have a starting point. What if you don’t have a starting point already?
That’s tricky, But surely someone has already explored good ideas for cursors for random paging.
I think an expert here might be @gajanan with his strong SQL background building query optimizers. I would be interested in his take on this topic. 
And to comment on this conversation again, I think this all comes down to these problems:
What do most all of these ^ have in common? — @cascade that is a key in this context. Even the @auth rules themselves work with the same @cascade logic. Fix @cascade and you can fix all of this 