GraphQL+- poor support in GraphQL

The more I try to exploit the GraphQL native support of Dgraph, the more it seems to be difficult to combine GraphQL± and GraphQL.
Sometimes it feels like Dgraph’s GrapQL is capable to properly handle any kind of external source but the underlying Dgraph itself.
Don’t get me wrong, I’m really happy about the Database and it’s potential, but I cannot wrap my head around the trouble one has to face and workaround just trying to speak to Dgraph thought it’s own GraphQL interface.

Context: Dgraph generates its own query and mutation set starting from the schema. That’s quite a big help for the scaffold of an application but many times it’s just not expressive enough (you might want to have a more specific filter? custom query? custom mutation?).

Today I spent quite a few time playing around trying to embed some GraphQL± logic (which is waaaaay more expressive) in the GraphQL schema also following suggestions and indications in the this thread:

Schema:

type Name {
	name: String! @id @search(by: [hash])
}

type Person {
	name: Name!
	surname: Name!
}

type Test @remote {
  data: Q
}

type Q @remote {
  q: [P]
}

type P @remote {
  Per: [Person]
}

type Query{
	peopleByName(name: String!, query: String!): Test @custom(http: {
        url: "http://localhost:8080/query"
		forwardHeaders:["Content-Type"]
		method: "POST"
		body: "{ query: $query, variables: {$name: $name }}"
    })
}

The goal was pretty simple: Being able to create a custom query peopleByName which would exploit the reverse directive to get all person related to a specific name. I know I could’ve created an hasReverse field in the Name type to get the full inverse relation, but right know I’m just trying to reduce the complexity at the minimum while still trying to integrate GraphQL± in GraphQL. I’ll also close my eyes aboutthe fact that it’s not possible to make a GraphQL± query in custom Field resolver at the moment.

After the push of the Schema I set the Person.name relation as inverse to being able to traverse it starting from the name to the person

<Person.name >: uid @reverse .

The proceeded to populate some data in the Database thought GraphQL mutations

mutation {
  addPerson(input: [{name: {name: "Mario"}, surname:{name:"Rossi"}},
{name: {name: "Mario"}, surname:{name:"Verdi"}},
{name: {name: "Carla"}, surname:{name:"Rossi"}}]) {
    person {
      name {
        name
      }
      surname {
        name
      }
    }
  }
}

and finally got to the point of testing some query.
What I found are quite odd behaviour:

query {
  peopleByName(name:"Mario", query:"query a($name: string): { q(func:eq(Name.name, $name)) { Per: ~Person.name { expand(_all_) } }}") {
    __typename
    data {
      __typename
      q {
        __typename
        Per {
          __typename
          name {
            name  
          }
        }
      }
    }
  }
}

Running the query above I thought that once dgraph returned a Person node to the GraphQL layer, it would’ve used it’s internal resolvers to continue the resolution of the inner fields (I’m still not an expert of GraphQL, but it was fair logic to me that once you have a Person node, GraphQL knew how to resolve the name field) but the response was not as expected:

{
  "data": {
    "peopleByName": {
      "__typename": "Test",
      "data": {
        "__typename": "Q",
        "q": []
      }
    }
  },
  "extensions": {
    "tracing": {
      "version": 1,
      "startTime": "2020-07-22T11:46:32.424152949Z",
      "endTime": "2020-07-22T11:46:32.429471649Z",
      "duration": 5318800
    }
  }
}

I then tried to expend a level deeper, to give GraphQL all the information about the person and about the inner Name nodes:

query {
  peopleByName(name:"Mario", query:"query a($name: string): { q(func:eq(Name.name, $name)) { Per: ~Person.name { expand(_all_) {expand(_all_)} } }}") {
    __typename
    data {
      __typename
      q {
        __typename
        Per {
          __typename
          name {
            name  
          }
        }
      }
    }
  }
}

Response:

{
  "errors": [
    {
      "message": "Non-nullable field 'name' (type Name!) was not present in result from Dgraph.  GraphQL error propagation triggered.",
      "locations": [
        {
          "line": 49,
          "column": 11
        }
      ],
      "path": [
        "peopleByName",
        "data",
        "q",
        0,
        "Per",
        0,
        "name"
      ]
    },
    {
      "message": "Non-nullable field 'name' (type Name!) was not present in result from Dgraph.  GraphQL error propagation triggered.",
      "locations": [
        {
          "line": 49,
          "column": 11
        }
      ],
      "path": [
        "peopleByName",
        "data",
        "q",
        0,
        "Per",
        1,
        "name"
      ]
    }
  ],
  "data": {
    "peopleByName": {
      "__typename": "Test",
      "data": {
        "__typename": "Q",
        "q": [
          {
            "__typename": "P",
            "Per": [
              null,
              null
            ]
          }
        ]
      }
    }
  },
  "extensions": {
    "tracing": {
      "version": 1,
      "startTime": "2020-07-22T11:51:34.651053286Z",
      "endTime": "2020-07-22T11:51:34.657374286Z",
      "duration": 6321000
    }
  }
}

Better but not good, Graph returns the data with its internal schema, and GraphQL is not able to map back the fields with its own types.

Finally running a very long and stupid query where i manually maped back the fields, I managed to get the data:

query {
  peopleByName(name:"Mario", query:"query a($name: string): { q(func:eq(Name.name, $name)) { Per: ~Person.name { name: Person.name{name:Name.name} surname: Person.surname{name:Name.name} } }}") {
    __typename
    data {
      __typename
      q {
        __typename
        Per {
          __typename
          name {
            name  
          }
        }
      }
    }
  }
}
{
  "data": {
    "personsByName": {
      "__typename": "Test",
      "data": {
        "__typename": "Q",
        "q": [
          {
            "__typename": "P",
            "Per": [
              {
                "__typename": "Person",
                "name": {
                  "name": "Mario"
                }
              },
              {
                "__typename": "Person",
                "name": {
                  "name": "Mario"
                }
              }
            ]
          }
        ]
      }
    }
  },
  "extensions": {
    "tracing": {
      "version": 1,
      "startTime": "2020-07-22T11:55:46.318815768Z",
      "endTime": "2020-07-22T11:55:46.324802667Z",
      "duration": 5986899
    }
  }
}

I know that there some points the team is planning to cover in the current roadmap to improve the integration between GraphQL± and GraphQL, but I think that being able to use the native expressiveness of GraphQL± in the GraphQL schema should be a major priority.
For this particular example, the problem could be workarounded transforming of results of the Dgraph query trimming the {namespace}. part of the field’s name (which is added during the schema generation). A more complete solution, if possible, should be to delegate entirely to GraphQL logic the resolution of fields once Dgraph returns a “known” type (a not @remote type)

2 Likes

Herein I believe lies the paradox. Looking at your schema, the @remote directive at the top layer of the return tells Dgraph that the incoming data is not from within but is remote data. Hence it does not try to do any internal mapping of data down any nested graph traversal but rather says the data we got from peopleByName must be all the data there is available. IMO, the purpose of the @custom and @remote directives up to this point is for external data and not internal more advanced queries, even though it can be used as such by pointing back to itself as you did above.

From what I understand about 20.11 and future versions is that Graphql± is going to be included in a native way. I suspect that this will come with additional directives that function somewhat like @custom but with a hint of @dgraph tied together where it maps internally with advanced graphql± queries/mutations/upserts and provides the internal mechanisms to keep traversing down the graph given a found set of UIDs match the expected type.

Peering through some of the graphql ↣ dgraph inner workings yesterday I received a deeper understanding of how native graphql syntax is literally rewritten into graphql± syntax. There is a lot going on under the hood given the graphql implementation of the @search, @hasInverse, @id, @cascade, @include, @skip, @custom, @dgraph, @auth and now @withSubscription directives. The sheer power of having that much graphql± power with very simply graphql syntax is mind boggling.

I believe this is understood and is what is being worked towards. The sheer massiveness of the scale of getting this done is bigger than what we can understand given that it must still integrate with all of the directives mentioned above.

BTW, if graphql± is this important to you at the time being you can use the /query and /mutate endpoints for native graphql± and bypass the /graphql endpoint completely which may be simpler for some instances, but could also lead to bugs depending on implementation that is otherwise controlled via the rewrites I mentioned above. This may be helpful and save from the need to declare the requested graph data wanted by both the graphql custom query/mutation and within the in-schema graphql± queries/mutations. This would return the intention of a graph db back to its original intent “To fetch the requested data in the requested shape within a single network trip” whereas right now, you have to return the entire graph from the custom query to suffice any data that might be requested. Imagine the longer life of this which may become impossible for a more advanced schema where a graph could be an almost infinite number of nested graphs

2 Likes

@pawan Can you reconcile this issue with the JS logic executor we’re thinking of working on?

1 Like

@core-devs, thinking more about this and don’t really know exactly where you are headed but having some assumptions, I had an idea, lol.

Instead of extending the @custom directive I think that the proper directive name should be @resolver with the following type:

directive @resolver(resolve: Resolver!) on FIELD_DEFINITION
input Resolver {
    type: ResolverType!
    code: String!
}
enum ResolverType { JAVASCRIPT DGRAPH }

#examples
type User {
  username: String! @id
  firstName: String
  lastName: String
  name: String @resolver(resolve: {
    type: JAVASCRIPT
    code: """"
      const firstName = ${firstName}
      const lastName = ${lastName}
      let name = `${firstName} ${lastName}`
      return name.trim()
    """
  })
  account: Account
}

type Account {
  id: ID!
  isActive: Boolean! @search
}

type Query {
  queryActiveUsers(filter: UserFilter order: UserOrder first: Int offset: Int): User @resolver(resolve: {
    type: DGRAPH
    code: """
      {
        User(func: type(User)) @cascade {
          uid
          User.account @filter(eq(Account.isActive, true)) {
            uid
          }
        }
      }
    """
  })
}
2 Likes

Right now I’m just experimenting things in order to figure out what’s the limit of GraphQL, of GraphQL± and the combination of both.
Using the /mutation and /query endpoint is not something I considered yet, because before starting to complicate the infrastructure with a mixed Apollo Server which resolves half of the query using the GrapQL± endpoint and half using the delegateToSchema pipeline I really wanted to check how much of custom logic are we able to embed in the GraphQL schema.
The thread wasn’t ment to criticize but just to state the (quite big) problem any developer has to face trying to capitalize on the current GraphQL nativity of Dgraph (which again, I’m impressed and happy to use).


I tried to remove the @remote, making all the type converted in the Dgraph internal schema, but it’s still unable to map back the response and continue the resolution internally sadface.

2 Likes

I don’t think it was taken that way. The most important feedback is the true feedback even if it might seem negative.

Let me rephrase this ↴

After I wrote that and reread it, I thought that I did not explain it very well. Anything that is resolved through a @custom directive is cannot be expanded upon. Basically consider anything returned by @custom to be static content from that point forward. You can hide some it by changing the size of the graph but you cannot make it any larger by expanding predicates/nodes past that point. I am also not so sure, but you might also lose any sub filters past the data fetch. Which would mean that once the data is fetched you can not apply any more filters upon it. That would be an interesting test.

2 Likes

Oh I see, that’s why GraphQL couldn’t resolve the fields without feeding them directly from the GraphQL± query

1 Like

Thanks for all the feedback @Luscha and for chipping in and explaining various bits here @amaster507.

It is a major priority and also on the roadmap for the next quarter. You can expect support for this within the next month. This would allow you to easily be able to execute GraphQL+- queries (without making a HTTP request) to resolve certain fields.

So you are right here and @amaster507 also got this right. We expect fields resolved through @custom directive to return all the nested data for the field. So the flow of execution right now is like

  1. Get data for fields from Dgraph that is asked by the user or is required for resolving custom fields.
  2. Use that data to resolve custom fields.

So we only go one way right now, get data stored inside Dgraph and use that to resolve custom endpoints. We don’t do the other way around yet.

Now as you noticed, we could keep on going like this and then again do this recursively. That would give you an ability to join different Graphs (from your custom API and the one stored with Dgraph). We don’t do that right now but will be working on that as well in next couple of months.

Also, as @mrjn pointed out we are going to add support for executing Javascript code as part of the pipeline. The exact specification for this is yet to be worked out but here are some of the things it should allow.

  • The ability to resolve a field through JS code given to us instead of resolving it over a HTTP/GraphQL endpoint.
  • The ability to define pre and post hooks for resolving a query/mutation which give you the ability to transform/modify the result that you get from them.

We might add more bits to it as we start working on it. So yeah support for executing GraphQL± queries, traversing from one graph to another and back and also support for JS resolvers is all coming soon. In the meantime, we are also working on adding more documentation and examples.

4 Likes