Inability to do simple GraphQL query filtering on node / edges like other GraphQl BaaS'es offer?

I may be missing something fundamental here, but building queries via Slash’s GraphQL API Explorer seems to limit query filters ONLY to String data types and not via Node/Edge relationships?

In example schema below, you can create a Slash GraphQL query with a filter on NodeX where id = 123, but I seem to be unable to filter on “Only return NodeX’es where their NodeY = 456?”

type NodeX  {
	id: ID!
	name: String @search(by:[fulltext])
 	nodey: [NodeY] @hasInverse(field:nodex)
}
type NodeY  {
	id: ID!
	name: String @search(by:[fulltext])
 	nodex: [NodeX] @hasInverse(field:nodey)
}

Expected filtering something like other GraphQL BaaS services offer:

query CorrectDataQuery {
  NodeXList(filter: {nodey: {every: {id: {equals: "456"}}}}) {
    items {
      id
      name
    }
  }
}

But, I’m only allowed to do?:

query ReturnsWrongDataQuery {
  queryNodeX {
    id
    name
    nodey(filter: {id: "456"}) {
      id
      name
    }
  }
}

Which results in the wrong set of data?

In the example above, you COULD query on NodeY (not NodeX) where id = 456, return that node, AND return its nested, related set of nodeX’s. This may be OK in that instance… but when you want to build rich queries with multiple filters on both string data and node/edge relationship data, that becomes impossible?

The entire reason to use GraphQL – and specifically a graph db-based BaaS like Dgraph – for me was to have rich node/edge data and – hopefully – make it super simple to work with those edge/node relationships? But maybe that is not the case?

Is this a limitation of Dgraph Slash or Dgraph graph database? Or am I missing something basic?

Thanks much.

OK, seeing that there are related discussions here via “nested filters” discussions.

What is the latest on this as it applies to Slash / Dgraph Cloud and GraphQL filters?

Feedback: it seems kind of odd to me that this kind of node/edge filtering functionality is not a fundamental, starting point, core element of a GraphQL API BaaS that is based on a graph (node/edge-rich) database… while GraphQL API BaaS alternatives have this easy-to-use functionality with their SQL-based solutions.

Again, though, I may be missing something fundamental. Thanks.

Hi @hellohello,
yeah, currently we don’t have support for nested object filters in GraphQl and it’s requested a lot from the community.
We recently compiled requests for nested object filters and detailed RFC for it coming very soon.
Thanks for the detailed feedback and concern.

Sure you can!..

There are currently two ways to do this and the latter will be more effective.

  1. Use @cascade if there are other fields that may be empty. Cascade removes any top level nodes post query pre response that have null fields.

  2. Flip the query. This is the better way to do it as it is the graph way. You know a edge that you want to filter by, then use that as the root. This does evolve the app development as the response may then not be modeled exactly as you might expect normally.

{
  getNodeY(id: "456") {
    nodey {
      id
      name
    }
  }
}

But, yes it would be really nice if we could filter by the edges without needing to flip the query.

Thanks @amaster507. Yeah, as I mentioned, you can flip it if that’s the sole filter to query, but if you are trying to do richer filtering with multiple filters, you’re out of luck it seems.

Example: List all nodeX’es that have related nodeY’s = 123 and related nodeZ’s = 456 and…

In other words: rich relationship querying and mutations.

Which is what I thought I was getting when turning down tried-and-true SQL-based GraphQL APIs and coming to a graph database-based GraphQL API. @JatinDevDG do you have a rough ETA for this or no? Thanks.

For what it’s worth, I completely understand that new services have to work through a backlog of features. But in this particular case, no matter when the feature comes, it now makes me super nervous to think that a graph database-based GraphQL API that is trying to separate itself from competitors that use more traditional, popular databases has launched without this kind of (fundamental?) node/edge relationship management as a fundamental core part of its offering?

I could be a dummy, but 'm not sure of the benefit of graph databases vs SQL competitors in this current GraphQL scenario? Despite Slash’s usage of a graph db, it seems to be at a big disadvantage, not an advantage? In other words, the rich node/edge relationship benefits promised are not the benefits delivered?

Add this to the more proprietary-than-others semi-lock-in of the Dgraph database and I think I’ve lost confidence that this is the right choice of team and technology to hitch my wagons to at this moment. Turns out the risk is higher than anticipated. And I’m unable to easily accomplish an important piece of my app. I do like many other things about Slash, but I guess I have to reexamine and reconsider options…

Thanks everyone for the help

I think what you are seeing here is the limitation of GraphQL. If you study this rabbit trail a little bit more, you will find that what you want is supported 100% in DQL with var blocks. So why is it not supported yet in Dgraph’s generated GraphQL API—because the GraphQL spec does not support the idea of var blocks. hence one of the reasons that Dgraph had to come up with DQL in the first place. I wrote up an idea of how to get Dgraph to generate var blocks in DQL for a given GraphQL filter input. This does become a quite complex rewrite process though which stands to reason why it has not been built out yet.

I have done what you are asking for in pure GraphQL, but it involves at least 2 separate query operations. I will be demoing this on Dgraph Day.

Thanks @amaster507. Actually I’m comparing Slash with other (SQL-based) GraphQL API backend as a service competitors that have this super easy node/edge relationship querying, filtering, sorting, mutations, subscriptions out of the box. In fact, it is the core beauty of GraphQL and what has drawn me to it.

Which is why it’s doubly perplexing to me that a GraphQL API services hooked up to graph db is seemingly so far behind the curve vs competitor GraphQL APIs based on SQL-based databases (MySQl, Postgres…). Again, it makes me wonder the benefit of a graph db vs simply going the route of a tried and true SQL db + GraphQL.

For example, one of those queries in the original post was taken from one of those SQL-based GraphQL API competitors:

query CorrectDataQuery {
  NodeXList(filter: {nodey: {every: {id: {equals: "456"}}}}) {
    items {
      id
      name
    }
  }
}

And you could, of course, add multiple and/or filters, sorts for nested nodes & edges beyond just that NodeY one.

I don’t mean to pile on, but I know the Dgraph team loves feedback – so if it’s helpful, a little more (small stuff) GraphQL API feedback:

GetX and QueryX Queries Are Redundant?
It’s confusing to me why you have getNodeX and queryNodeX as separate queries, where the only difference seems to be that getNodeX only allows you to filter on ID while queryNodeX lets you filter on ID and other filters. Is this some kind of non-breaking legacy thing perhaps?

Inability To Get Count Within a Content Query?
Maybe I’m being dumb here, but unless I’m mistaken, I’m unable to get “count” kind of aggregate data within a normal, nested queryNodeX query? I have to instead use a separate Aggregate query? So to get a count of NodeX’s related NodeY’s (and Node(n)'s…), I’d have to run 2 queries (times the number of nested nodes I’m querying)? One for the (first 10, for example) content and another for the full count(s)? Though maybe I’m missing something here.

Expected (and possibly what you see in competitors if I recall?) is that an aggregate “count” is available along with the other “name”, “description” etc fields/properties/predicates within any query.

the getNodeX ensures only a single node is returned while queryNodeX may return multiple. It would be similar to a GraphQL API with queries me and users where me returns just a single user, and users returns a list of users.

This was how it was designed. There is an RFC floating around here somewhere when counts was added.

Thanks @amaster.

We have started working on RFC for it and currently looking at the similar requests and ways to best support them in GraphQL. As we have graph database behind the GraphQL which already have support for nested filters, we just need to correctly map them in GraphQL.

This can take some time because we need to efficiently rewrite the GraphQL query to DQL for every possible scenarios. Our target is to include this in 21.07 (releasing in July). Once we have this added in GraphQL it will be fast and efficient than other options which have SQL or any other db to run GraphQL query because on those db’s they need to perform the join operation to get nested result which is very costly.
Thanks for your detailed feedback .

1 Like

Can anyone help me do what seems like the simplest query - and to do it in the current version of Dgraph Slash before the updated remodel mentioned above is released?

Schema:

type Object  {
	id: ID!
	name: String @search(by:[fulltext])
	anotherObject: AnotherObject!
}
type AnotherObject  {
	id: ID!
	name: String @search(by:[fulltext])
}

Intended query: Return all Objects where anotherObject = “0x123” and name = “Some search term string”?

Seems that Dgraph Slash is capable of querying Objects where “name = “Some search term string”” but not where "anotherObject = “0x123"” ??

Surely, there must be a simple way to query based on values of ALL predicates/relationships (terminology)? of an Object in the current form of Slash, yeah? This is the equivalent to being able to search in the SQL world the value of ANY column, yeah?

In other words, there MUST be a way, yeah?

Thanks.

Yes there is a way, but maybe not a way that is easy to do right now. There are two approaches.

  1. Use DQL and var blocks
  • pros: the syntax is not that far different from GraphQL and the possibilities are limitless
  • cons: It ignores any and all @auth rules from the GraphQL schema and it does not map nicely to GraphQL API unless you write up a bunch of custom DQL queries for everywhere you want this at.
  1. Use multiple GraphQL queries in separate requests
  • pros: honors @auth rules, no additional custom DQL queries to write
  • cons: application layer has to be a tad bit smarter and at least two requests are needed for a somewhat simple query

^ Both of these I have detailed in other posts but don’t have the time at the moment to go dig them up.

Bonus, I guess there is also a third way that involves cascade but it might put a hurtin’ on your instance for performance depending on your query, structure and amount of data. Cascade is a post-query pre-response action so it still queries all but limits the response. Here is a quick mockup of that with your query above:

{
  queryObject(filter: { name: { eq: "Some search term string" } }) @cascade(fields: ["anotherObject"]) {
    id
    name
    anotherObject(filter: { id: ["0x123"] }) @cascade(fields: ["id"]) {
      id
      name
    }
  }
}

The only way I can think of to pull this simple query off in current Dgraph Slash is to change schema like below:

type Object  {
	id: ID!
	name: String @search(by:[fulltext])
	anotherObject: AnotherObject! @hasInverse(field:object)
}
type AnotherObject  {
	id: ID!
	name: String @search(by:[fulltext])
	object: Object @hasInverse(field:anotherObject)
}

And then query AnotherObject where ID = 0x123 and return related Objects filtered by Name = "Some search term string.” I’m assuming this may be possible.

And I’ll have to figure out how to retroactively fix all my current data: how to relate all these existing Objects & AnotherObjects efficiently, especially since I can’t currently query based on the info I need to filter them! Ha.

And I’m not sure exactly how Badger structures data, but it seems like this (otherwise not needed?) inverse requirement may effectively double the size of my data? (But maybe improve search performance?)

Is there’s another way? Thanks.

Thanks a lot. Will investigate.

Not doubling data just creating two posting lists of ids that the GraphQL API manages to keep in sync for mutations.

A quick test seems to show the @cascade directive approach you mentioned @amaster507 working. Though, yeah, I’m not sure how to judge the real performance implications as I’m not clear the architecture of how it does what it does.

I may use it temporarily or I may go through the daunting task of the schema change.

Thanks for your help @amaster507 .

If you look at the RFC: Nested Filters in GraphQL in progress, they will be using a cascade method as well instead of var blocks last time I reviewed it. So… there’s that too. Use it until it hurts I guess. Others have found out how painful something like cascade can be if not done correctly:

Got it. Thanks @amaster507.

1 Like