Filter on non-scalar fields or relations (Types)

Consider the schema

type User {
  id: ID!
  pets: [Badger]
}
type Badger {
  id: ID!
  owner: User! @hasInverse(field: pets)
  age: Int! @search
}

Dgraph generates CRUD schema for the types with filter only on scalar types. What if I want to query all users who own boomer badgers (age > 40 years)? It would be great if I could just query

queryUser(filter: {pets: {age: {gt: 40}}}) {
  id
  pets {id}
}
3 Likes

It’s there though!

Checkout documentation: Filtering List

You can also filter nested objects while querying for a list of objects.

query {
  queryAuthor(filter: {
    name: {
      anyofterms: "Lee"
    }
  }) {
    name
    posts(filter: { // Nested filter
      score: {
        gt: 10
      },
      and: {
        completed: true
      }
    }) {
      title
      text
      datePublished
    }
  }
}

So maybe this should solve ur issue.

queryUser() {
  id
  pets(filter: {
      age: {gt: 40}
  }) {
    id
    name
  }
}

N.B. You can go as deep as u want with this, as this is way scalable than a single line approach!

P.S. Typing in mobile, not sure of outcome.

1 Like

@abhijit-kar the solution you mentioned put a filter on a particular field based on scalar fields of that type. in other words, it will return all users and will filter the field ‘pets’ with only the ones above age 40.
It won’t return only users who own pets above age 40.

2 Likes

How about using @cascade directive to remove unnecessary users from the result who don’t have pets aged over 40?

Checkout:
https://dgraph.io/docs/graphql/queries/cascade/#sidebar


e.g.

queryUser @cascade {
  id
  pets(filter: {
      age: {gt: 40}
  }) {
    id
    name
  }
}
2 Likes

@abhijit-kar, I thought of that. But it’s just a workaround. Better feature is to use it in the way I described in the beginning along with already developed filters of ‘and’, ‘or’, ‘not’.
That will make it useful to make queries where I want users who have boomer badgers, but not who are about to die (may be age < 80?) and may be a mix of other criteria depending on other fields. Prisma / graphcool implemented it since the earlier days. I am wondering if this is something that dgraph team is keen on implementing. Anyways, I am eagerly looking forward to feature parity between DQL and GraphQL

1 Like

We ran into this. Want to filer like DQL does with combining var blocks. GrqphQL spec and generated filters can do the depth of what we needed. So we devised a plan to filter using a custom query backed by some predefined DQL options (stored in the database) and then provide back a list of IDs that suffice the filter requested. This is the only solution that we have been able to find.

Think of the complexity of generating a multi-level deep filter object for a type that is linked to dozens of other types. And that is just one layer deep. I think my deepest query filter right now is 5 levels deep. That would be on the verge of impossible with a simple graphql input object. Not to even mention the problem with creating an endless loop of filters that down several depths point back to itself. I don’t know if there is a graphql spec compliant solution at this time. I have tried thinking of how a filter directive might solve the problem, but I think it just makes matters more complicated. Cascade with parameters is the best I think we will get from graphql for a while.

It is a problem though that needs a solution developed if possible.

2 Likes

Consider updated schema with some Cats thrown in

How do I search for users who either have boomer badgers (age > 40) as pets or are enslaved by cool cats (age < 20) using @cascade directive?
Similar to the query below

queryUser(filter: {or: [{pets: {age: {gt: 40}}}, {owner: {age: {lt: 20}]}) {
  id
  pets {id}
}

If it was ‘and’ filter I could think of using @cascade like this

{
  queryUser @cascade {
    id
    pets(filter: {age: {gt: 40}}) {
      id
    }
    owner(filter: {age: {lt: 20}}) {
      id
    }
  }
}

Even for a ‘not’ filter condition I can write the inverse of the filter condition, but I can’t get my head around using ‘or’

Also @amaster507, could you please elaborate the first paragraph with an example. I am really keen on understanding DQL more wrt GraphQL. Thanks a lot for your responses @amaster507 & @abhijit-kar

1 Like

I believe parameterized cascade can help you.

It’s added to master, I have no idea if it’s in Slash yet.

Checkout: Parameterized Cascade

Currently @cascade directive does not take any parameters. This means that, for a node, all its children in the sub-graph must be present for it to be part of the response. The directive is implied at all levels from the level it has been declared.
This is too strict for use-cases where @cascade needs to be done only on some field/children and not all.

P.S. I don’t know if, it will be equivalent of OR logic! :sweat_smile:

No, parameterized cascade does not quite fit the bill here. It would for some use cases but not this one. The general use case is combined filters at multiple levels. Here is my related topic when I was learning about this restriction:

I don’t have the time for a full ellaboration, and it does impose some additional challenges such as DQL does not honor any @auth directive rules in GQL. But the overall idea is this:

  • Predefine DQL queries that will be alllowed to use as filters. I save these as FilterOptions type in the DB. I use some place holders for variables and the return variables to enable grouping of these filters using the AND, OR, NOT logic allowed in DQL @filter.
  • I added a type Filter that holds the subFilters, joining conjunction, variable values defined by the user and the related FilterOption defined above.
  • The subFilters connects to more Filters so a AND filter can link one or more subFilters together.
  • I wrote a lambda function that accepts a few parameters and the JWT if available. The main parameters it needs though is which Filter to use and what type to connect it back to.
  • The lambda function generates a full DQL query grouping different variable blocks together and how they link back to the requested type. The final line of the script returns a set of uids.
  • I then threw this lambda function into a custom query named useFilter and pass the JWT and define the variables needed the the remote type produced.
  • Then in the UI I make a call to the useFilter query using graphql and get a list of IDs that match the filter
  • Then to conclude this whole hot mess, I pass those IDs received from useFilter to my actual graphql request that gets the filtered data to display.

Maybe this will make it’s way into a full write up benefiting the community sometime later.

1 Like

Thanks for suggesting this @yodaheis.
We are accepting this as a feature request. Can’t give any ETA at the moment, but this will probably be worked upon in the next quarter.

2 Likes

@abhimanyusinghgaur, Thank you for accepting this! We are currently experiencing a hard time with the performance of my above mentioned work around. The main problem being the amount of data we are working with. If the filter is too broad then there could be a list of over 50K ids that match the filter. This list of ids is then sent to the graphql query where auth gets applied and the end result set is only a few hundred. We discussed today about possibly adding auth on our DQL filtering level but that would mean managing GraphQL auth and DQL auth or writing a script that takes GraphQL auth and converts it to DQL auth. That is way more work than what we want to do. I believe if this gets done it will solve our problems.

I am looking forward to a RFC on how exactly this would work.

I would like to see it work as the OP mentioned:

input BadgerFilter {
	id: [ID!]
	owner: UserFilter
	age: IntFilter
	and: BadgerFilter
	or: BadgerFilter
	not: BadgerFilter
}

input UserFilter {
	id: [ID!]
	pets: BadgerFilter
	and: UserFilter # this would then be useful, not currently generated
	or: UserFilter # this would then be useful, not currently generated
	not: UserFilter
}
1 Like

Hi!
Are there any news on this issue?
I feel like this is an super-important and often-used feature. We would use it heavily in our software.

2 Likes

Hm, can’t edit my previous post.

Just wanted to re-express my interest in this as I’m stuck once again without this functionality.
What will be a rough ETA for this? Also, will we be able to filter for child-child-child-child… properties?

Hi @maaft,

Thanks for expressing interest in this feature.

This feature is part of the features being discussed to be implemented in 21.03 release. It will most probably be part of 21.03 Dgraph release.

Can you share your specific use case or is it the same as OP ?. If your use case is any different, it will help us while making an RFC.

@rajas sure, I will just give you the current use-case where I’m stuck right now:

type FooCollection {
   foos: [Foo!]!
}

type Foo {
   producedBy: FooProducer!
}

type FooProducer {
  id: String! @id
}

Here I want to query all FooCollection's that don’t have at least one Foo that is produced by FooProducer with id: "123"

So, given these Collections:

{
   foos: [
      {
         producedBy: {
            id: "1234"
         }
      }
   ]
}
{
   foos: [
      {
         producedBy: {
            id: "1234"
         }
      },
      {
         producedBy: {
            id: "123"
         }
      }
   ]
}

I only want to query the first Collection.

Of course you could argue that I could do this on the client. But when the number of collections grows (and it does in my case) I have huge traffic.

Edit:
Hi @abhimanyusinghgaur! You said you will probably work on this last quarter. Is there an update or RFC for this?

2 Likes

Hi @maaft,

This is in our roadmap for 21.03, and thanks to @amaster507 we have some initial thoughts on this here: Proposal Nested Object Filters for GraphQL rewritten as var blocks in DQL

Feel free to share any ideas you may have on that post.

3 Likes

Hi @abhimanyusinghgaur

This would be totally awesome. Currently we have tons of extra fields on the parents that mirror the child fields. But this is very inconvenient and suddenly you have multiple sources of truth which you need to maintain.

Can you give me a notification when this is ready on master? Can’t wait to try it out.

Sure, we will update Anthony’s post when this is ready.

1 Like

So, how is it going? Can we expect this in the 21.03 release?

No, this is not going to be part of 21.03. It should be part of next release. We are currently working on designing this feature and you should soon expect an RFC.

3 Likes