Expand with predicate filters

Experience Report for Feature Request

What you wanted to do

The use case is building a query from url parameters that can achieve N level deep with predicates that reference other types. The number of types, levels, filters, etc. are given at runtime and a query to Dgraph is constructed, executed, and the data returned as a search result.

I’d like to be able to use expand(all) and include a filter to one/many predicates at the same level. The “repeated subgraph” error is given when trying this approach.

{ 
  var(func: type("orderstest__type0")) { 
    T0 AS uid 
  } 
  var(func: type("orderstest__type1")) {
    T1 AS uid
  } 
  t as total(func: uid(T0,T1)) 
    @filter(((eq(caseowner,"test@gmail.com") AND has(caseowner)))) {
      count(uid) 
  }
 
  query(func: uid(t), orderasc: createdat,  first: 10, offset: 0) { 
    expand(_all_)
    task @filter((eq(assignee,"test@gmail.com") AND has(assignee))) { 
      expand(_all_)
      taskcomment @filter((eq(comments,"test comments") AND has(comments))) { 
        expand(_all_)
      }
   }
  } 
}

What you actually did

My workaround is retrieving all predicates for the given types while building the query.

{ 
  var(func: type("orderstest__type0")) { 
    T0 AS uid 
  } 
  var(func: type("orderstest__type1")) {
    T1 AS uid
  } 
  t as total(func: uid(T0,T1)) 
    @filter(((eq(caseowner,"test@gmail.com") AND has(caseowner)))) {
      count(uid) 
  } 

  query(func: uid(t), orderasc: createdat,  first: 10, offset: 0) { 
    caseowner 
    createdat
    notes
    priority
    reservationnumber
    space
    status
    task @filter((eq(assignee,"test@gmail.com") AND has(assignee))) { 
      assignee 
      space
      testdatetime
      title
      taskcomment @filter((eq(comments,"test comments") AND has(comments))) { 
        expand(_all_)
      }
    }
  } 
}

Why that wasn’t great, with examples

This is not ideal because there are types with 100+ predicates, and queries to many types could yield an even higher number. This requires a call to cache/network to retrieve the predicates for each level. We need additional code to make sure we retrieve all the unique predicates except the ones we are filtering on. We also would ideally like to retrieve the data in this format from the “query” block.

Any external references to support your case

These are similar requests:

1 Like

Have you tried to use Types instead of _all_?

e.g:

query(func: uid(t), orderasc: createdat,  first: 10, offset: 0) { 
    expand(Somthing)
    task @filter((eq(assignee,"test@gmail.com") AND has(assignee))) { 
      expand(Assignee)
      taskcomment @filter((eq(comments,"test comments") AND has(comments))) { 
        expand(Comments)
      }
   }
  } 

If you use the DQL Types you won’t see and “repeated subgraph” error.
You can also try to create a “dummy type”. A type that you will only use for expanding purposes. That’s a “hacky way” of doing it, but works fine. Create a type with all possible predicates you have. And then use it on your query.

e.g:

query(func: uid(t), orderasc: createdat,  first: 10, offset: 0) { 
    expand(Dummy)
    task @filter((eq(assignee,"test@gmail.com") AND has(assignee))) { 
      expand(Dummy)
      taskcomment @filter((eq(comments,"test comments") AND has(comments))) { 
        expand(Dummy)
      }
   }
  } 

BTW, That tip does not invalidate this request.

Cheers.

@MichelDiz @dougdoenges, I have created a smaller dataset and a potential query after the changes in this PR: feat(Query): Allow filters in expand(_all_) queries on predicates pointing to nodes by anurags92 · Pull Request #6752 · dgraph-io/dgraph · GitHub. Please take a look and suggest if this matches your expectations with this feature.

For the following dataset and query:
Schema + Data:

    name: string @index(hash) .
    age: string .
    animal: string .
    hasPet: [uid] .
    lives: [uid] @reverse .
    type TypeName {
        name: string
        age: string
        hasPet: [uid]
    }
    type Pet {
        name: string
        animal: string
        age: string
        lives: [uid]
    }
    type Place {
        name: string
        lives: [uid]
    }

        <1> <name> "Anurag" .
        <1> <age> "10" .
        <1> <dgraph.type> "TypeName" .

        <2> <name> "Brad" .
        <2> <age> "20" .
        <2> <dgraph.type> "TypeName" .

        <3> <name> "Charlie" .
        <3> <age> "12" .
        <3> <dgraph.type> "TypeName" .

        <4> <name> "Tommy" .
        <4> <animal> "Dog" .
        <4> <age> "4" .
        <4> <dgraph.type> "Pet" .

        <5> <name> "Meow" .
        <5> <animal> "Cat" .
        <5> <age> "6" .
        <5> <dgraph.type> "Pet" .

        <6> <name> "Delhi" .
        <6> <dgraph.type> "Place" .

        <7> <name> "Mumbai" .
        <7> <dgraph.type> "Place" .

        <8> <name> "Kolkata" .
        <8> <dgraph.type> "Place" .


        <1> <hasPet> <4> .
        <2> <hasPet> <5> .
        <1> <lives> <6> .
        <2> <lives> <7> .
        <3> <lives> <7> .

        <4> <lives> <6> .
        <5> <lives> <7> .

Query:

    U as q1(func: type("TypeName"))

    q2(func: uid(U) ){
        expand(_all_)
        hasPet @filter(eq(animal, "Dog")){
            expand(_all_)
            lives @filter(eq(name, "Delhi")) {
                expand(_all_)
                ~lives{
                    expand(_all_)
                }

            }
        }
    }

Result:

  "q2": [
    {
      "name": "Anurag",
      "age": "10",
      "hasPet": [
        {
          "animal": "Dog",
          "age": "4",
          "name": "Tommy",
          "lives": [
            {
              "name": "Delhi",
              "~lives": [
                {
                  "name": "Anurag",
                  "age": "10"
                },
                {
                  "name": "Tommy",
                  "age": "4",
                  "animal": "Dog"
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "name": "Brad",
      "age": "20"
    },
    {
      "name": "Charlie",
      "age": "12"
    }
  ]

Note: Queries like:

   q2(func: uid(U) ){
        expand(_all_)
        name
    }

Note: specifying scalar predicates in presence of expand(all) is redundant and not supported.

  1. Thanks to @dmai for finding the below edge case.
q2(func: uid(U) ){
  expand(_all_)
    hasPet {
        expand(_all_)
    }
}

The above is a valid use case and should be supported. Ticket: https://dgraph.atlassian.net/browse/DGRAPH-2605

2 Likes

This is it, thanks for working on it!

1 Like

Awesome, this is exactly what we were looking for!

We are thinking of making the change now to use this new feature- has this been released yet or been planned for release? Still no rush on our end, we have a workaround, but this will let us build cleaner queries. Thanks!

1 Like

This is planned for release in 20.11 which is just around the corner. It is present in master if you want to test out the feature.

Want to highlight the below query (without filters) wouldn’t work. But your specific use case where you expand nodes with some filters would work fine.

Great thanks for the info!

Quick question, would this be valid? In our use case, we would be expanding to a given depth but also filtering on edges to the same depth.

    U as q1(func: type("TypeName"))

    q2(func: uid(U) ){
        expand(_all_)
        hasPet @filter(eq(animal, "Dog")){
            expand(_all_) {
                expand(_all_)
            }
            lives @filter(eq(name, "Delhi")) {
                expand(_all_)
                ~lives{
                    expand(_all_)
                }

            }
        }
    }

filters on true edges(as your example) are valid. What would not be valid is

expand(_all_) @filter(eq(animal, "Dog")) {