IMPORTANT: new behavior for expand, deprecation of _forward_ and _reverse_


(Francesc Campoy) #1

Hi everyone!

Now that types have made it into Dgraph with v1.1.0, it is time to reconsider how expand works.

In v1.2.0 we will update the behavior of expand to the following specs:

  • expand(_all_) expands all of the predicates found in all of the types associated with any of the nodes at the level where expand appears. This includes any predicates appearing in those types, which could be reversed edges.

  • expand(_predicate_) expands all of the predicates for all of the nodes at the level where expand appears. This includes any predicate coming out of the nodes, no matter whether any types are associated with those nodes.

  • expand(_forward_) and expand(_reverse_) will be REMOVED FROM THE LANGUAGE, as their interaction with types makes them redundant.

  • expand(typeNameA, typeNameB) will expand all of the predicates in the types given as parameters.

Some examples

Let’s define this schema:

<name>: string @index(exact) .
<parent>: [uid] @reverse .
<lives_in>: string .

type Person {
    name: string
    parent: [uid]
    <~parent>: [uid]
}

The type Person has three fields, corresponding to two predicates defined in the schema plus a reversed predicate. Let’s assume we have performed this mutation:

{
  set {
    _:f <name> "Francesc" .
    _:f <dgraph.type> "Person" .
    _:f <lives_in> "San Francisco" .
    _:f <parent> _:p .
    _:f <parent> _:l .

    _:p <name> "Paco" .
    _:p <dgraph.type> "Person" .

    _:l <name> "Lucia" .
    _:l <dgraph.type> "Person" .
  }
}

Using expand(_all_) on this query will return only the predicates in Person type:

{
  q(func: eq(name, "Francesc")) @recurse(depth:3) {
    expand(_all_)
  }
}

Since we know that all of the nodes in this dataset are of type Person, the query above is equivalent to:

{
  q(func: eq(name, "Francesc")) @recurse(depth:3) {
    expand(Person)
  }
}

Both of the queries above return the same response:

{
  "data": {
    "q": [
      {
        "name": "Francesc",
        "parent": [
          {
            "name": "Paco",
            "~parent": [
              {
                "name": "Francesc"
              }
            ]
          },
          {
            "name": "Lucia",
            "~parent": [
              {
                "name": "Francesc"
              }
            ]
          }
        ]
      }
    ]
  }
}

Notice how even though "Francesc" also has a predicate lives_in it will not appear in the response, as it’s not part of the type Person. Also, notice how ~parent is included in the response, as it’s also part of the type.

On the other hand, expanding on _predicate_ will return all of the predicates associated with the nodes, no matter whether they’re part of the type or not.

{
  q(func: eq(name, "Francesc")) @recurse(depth:3) {
    expand(_predicate_)
  }
}
{
  "data": {
    "q": [
      {
        "name": "Francesc",
        "parent": [
          {
            "name": "Paco",
            "~parent": [
              {
                "name": "Francesc",
                "lives_in": "San Francisco"
              }
            ]
          },
          {
            "name": "Lucia",
            "~parent": [
              {
                "name": "Francesc",
                "lives_in": "San Francisco"
              }
            ]
          }
        ],
        "lives_in": "San Francisco"
      }
    ]
  }
}

Notice how lives_in appears, although it’s not part of the type, and how ~parent appears as reverse predicates will still be fetched.

We need your opinion!

It is your time to tell us whether this will impact your usage of Dgraph so we can consider your needs in our design. Please let us know in this thread (or directly to francesc@dgraph.io) if you have any concerns regarding these changes.

We have other options, such as keeping _forward_ and _reverse_ but basing them on the behavior of _predicate_ rather than _all_. This would lead to not very performant queries, so that’s why we’re considering removing them completely.

Thanks!

:badger:


Upsert with multiple UIDs
(Joel Poulin) #2

The expand by type is a great idea !
I already need it desperately :scream:


(August Hell) #3

This looks interesting and what cames first to my mind, it’s the right evolution of expand for the types.
But one line in the type definition of this example leaves me sceptical:

<~parent>: [uid]

Isn’t this declaring an index as a type? That’s some kind out of the scope of a semantic definition, dont you think?
The following type declaration should be enough:

type Person {
    name: string
    parent: [uid]
}

(Francesc Campoy) #4

Hey @AugustHell,

Thanks for your feedback!

I’d say <~parent>: [uid] is a totally valid field, as <~parent> is a predicate, not an index.

That’s why you can even use it on a query block:

{
  q(func: has(name)) {
    name
    ~parent {
      name
    }
  }
}

We’re also working on a different proposal that would allow you to give a better name to a reverse predicate by passing it as a parameter to the @recurse directive.

You could then write:

<name>: string @index(exact) .
<parent>: [uid] @reverse(child) .

Then you could write the query above as follows:

{
  q(func: has(name)) {
    name
    child {
      name
    }
  }
}

I think it’s time to be able to call your children something better than your ~parents :smile:


(August Hell) #5

Hmmm, the reverse being a predicate not really convincing me.
If so you should be able to store something different, then with parent.
If not, it’s redundance, which is to avoid in data stores.
~parent is storing the same information as parent, just in another order, so you get easier access to the information you want. That’s what an index does.
In the schema it is declared not on first position, as all other types names.
schema {
name
parent
~parent (wont fit here)
}
Instead it is declared at the end and till now not even with its name
parent: [uid] @reverse
the same position like the other indexes
name: string @index(exact)

if you place now in parantheses a name like
: [uid] @reverse(child)
it’s even get more confusing, as there is usually till yet the type of the index.

Defining that schema in go as struct

type Person struct {
    Name string `dgraph:"person @index(exact)"`
    Parent []Person `dgraph:"parent @reverse"`
}

… the reversed part is not even defined as a variable.
How on earth you should know now, that you need it to define separately for the type if you want to have the full usage of it? That’s more something like hidden secrets of an untyped language.

I think declaring the reverse in the type Person { <~parent> } is wrong and not logical. That should be done by the database or the data user needs to handle types and schemas separately while being always aware of, that if he changes the first he could break the other.


(Lee Hui) #6

Would you consider adding the ‘dgraph.type’ of node in the response of expand() ? If you do, i think it’s will be more convenient to recognize the type of node and to deal with it.


(Daniel Mai) made this a banner . It will appear at the top of every page until it is dismissed by the user. #7

(Daniel Mai) removed this banner . It will no longer appear at the top of every page. #8

(Daniel Mai) pinned globally #9

#10

Shouldn’t expand(__all__) include predicates that are not associated with any types too? Semantically, all implies everything. Maybe it would be a better for expand(__type__) to return predicates that belong to all types


(Francesc Campoy) #11

You can do it yourself by also adding <dgraph.type> in the request, or even in the type definition if you prefer.


(Francesc Campoy) #12

~parent is technically not an existing predicate, it’s an index.

We are not proposing to add a new feature here, we’re just proposing a better name for reverse edges.
There’s no change in what information will be stored, just a syntactic help to avoid having field names in a type with awkward names like ~parent when child is more concise.

I sincerely not follow what your concern is, I do not really appreciate your dismissive tone to be honest.
If you’d like to explain why this is “wrong and not logical” please expand on your reasoning.


(Francesc Campoy) #13

We’re considering that option too, thanks for your feedback @san8pai!


(August Hell) #14

Sorry, I apologize, that was not intended. I’m not a native english speaker, nor am I that educated and skilled that you are. I have a huge respect for what you’re doing and am curious on all new I can see or read. I don’t wanted to sound destructive, instead I’d like to bring a constructive point to develop this feature further.

My concern here is the declaration the reverse edge (<~parent>) in the type:

If this reversed edge <~parent> need to be declared this way in the future in types to be expandable, it is questionable, because

  1. it’s not named that way in the schema. There it is noted as @reverse, like an index,
  2. it is by its own not an independent value as the others in the type. It depends on parent, so declaring it in the type it is redundant and that might cause other problems, (like, what happens if someone removes @reverse later in the scheme)
  3. as its not in the schema by name, it’s not in a struct as a variable/member if working with go. So finding a way to programmatically create the type this way is cumbersome.
  4. it is confusing being new with graphql, as it is not logical (in my eyes) to need to define something that looks like an index as a type.

I might be totally wrong and misunderstand it. Maybe the declaration of the @reverse as a type member is not needed and wont effect the results if not done. I just can’t test it yet and dont have the fully insight and understanding to be sure. But if results are affected, I need to go back to see how I can handle it in my go code :roll_eyes:

Again sorry, hope it is more clear now, what my thoughts are.