Identify query processing steps in nested queries based on parsed function

Users Dgraph
1

I want to understand how does query processing happen once the query has been parsed and we have identified main and nested functions and their types. [Query parsing and function identification happens in parseFunction here]

Let’s discuss this with examples. Suppose my query looks like:

nameVar as var(func: eq(name, "Lucas Lee")){}

q(func: eq(name, "Jane Doe") ){
    name
    uid
    friend @filter( uid(nameVar) ){
        name
        uid
        memberOf {
            uid
            Party
        }
    }
}

This is broken into three top level functions (2 eq and 1 uid) viz.

1.
{
  "Attr": "name",
  "Lang": "",
  "Name": "eq",
  "Args": [
    {
      "Value": "Lucas Lee",
      "IsValueVar": false,
      "IsGraphQLVar": false,
      "IsUIDVar": false
    }
  ],
  "UID": null,
  "NeedsVar": null,
  "IsCount": false,
  "IsValueVar": false,
  "IsLenVar": false,
  "IsUIDVar": false
}

2.
{
  "Attr": "name",
  "Lang": "",
  "Name": "eq",
  "Args": [
    {
      "Value": "Jane Doe",
      "IsValueVar": false,
      "IsGraphQLVar": false,
      "IsUIDVar": false
    }
  ],
  "UID": null,
  "NeedsVar": null,
  "IsCount": false,
  "IsValueVar": false,
  "IsLenVar": false,
  "IsUIDVar": false
}

3.
{
  "Attr": "",
  "Lang": "",
  "Name": "uid",
  "Args": null,
  "UID": null,
  "NeedsVar": [
    {
      "Name": "nameVar",
      "Typ": 1
    }
  ],
  "IsCount": false,
  "IsValueVar": false,
  "IsLenVar": false,
  "IsUIDVar": false
}

None of the above has any nested functions and these independent functions are evaluated separately and combined via some mechanism. Can you suggest if that is a correct understanding and also point me where does this happen in code?

Now let’s look at an example with nested functions:

var(func: uid(0x0)){
    nameVar as math(5190001) #decimal representation of uid - 0x4f3171
}
q(func: eq(name, "Jane Doe") ){
    name
    uid
    friend @filter( uid_in(memberOf, val(nameVar) )){
        name
        uid
        memberOf {
            uid
            Party
        }
    }
}

This gives following three functions:

1.
{
  "Attr": "",
  "Lang": "",
  "Name": "uid",
  "Args": null,
  "UID": null,
  "NeedsVar": null,
  "IsCount": false,
  "IsValueVar": false,
  "IsLenVar": false,
  "IsUIDVar": false
}

2.
{
  "Attr": "name",
  "Lang": "",
  "Name": "eq",
  "Args": [
    {
      "Value": "Jane Doe",
      "IsValueVar": false,
      "IsGraphQLVar": false,
      "IsUIDVar": false
    }
  ],
  "UID": null,
  "NeedsVar": null,
  "IsCount": false,
  "IsValueVar": false,
  "IsLenVar": false,
  "IsUIDVar": false
}

3.
{
  "Attr": "memberOf",
  "Lang": "",
  "Name": "uid_in",
  "Args": [
    {
      "Value": "nameVar",
      "IsValueVar": true,
      "IsGraphQLVar": false,
      "IsUIDVar": false
    }
  ],
  "UID": null,
  "NeedsVar": [
    {
      "Name": "nameVar",
      "Typ": 2
    }
  ],
  "IsCount": false,
  "IsValueVar": false,
  "IsLenVar": false,
  "IsUIDVar": false
}

In the above functions 1 & 2 are as expected but function 3 has a field NeedsVar which captures the variable required inside the nested function. What is the order of evaluation in this case and where can I see it in code? Essentially how does NeedsVar govern evaluation?

1 Like
2

I am not sure about the specifics of NeedsVar but I can answer on the rest.

The query processing happens in three main places.

  1. The query package.
  2. Some files in the worker package (mainly task.go).
  3. The file server.go in the edgraph package.

Most of the query processing happens inside the query package. task.go contains methods to retrieve the data. In a nutshell, the query processing happens as follows:

  1. The parsed query is converted into a list of subgraphs.
  2. A subgraph can have children as well. You can think of the subgraphs as mirroring the structure of the query. If you send a single query, there will be a subgraph representing the root node and each of the fields of the sub-query is a child of this subgraph.
  3. This tree of subgraphs is evaluated, starting at the top and going recursively to the children. As the query is processed, the results of the query are inserted into the appropriate subgraph.
  4. At the end of the evaluation, the subgraphs are converted into a json object, which can also be thought of as a tree.

Take this query for example.

{
names(func: has(name}) {
name
address
}
}

This query corresponds to a list with a single subgraph (corresponding to query names)
The root of this subgraph tree would correspond to the has(name) functions since that is used to generate the nodes that are traversed initially. This subgraph in turn has two subgraphs, one for the data of the “name” predicate and the other for the data of the “address” predicate.

There’s a lot of code in the query package that does a lot of different things so I can’t really be more specific right now. Hopefully this gives you a general idea of where to look.

1 Like