V21.03: After pagination+cascade change, queries are too slow to finish

Issues Dgraph
,
22

Hmm, I wonder if a double pagination would remedy the problem and be the best of both worlds. Pagination before cascade, and pagination after cascade. I am not sure how to easily convey this but maybe a query would look like:

{
  var_a as var(func: eq(<qa.type>, "Device"), limit: 1000, first: 12) {
    ...

limit being the new suggestion. Then you could add a limit pre the cascade and a pagination post the cascade. This still would not solve all use cases but would be better then iterating over 6M nodes.

To define limit: To limit the galaxy in queries using pagination with cascade.

1 Like
23

Yea thats very much like my query I am settling with. Sorta like this:
edit: (to change my wording:) yea that sounds better than what I have to do, which is like the following

query q($prelimit: int = 1000, $limit: int = 10) {
  var_a as var(func: eq(<qa.type>, "Device"), first: $prelimit) {
    var_b as <qa.has_object> @filter(eq(<qa.type>, "Object"))  (first: $prelimit) {
      var_c as <qa.has_indicator> @filter(eq(<qa.type>, "Indicator")) (first: $prelimit)
    }
  }
  a(func: uid(var_a), first: $limit) @cascade {
    ...commonFields
    b: <qa.has_object> @filter(uid(var_b)) (first: $limit) {
      ...commonFields
      c: <qa.has_indicator> @filter(uid(var_c))  (first: $limit) {
        ...commonFields
      }
    }
  }
}

and just guess how many you are going to filter out with cascade… which I just have set to the same thing at this point because there is no way for me to guess how much will be filtered.

2 Likes
24

@iluminae Have you tried version dgraph/dgraph:v21.03-slash? I had some issues with 21.03.0 in which some queries never returned results.

1 Like
25

@matthewmcneely no, I have not - but the issue is a fundamental design decision (to remove all pagination for any query that includes any level of cascade until the end of processing, introduced by this pr) as opposed to a unknown bug.

1 Like