Pagination in Dgraph

We have support for First N result currently. I think adding an After X should be good to attain the pagination functionality. Eg:

friends(after: String, first: Int)

We could also support Before and last which are similar to After and First:
friends(after: String, first: Int, before: String, last: Int)
To enable support for reverse iteration.

Please leave your comments on other requirements, thoughts, suggestions regarding it in this thread :slight_smile:

Shouldn’t after be an int as well? And is after an index or a UID? Currently, we use a negative number for after to show last. Maybe avoiding the negative and using the word last might be better.

before might or might not make sense depending upon what it stands for – index or UID.

It depends. We could have after as an index or make it synonymous to SQLs Where FIELD > VALUE in which case it’ll be UIDs. Its type will be the type of the field based on which we want to order and retrieve the results .

I feel going with fields (UID or some other field, once we have orderby operation supported) would be better that index.

We’re not close to sort by, so I’m not sure how that might fit here. Also, our values are currently just considered binary blobs, and we haven’t yet tried to interpret them. So, let’s leave that aside for now.

For this particular task, i.e. pagination for UIDs; what do you suggest?

We could go with the UIDs itself instead of indexs.

1st Request: friends(after:0, first: 20)
2nd Request: friends(after: < last uid of prev result >, first: 20)

This is recommended as it would avoid repeat of some results in case a new entry is added. https://github.com/facebook/graphql/issues/4

1 Like

I’d recommend using LIMIT and OFFSET for pagination. They are standard across databases, developers are already aware of these terminologies, and work well with Filter and Order clauses.

Edit: Sticking to a int for OFFSET (instead of <last uid of the previous result>) would also allow to directly go to the nth page results without going to (n-1)th page results.

True. It does make it easier for developers to use. But

You could also imagine a much simpler pagination
model that just does offset, limit. There are
some issues with this approach (for example, if the front of
the list changes often, or when deep
into a very long list), but it's simplicity is compelling.

As is discussed in the github issue, it has some drawbacks. Especially if many writes are made to the database. I think Limit-Offset is more suited for cases where the writes are rare?

Edit: We could have AfterN which would take the offset and warn the developer about this potential issue. As in some cases it is helpful, like going directly to the Nth page

2 Likes

True. Both approaches have their use cases and drawbacks

  1. After/Before - Pagination has to be linear.
  2. Limit/Offset - Jumping between result pages randomly is not efficient.

As noted by Lee, the model is selected by FB and not GraphQL, and it’d comes down to the design choice. My advice would be to also look into which implementation would be more efficient for Dgraph’s usecase. It is perfectly reasonable to make decision choice based on performance criteria and limit functionality.

1 Like

For Dgraph, basing this on uid would be better. Because we can do a binary search for that uid.

Also, it avoids the problems that Ashwin mentioned, and is a cleaner approach design wise.

2 Likes

Sure. As long as it is efficient :). Don’t agree with the other approach being any less cleaner though! :stuck_out_tongue:

@ashwin95r, Are you planning to support afterN alongside after? I feel it is a significant functionality to jump to a random page – so if you can also support it, It’d be awesome.

Yeah, I’m just starting to work on this and was getting some background of the general practice and the views of the community. Hopefully, we support both the ways of accessing :slight_smile:

1 Like

Appreciated. I also learnt about the drawbacks of LIMIT/OFFSET approach (all these years, it never occurred to me :frowning: ), so it was a fruitful discussion for me. :slight_smile:

1 Like

I’m not convinced we should do that. Just like Go language, one of the principles of Dgraph is to have a minimalist core, and that means deciding the one best way to do things, and decreasing the surface area of features.

When we think about pagination, after makes sense. And it provides the developer everything they need to build pagination. afterN seems like another way to achieve the same thing, but with the caveats where there’d be repetitions of the same data. I don’t see a strong use case where we’d need to randomly start from say, page 10.

Also, note that graphs don’t really work the same way as SQL does. Doing pagination at an intermediate step means your final result set (after a few joins or intersections) would be incomplete. So, usage of this pagination technique needs to be done in awareness of that limitation.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.