Keyset pagination with ordering using a deterministic approach

Users
,
#1

As the title suggests, I am looking for a way to perform keyset pagination like in PostgreSQL. However, I believe that the uid is an arbitrary number and is non-deterministic of order. Then, I found this discussion which builds upon using the lt function to verify what comes before said node. However, this skips certain datasets with exactly the same values as the paginated field, so I thought of a solution by using:

asc: not(uid($uid)) and ge(field, value)
or
desc: not(uid($uid)) and le(field, value)

Is this a good way for doing pagination? Are there supposed risks or downsides to using this “hack”? Or are there other ways of pagination with ordering? Thanks in advance!

#2

Typically, to order with pagination, you would use window paging in Dgraph (first: n, offset: n).

#3

Hi David! Thank you for the reply. I agree that using first and offset would be the simplest solution, but as far as I know, it may not be scalable with larger datasets because of how limits work (refer to this GitHub issue). I’m actually outdated with this issue. Has pagination performance with first/offset already been fixed?

#4

While I agree that window paging isn’t ideal in this situation, I haven’t had performance issues using it to paginate through several million records.

#5

Would it be all right to ask approximately how long the response would be for larger offset sizes in your case?

#6

I suggest generating some sample data that represents your own dataset & measuring response times. It’s very query / data / hardware / configuration dependent.

1 Like
#7

Thank you for the suggestion. I’ll try to check on the scaling properties.