No cursor-based pagination and offset-based pagination is slow

bug

(Roman Sharkov) #1

Working on the pagination feature for the Dgraph tech-demo I discovered two major flaws.

Linked issues: #3472 and #3473

Initial database state: there’s 83.719 post nodes in Dgraph:


Post.id is indexed with exact. Getting the last 6 posts ordered by Post.id returns [0x78b52, 0x68678, 0x59722, 0x726ff, 0x65dbd, 0x78324]

1. first and after don’t work together:

I failed implementing cursor-based pagination because first and after don’t seem to work in combination. I’d expect this query:

{
  pageAfter(
    func: has(Post.id),
    orderdesc: Post.id,
    first: 3,
    after: 0x59722
  ) {
    uid
    Post.id
    Post.title
  }
}

to return uids: 0x726ff, 0x65dbd, 0x78324 since the predicate is sorted by Post.id but as you can see the actual results ([0x5aefe, 0x73133, 0x74920]) are very different. Is this a bug?!

2. offset based pagination is very slow

Later I tried implementing an offset` based pagination but it turned out to be very slow:

{
  pageAfter(
    func: has(Post.id),
    orderdesc: Post.id,
    first: 3,
    offset: 83716
  ) {
    uid
    Post.id
    Post.title
  }
}

Getting the last 3 posts in the list ordered by Post.id takes 2.5 - 4 seconds which is nuts!

I’ve tried a hash index for Post.id which is slighly faster but still takes 1.5 - 2 seconds!

Dgraph seems to be doing “full-table-scans” all the time even though Post.id is indexed and unique (which the database doesn’t know since there’s no unique constraint).

Is there anyway to get pagination right?! :sweat_smile: