Streaming response


I failed to find an answer to my question, so I am here.

I wonder if dgraph can stream response. The need is we are planning to use dgraph for catalog service (users, groups, etc) and some responses can be a little too bulky. Something like

get all users who are members of group ALL or one of its child group (child of child, child of child of child, etc)

A catalog may have millions of users and returning them in one big group is not the right thing to do. Does dgraph have this kind of functionality?

No, there’s no such functionality.
You can do instead, is pagination.

1 Like

Well, better than nothing yet proper streaming seems to be a better match for these kind of tasks. Hope this will work fast enough, although I am highly suspicious about it: sorting means prior fetching, i.e. the database get the full response first, then sort it, than return Nth, (N+1)th, …, Mth elements. Each time for each page and the number of collected elements can be quite large at that.

you can wrapper a “stream” with pagination.
every time you pull your stream, you get one page as your stream element.

Pagination requires ordering, ordering requires all data to be collected first. This means costly traversal procedure on each request.