Ask Dgraph Founder Anything

Users
28

I have read in detail your other latest blog post:

It was so mesmerizing. Your engineering mind is exceptional!! :exploding_head:

The optimization improvements are terrific for a database that already touts ingestion speeds keeping up with major databases.

And then I saw the type problem, and you confirmed the problem with indexes. So, what if, types weren’t indexes, they were edges??

I can hear Chewxy correcting me already, that everything is already an edge, it is turtles all the way down. But please hear me out.

So sroar comes along and fixes the posting list problem. Not only with memory consumption but storage on disk, encoding/decoding, transmitting between nodes (Dgraph really uses the same terms for multiple meanings way too often) of the cluster.

Sroar has been optimized for the various intersection and union operations expected from RBs.

So let’s store types as a posting list. Let there be special edges that are types that store a posting list of uids that point to all of the nodes of the type.

Maybe under the hood even beneath DQL this edge would be named something that would be scoped into dgraph only like maybe even dotting notation the dgraph.type out further:

dgraph.type.User: [uid]
dgraph.type.Post: [uid]

And then using these bitmaps quickly retrieve all nodes of a type very very easily without hitting any indexes anywhere. No need to go through a big index tree of all the types to get a single type, just graph the single edge. This would in theory be the opposite of:

which uses some kind of reverse index I assume somehow with the has function.

Instead this proposal to fix types would be just like grabbing an edge. You could even internally in the datastore have a root node that then fans out to the types that fans out to the nodes. In DQL pseudo code:

{
  var(uid("0x1")) { # 0x1 is already reserved for GraphQL stuff AFAIK
    User as dgraph.type.User
  }
  queryUser(func: uid(User)) {
    id: uid
    username: User.username
  }
}

Fundamentally, these triples:

<0x2a> <dgraph.type> "User" .
<0x2b> <dgraph.type> "Post" .

would be rewritten to:

<0x1> <dgraph.type.User> <0x2a> .
<0x1> <dgraph.type.Post> <0x2b> .
1 Like