A low–level serialization form to speed up querying by wire

[badt (Ian P Badtrousers)]

Hello,

I must admit I’m late to the party and I’ve only been researching graphs for the last couple of years, but after a worthwile struggle with SPARQL and RDF ontologies, I’ve now decided to partake in a very interesting project, which involves using dgraph a lot.

We’re trying to build a graph VM with its own bytecode, assembler, sophisticated caching mechanism, etc. with dgraph as the primary I/O. I haven’t measured this, but I can imagine there’s quite a bit of overhead constructing GQL queries in Go (our application), pushing them over the API, and deconstructing them in Go (dgraph) again. Would it be reasonable to straight up import the GQL packages in the application, manually construct the queries, and send them over to a patched dgraph instance by wire as a binary or gob–like low–level construction [according to https://godoc.org/github.com/dgraph-io/dgraph/gql] of the query?

I’m not sure whether if this is going to work, but I’ve got this hypothesis that in graph calculus arbitrary morphisms can be viewed as monads; if that’s true, we should be able to describe any graph computation as a composition of morphisms like we would normally do with monads in functional compositions.

We want to create an assembly language that would have instructions to express both low–level ideas, such as vertices, edges, as well as high–level operations and search algorithms, so the byte–code could be iteratively optimized in pursuit of an optimal computation.

Does any of this make any sense?

Cheers,
Ian

[chewxy]

I’m sorry for the late reply @badt, we as a team at Dgraph need to do better.

To answer your questions:

@badt

Would it be reasonable to straight up import the GQL packages in the application, manually construct the queries, and send them over to a patched dgraph instance by wire as a binary or gob–like low–level construction

Internally dgraph communicates via protocol buffers. In fact, the client libraries all connect to dgraph via protobuf.

@badt

but I’ve got this hypothesis that in graph calculus arbitrary morphisms can be viewed as monads;

But you can generate a monad for most things. Just find three functions that perform the monad rules and make sure you have an identity functor to make it an endofunctor. Bam, monad (see also: free monads, where you can get monads from functor for free). That doesn’t say much.

I’m curious about your graph calculus. Do tell me more @badt - if you can provide a succint description (I’m thinking BNF + reduction/rewrite rules).

@badt

We want to create an assembly language that would have instructions to express both low–level ideas, such as vertices, edges, as well as high–level operations and search algorithms, so the byte–code could be iteratively optimized in pursuit of an optimal computation.

I have a bit of experience in this (building graph-based VMs). You will want assembly functions for adding, copying and removing vertices from a graph. Everything else you can build up from those primitives. Even a Y combinator.