Binary formats no longer supported?

Users Dgraph
#1

I stumbled across this Dgraph blog post mentioning Protocol Buffers-based binary formats in Dgraph from a few years ago Dgraph: JSON vs. Binary clients - Dgraph Blog

But as I was going through the client code (the dgraph-rs community client as well as the official Golang client) I noticed that the Response message now has an embedded field that features json, so it would seem like the current state of Dgraph is that there’s no more binary format as much as a gRPC-wrapped JSON. Is this the case?

Our application uses binary protocols from top down and we’re seeing a bunch of time lost in the middle to deserializing JSON responses from Dgraph and wondering if it’s really necessary.

#2

Tagging @pawan for help.

#3

We earlier returned the response in protocol buffer format through the gRPC endpoint. Our users told us that they had to convert the response to an intermediary JSON like structure because the protocol buffer response was not easy to work with. Hence, we dropped the protocol buffer response which is not that great for representing Graph (or cyclic) data structures. Now you just unmarshal once and get something that can be sent to the client or even used directly by the client.

1 Like
#4

Now you just unmarshal once and get something that can be sent to the client or even used directly by the client.

Unless the result isn’t going to a client directly, in which case we spend a lot of time parsing JSON. Is there no way to opt into a binary result format anymore?

#5

All responses from Dgraph are in JSON format. How much time are you spending on deserializing JSON? That part shouldn’t be that expensive compared to the time taken for query processing.

#6

@dmai We’re losing about ~1 second to JSON deserialization right now in our requests. Even if it’s less than the query processing, it’s still time lost.

It’d be neat to have the option of encoding the result in something like MessagePack at least.

2 Likes
#7

Yes I agree some sort of wire format other then JSON would be a benefit. Epically if you just use dgraph internally and map to GRPC/Protobuf outputs. Does not have to be be protobuf, maybe something like BSON, or even just send over the dgraph internal format.

3 Likes