Understanding protobuf plugins/runtime library

Users
1

So @mrjn mentioned a doubt about whether a couple of libraries regarding gogo protobuf should be vendored in or not. I did some experimentation and learnt some things which I wanted to jot down in a topic for future reference. Maybe this can be part of the Wiki later. Maybe you guys already know these things still thought I’d share what I learned here.

For those who haven’t used proto files much, I will try to give some background. To use protocol buffer, you need three things.

  1. Protocol buffer compiler protoc
  2. Compiler plugin to generate .pb.go files from .proto files. This could be github.com/golang/protobuf/protoc-gen-go or recently we started using github.com/gogo/protobuf/protoc-gen-gofast.
  3. Runtime library which does unmarshalling and marshalling at runtime. This could be github.com/golang/protobuf/proto or github.com/gogo/protobuf/proto. This is part of your code.

Now with regards to vendoring in the compiler plugin. That won’t help us because while running this command protoc --gofast_out=plugins=grpc:. *.proto, proto compiler searches for the plugin in our PATH.

As you know, when we do a go get github.com/gogo/protobuf/protoc-gen-gofast, it installs the binary under $GOPATH/bin, hence its available in our PATH and things work. So it’s important for the user to install the plugin.

I still have to check about using the runtime library from Gogo guys instead of the Golang one. There are also a bunch of extensions which can make gogo even faster protobuf/extensions.md at master · gogo/protobuf · GitHub. Though I am not sure if using these would break the compatability of the Python/Java clients.

All this understanding helped in deleting a couple of lines from the Wiki :stuck_out_tongue: .

Updated Wiki section - https://wiki.dgraph.io/Contributing_to_Dgraph#Protocol_buffers_.28Optional.29

2 Likes
2

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.