Mystery around embedded binary sizes on Mac

We build our binaries using go build -tags=embed -v. This allows gorocksdb to embed RocksDB into the binary. Then it can be run on any instance without the need to even install RocksDB.

The funny thing is that if I delete c-rocksdb, c-snappy and c-lz4 from my vendor and they aren’t even in my GOPATH, then also the above command succeeds and result in a much smaller binary size which is also built pretty fast.

After deleting rm -rf vendor/github.com/cockroachdb

➜  dgraph git:(master) ✗ go build -tags=embed -v .
github.com/dgraph-io/dgraph/vendor/github.com/google/flatbuffers/go
github.com/dgraph-io/dgraph/vendor/github.com/tecbot/gorocksdb
github.com/dgraph-io/dgraph/lex
github.com/dgraph-io/dgraph/commit
github.com/dgraph-io/dgraph/gql
github.com/dgraph-io/dgraph/vendor/github.com/zond/gotomic
github.com/dgraph-io/dgraph/vendor/github.com/golang/protobuf/proto
github.com/dgraph-io/dgraph/posting/types
github.com/dgraph-io/dgraph/task
github.com/dgraph-io/dgraph/x
github.com/dgraph-io/dgraph/vendor/google.golang.org/grpc/credentials
github.com/dgraph-io/dgraph/vendor/google.golang.org/grpc/naming
github.com/dgraph-io/dgraph/vendor/github.com/gogo/protobuf/proto
github.com/dgraph-io/dgraph/vendor/google.golang.org/grpc/peer
github.com/dgraph-io/dgraph/vendor/google.golang.org/grpc/transport
github.com/dgraph-io/dgraph/vendor/google.golang.org/grpc
github.com/dgraph-io/dgraph/query/graph
github.com/dgraph-io/dgraph/store
github.com/dgraph-io/dgraph/posting
github.com/dgraph-io/dgraph/uid
github.com/dgraph-io/dgraph/worker
github.com/dgraph-io/dgraph/rdf
github.com/dgraph-io/dgraph/query
github.com/dgraph-io/dgraph/cmd/dgraph

The size of the binary that is built is 18M. Verified on a couple of Mac which didn’t have RocksDB that it worked. I suppose it uses the RocksDB installed on my machine somehow for embedding.

Without deleting the size of the binary is 55M and takes much longer to build. In this case the major time is spent on embedding c-rocksdb. Have to dig deeper into gorocksdb to understand whats happening here.

2 Likes

This is interesting, and also signifies that magical external packages are magical. And magic is bad in engineering :slight_smile:.

I knew that it uses the rocksdb installation on the compiling machine for the embedding, but this sort of inter-effect is really strange.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.