Build file for DGraph Docker

Continuing the discussion from Vendoring dependencies for dgraph:

Hey @compton,

Interested in this approach. I noticed that you use gliderlabs/alpine for this. Is that the reason why your docker image is so small? Also, does gliderlabs/alpine has Go 1.6.2?

That’s probably ~98% of the difference. You could probably consider this to be an entirely separate issue and net an immediate, huge win in simply switching the base image to go-1.6.2:alpine.

I tried this:

-FROM golang:1.6.2
+FROM golang:1.6.2-alpine

But, of course, that doesn’t work. Because we need apt-get packages to install RocksDB. If you want to send a pull request to do what you’re proposing and with RocksDB installed in the final Docker image, more than happy to accept it.

Thanks @compton!

Yeah, I just waded through all of that. It will be up in about 20 mins. The PR that is. Assuming I have all of the deps right this time…

http://git.alpinelinux.org/cgit/aports/plain/testing/rocksdb/

I’ve run into the final hurdle it seems. Two patches

Edit: Yah, I’ll have to finish this later. I assume that you want to own those patches in some way. Could just fork rocksdb or throw them into gists. Of course, we could also just use the alpine rocksdb and rocksdb-dev packages. Seems a little sketch though.

1 Like

Update: I have not been able to get the dockerfile under 400mb using golang:1.6.2-alpine w/building in the dockerfile.

# Dockerfile for DGraph
FROM golang:1.6.2-alpine

MAINTAINER Manish Jain <manishrjain@gmail.com>

ENV DGRAPH github.com/dgraph-io/dgraph

COPY . $GOPATH/src/$DGRAPH
RUN mkdir -p $GOPATH/src/$DGRAPH && \
    mkdir /dgraph && mkdir /data && \
    sed -i -e 's/v3\.3/edge/g' /etc/apk/repositories; \
    echo 'http://dl-4.alpinelinux.org/alpine/edge/community' >> /etc/apk/repositories && \
    echo 'http://dl-4.alpinelinux.org/alpine/edge/main' >> /etc/apk/repositories && \
    echo 'http://dl-4.alpinelinux.org/alpine/edge/testing' >> /etc/apk/repositories && \
    apk update; \
    apk add rocksdb rocksdb-dev build-base

RUN go build -v $DGRAPH/... && \
    go test $DGRAPH/... && echo "v0.2.3" && \
    apk del rocksdb-dev build-base go && rm -rf /var/cache/apk/*

^ That’s the last version I was playing with after deciding to just use the rocksdb-dev repo instead of building manually and applying patches.

It’s too much effort for me rn. I’ll have to wait until the weekend. I’ll open a PR though

1 Like

I’m wary of going with the edge branch for our docker image, which is supposed to be production ready.

https://wiki.alpinelinux.org/wiki/Edge

Instead, if we want to gain the benefits of alpine linux, we should build RocksDB within it; same way we were doing it for Ubuntu.

That is a fair call and I assumed that this would be the case. Most if not all of the dependencies required to build the rocksdb package in alpine are also in edge. At that point it probably becomes non-trivial work and deviates too far my goals (minimal container, add ci, determine best way to deploy to AWS (ECS would be neat), deploy, evaluate) to justify following through with the PR. I’ll keep this in mind as I move forward though.

We’ve gone far off topic!

1 Like

Thanks @compton for spending effort in moving in this direction. At least now, this is something on our radar. If some day, rocksdb-dev moves to the stable branch of alpine, we can move to it.

1 Like

Hey @compton,

We just figured out a way to not require installation of RocksDB. Would be great if you took another stab at the Docker image with these changes. @ashwin95r can fill you in with the details.

Cheers,
Manish

2 Likes

Would love to. Fill me in and I’ll take a stab at it this weekend. Currently catching up on work. Just moved into a new place here in SF so a little behind.

1 Like

Hello @mrjn,
just a suggestion of getting this working with alpine. Would it be possible to compile the rocksdb and also dgraph code on host machine and then move the binary version of them into alpine docker and upload the image to docker hub. I have been using this way for quite a while and I’ve managed to reduce the docker size down to 20mb.

p.s. This is an amazing project and thanks for doing this.

1 Like

Hey @alinz,

The next release of Dgraph would hopefully do away with RocksDB and use a pure Go alternative for key-value store that we’re building in-house. That would help with alpine linux as well.