Docker run ratel/dgraph behind a reverse proxy (apache mod_proxy)

Users
,
1

Hi,

Anyone out there been able to run docker base (dgraph/dgraph:latest) behind a reverse proxy - I can get access to ratel but have not yet been able to get any access to alpha or zero instances.

What I’m looking for is some idea of what the basic concept and thus configuration is required by the dgraph servers (ratel, zero, alpha) to make it work - I can easily point the docker container at my host IP (–p hostip:ext-port:int-port) and all is good but this is a multi-domain host and I’d like access to dgraph via it’s own subdomain - e.g. dgraph.my-domain.com.

I’m using Apache reverse proxy (mod_proxy) with VirtualHost configurations.

Any help would be appreciated!

1 Like
2

Hi @clintwood, Welcome to the community.
You can look at Dgraph http port forwarding using traefik thread. The user uses Traefik for as reverse proxy.

Tagging @joaquin , if he can provide some better solution.

3

Hi @Naman

Thanks for the quick response - I did look at that and didn’t get much from it but will look a bit deeper!

I have also looked at the section regarding port usage at Ports used by different nodes so I’m puzzled as to why it didn’t work behind the proxy.

Basically I want to frontend dgraph (ratel, zero, alpha) with HTTPS subdomains e.g. ratel.xyz.com, zero.xyz.com and alpha.xyz.com. But perhaps the ratel UI (web UI) is trying to access a URL that it can’t find!?

4

For the HTTPS thing you can look at TLS Configuration.
Can you provide us the configuration you are using for reverse-proxy?

5

So I’ve reverted most of the config to this which is just for ratel which works but I had similar for alpha and zero but the UI was flagging errors with the connection - I’ll have to set it up like it was again then I can give you better feedback:

 <VirtualHost *:443>
    ServerAdmin             issues@xyz.com
    ServerName              dgraph.xyz.com

    SSLEngine on
    SSLCertificateFile      /etc/ssl/cert.pem
    SSLCertificateKeyFile   /etc/ssl/privkey.pem
    SSLCertificateChainFile /etc/ssl/chain.pem

    ProxyRequests           Off
    ProxyPreserveHost       On
    AllowEncodedSlashes     NoDecode
    #RequestHeader           set X-Forwarded-Proto "https" # this makes no difference if here or not
    ProxyPass               / http://ratel:8000/ nocanon
    ProxyPassReverse        / http://ratel:8000/

    LogLevel                notice
    ErrorLog                ${APACHE_LOG_DIR}/proxies-xyz.com-error.log
    CustomLog               ${APACHE_LOG_DIR}/proxies-xyz.com-access.log combined
  </VirtualHost>

And my docker-compose.yml looks like this:

version: "3.5"
services:

  zero:
    image: dgraph/dgraph:latest
    container_name: awt-dgraph-zero
    volumes:
      - ./data:/dgraph
    expose:
      - "5080"
      - "6080"
    networks:
      - dgraph-network
      - apache-network
#    ports:
#      - '${HOST_IP_ADDRESS}:5080:5080'
#      - '${HOST_IP_ADDRESS}:6080:6080'
    restart: on-failure
    command: dgraph zero --my=zero:5080 --enable_sentry=false

  alpha:
    image: dgraph/dgraph:latest
    container_name: awt-dgraph-alpha
    volumes:
      - ./data:/dgraph
    expose:
      - "8080"
      - "9080"
    networks:
      - dgraph-network
      - apache-network
#    ports:
#      - '${HOST_IP_ADDRESS}:8080:8080'
#      - '${HOST_IP_ADDRESS}:9080:9080'
    restart: on-failure
    command: dgraph alpha --my=alpha:7080 --lru_mb=2048 --zero=zero:5080 --enable_sentry=false
#    command: dgraph alpha --whitelist 172.0.0.0:172.254.254.254,${HOST_IP_ADDRESS} --my=alpha:7080 --lru_mb=2048 --zero=zero:5080 --enable_sentry=false

  ratel:
    image: dgraph/dgraph:latest
    container_name: awt-dgraph-ratel
    expose:
      - "8000"
    networks:
      - dgraph-network
      - apache-network
#    ports:
#      - '${HOST_IP_ADDRESS}:8000:8000'
    command: dgraph-ratel

networks:
  dgraph-network:
    name: dgraph-network
  apache-network:
    external: true

I realise this is slightly off topic with dgraph specific issues so I’d be happy with just a few pointers if you can see any glaring issues! I have a number of other services which I host like this!

6

This is a non-Kubernetes scenario?

We have used K8S ingress controllers to automatically configure reverse proxy based on ingress resource settings with a variety of solutions: ingress-nginx (GKE, EKS), ALB (EKS), GLBC (GKE), Traefik, Ambassador (Envoy).

I have used nginx and haproxy in the past, but not with Dgraph outside of K8S. I could try mod_proxy out later.

Where do you place the reverse-proxy?

  • running on a docker container? then route back to internal ports within docker’s network?
  • running on the host, and would then route to ports exposed to localhost?

The two main ports you’ll want would be:

  • 8080 alpha’s http/s port
  • 8000 ratels ui port - ratel can run anywhere, but needs access to an alpha endpoint that connects to internal 8080)

My approach would be to try it out non-encrypted first, then add the TLS after, to debug one layer at a time.

The TLS Configuration docs mentioned by @Naman is used for encrypting at the alpha service so that alpha-2-alpha internal traffic is encrypted as well. Given the config code, it looks like the preferred route is to terminate the encryption at the endpoint (reverse proxy).

7

Hi @joaquin

Thanks for that info!

No it’s straight up Docker for now - K8S would ultimately be how we go moving forward - we in the evaluation phase. Also I run an apache server native on an Ubuntu 18.04 - have several other services in docker containers that I reverse proxy as I have tried with dgraph.

One thing I did try was put each of the 3 services on separate subdomains as I explained above. Would they need to all be on the same subdomain but with different ports or is it just the endpoint that is required?

I think the confusing thing is which ports can communicate internal to the docker container and which ports need to be public facing in this specific configuration.

I’ve just configured the docker container to run via an internal VPN for now - to evaluate internally. I will come back to this issue and see if I can resolve it. I will post the results here if I win - I’ll report back either way! I will be looking into this further tomorrow!

8

Thanks for the information. When you mean subdomains, do you mean alpha.west.example.com and zero.east.example.com? For internal, this shouldn’t matter, as long as they can find each other and route there, e.g. alpha is able to find zero. For external, if it is HTTP, this shouldn’t matter. Ratel especially is wholly independent, only needs an alpha endpoint. For HTTPS, it depends on the type of certificate.

For these endpoints, are they exposed on a private network or on the public Internet? For security reasons best practices are to keep database ports private (so internal LB), and if opened for on the public internet for convenience, use a white list to restrict the IPs. Mostly for Ratel, you would only hit 8000 and 8080. The Zero service would be used for administrative chores, so normally this wouldn’t need to be exposed through an endpoint.

This is the complete list of ports:

Dgraph Node Type gRPC-internal gRPC-external HTTP-external
zero 5080 - 6080
alpha 7080 9080 8080
ratel - - 8000

(from https://dgraph.io/docs/master/deploy/ports-usage/)

9

Hi

I have this working now - I suspect it was purely the whitelisting of the HOST_IP_ADDR for the alpha service that was the issue - I was assuming that would be localhost but it seems the oproxy reports the actual HOST IP address as the requesting IP - makes sense really!

For anyone else who may want to run dgraph behind an apache2 mod_proxy here is what worked for me on Ubuntu server 18.04.

.env file in my docker-compose project directory

HOST_IP_ADDRESS=123.123.123.123

docker-compose.yml in my docker-compose project directory

version: "3.5"
services:

  zero:
    image: dgraph/dgraph:latest
    container_name: awt-dgraph-pub-zero
    volumes:
      - ./data:/dgraph
    expose:
      - "5080"
      - "6080"
    networks:
      - dgraph-pub-network
      - apache-network # not exposed but could be as was alpha
    restart: on-failure
    command: dgraph zero --my=zero:5080 --enable_sentry=false

  alpha:
    image: dgraph/dgraph:latest
    container_name: awt-dgraph-pub-alpha
    volumes:
      - ./data:/dgraph
    expose:
      - "8080"
      - "9080"
    networks:
      - dgraph-pub-network
      - apache-network
    restart: on-failure
    command: dgraph alpha --whitelist ${HOST_IP_ADDRESS} --my=alpha:7080 --lru_mb=2048 --zero=zero:5080 --enable_sentry=false

  ratel:
    image: dgraph/dgraph:latest
    container_name: awt-dgraph-pub-ratel
    expose:
      - "8000"
    networks:
      - dgraph-pub-network
      - apache-network
    command: dgraph-ratel

networks:
  dgraph-pub-network:
    name: dgraph-pub-network
  apache-network:
    external: true

dgraph-proxies.conf in sites-enabled directory of native apache2 installation (no dockerized)

<IfModule mod_ssl.c>

  <VirtualHost *:443>
    ServerAdmin             info@xyz.com
    ServerName              dgraph.xyz.com

    SSLEngine on
    SSLCertificateFile      /etc/ssl/cert.pem
    SSLCertificateKeyFile   /etc/ssl/privkey.pem
    SSLCertificateChainFile /etc/ssl/chain.pem

    ProxyRequests           Off
    ProxyPreserveHost       On
    AllowEncodedSlashes     NoDecode
    ProxyPass               / http://ratel:8000/ nocanon
    ProxyPassReverse        / http://ratel:8000/

    LogLevel                notice
    ErrorLog                ${APACHE_LOG_DIR}/proxies-xyz.com-error.log
    CustomLog               ${APACHE_LOG_DIR}/proxies-xyz.com-access.log combined
  </VirtualHost>

  <VirtualHost *:443>
    ServerAdmin             info@xyz.com
    ServerName              dgraph-alpha.xyz.com

    SSLEngine on
    SSLCertificateFile      /etc/ssl/cert.pem
    SSLCertificateKeyFile   /etc/ssl/privkey.pem
    SSLCertificateChainFile /etc/ssl/chain.pem

    ProxyRequests           Off
    ProxyPreserveHost       On
    AllowEncodedSlashes     NoDecode
    ProxyPass               / http://alpha:8080/ nocanon
    ProxyPassReverse        / http://alpha:8080/

    LogLevel                notice
    ErrorLog                ${APACHE_LOG_DIR}/proxies-xyz.com-error.log
    CustomLog               ${APACHE_LOG_DIR}/proxies-xyz.com-access.log combined
  </VirtualHost>

</IfModule>

Please note that this is just for our evaluation phase - ordinarily we would be access the dgraph services within our virtual network from server side API’s.

Thanks for the help!

3 Likes
10

That’s great @clintwood. As this solution worked for you, we are very grateful that you shared it too.
I would like you to mark your answer as solution so as to make this more beneficial for community. :slight_smile:

11

Thanks Clinton for this solution.
Guys I had ground away for hours trying to get a Caddy reverse proxy working with Ratel and Alpha before I found this. I was a gnat’s whisker away from ditching my trial and going to Neo4J or something else.
It’s great that you have the sample Docker Compose and containers, but it’s common these days to host in the cloud and have a reverse proxy Caddy/Traefik/Nginx/Load balancer in front, some modelling of such scenarios would help - how many others have given up and bounced.
It’s going now though so am good to go!

12

You’re most welcome!!

1 Like