Tweaks for K8s 1.9.5

Users
1

I’ve struggled through getting a dgraph running on an on-premise K8S cluster (version 1.9.5) and there are number of tweaks. Most importantly, I needed the set the hostPort property on the StatefulSet for the dgraph server replicas. Have others run into this?

Essentially, it looks something like this:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: dgraph-server
spec:
  serviceName: "dgraph-server"
  replicas: 3
  selector:
    matchLabels:
      app: dgraph-server
  template:
    metadata:
      labels:
        app: dgraph-server
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - dgraph-server
              topologyKey: kubernetes.io/hostname
      containers:
      - name: server
        image: dgraph/dgraph:latest
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 7080
          hostPort: 7080
          name: server-grpc-int
        - containerPort: 8080
          name: server-http
        - containerPort: 9080
          name: server-grpc
        volumeMounts:
        - name: datadir
          mountPath: /dgraph
        command:
          - bash
          - "-c"
          - |
            set -ex
            dgraph server --my=$(hostname -f):7080 --memory_mb=65535 --zero=dgraph-zero-0.dgraph-zero:5080
      terminationGracePeriodSeconds: 60
      volumes:
      - name: datadir
        persistentVolumeClaim:
          claimName: datadir
  updateStrategy:
    type: RollingUpdate
  volumeClaimTemplates:
  - metadata:
      name: datadir
      annotations:
        volume.beta.kubernetes.io/storage-class: dgraph-storage-server
    spec:
      accessModes:
        - "ReadWriteOnce"
      resources:
        requests:
          storage: 100Gi

Also, oddly, this service seems to be broken:

apiVersion: v1
kind: Service
metadata:
  name: dgraph-server-0-http-public
  labels:
    app: dgraph-server
spec:
  type: LoadBalancer
  ports:
  - port: 8080
    targetPort: 8080
    name: server-http
  selector:
    statefulset.kubernetes.io/pod-name: dgraph-server-0

I can port-forward the dgraph-server-0 http protocol and everything is fine. This might be an issue with our on-premise k8s deployment too.

2

Another note, you should comment out one of the public dgraph server public services. The service dgraph-server-public and dgraph-server-0-http-public are both load balanced and run on port 8080; may confuse k8s. In my case, the dgraph-server-public service did not function, dgraph-server-0-http-public was silently not created.

An alternate definition for dgraph-server-0-http-public might be:

apiVersion: v1
kind: Service
metadata:
  name: dgraph-server-0-http-public
  labels:
    app: dgraph-server
spec:
  type: LoadBalancer
  ports:
  - port: 8081
    targetPort: 8080
    name: server-http
  selector:
    statefulset.kubernetes.io/pod-name: dgraph-server-0

Note the different port for dgraph-server-0-http-public.

It may be the case that load balanced services and/or ingress to the service will run differently depending on how k8s is deployed, what version you have, and on-premise versus cloud providers.

Certainly having two services, load balanced, on the same port will cause some issues.

3

I have a solution for the hostPort issue - as hostPort should be avoided (see [1]). The Dockerfile for dgraph needs to expose port 7080 so that k8s will make it available internally to the cluster. Right now it only exposes 8080 and 9080.

[1] k8s - Port 7080 not exposed · Issue #2328 · dgraph-io/dgraph · GitHub

4

I’m setting this up right now as well and will take some of your changes into consideration. Using the same setup with 3 separate AWS instances.

Have you had a situation where your pods remain in ContainerCreating state? Mine looks like this:

NAME                            READY     STATUS              RESTARTS   AGE
dgraph-ratel-7b7bc97649-nj7mf   0/1       ContainerCreating   0          15h
dgraph-server-0                 0/1       Pending             0          15h
dgraph-zero-0                   0/1       Pending             0          15h
5

I have not see this. The dgraph-ratel container is just a deployment and independent of the state of the cluster. Since that is still being created after 15 hours, I think something else must be amiss.

6

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.