# Tweaks for K8s 1.9.5

**URL:** <https://discuss.dgraph.io/t/tweaks-for-k8s-1-9-5/2412>\
**Category:** Users\
**Created:** [April 4, 2018, 6:34am UTC](https://discuss.dgraph.io/t/tweaks-for-k8s-1-9-5/2412 "2018-04-04T06:34:09Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![alexmilowski](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/alexmilowski/32/1246_2.png) [@alexmilowski](https://discuss.dgraph.io/u/alexmilowski)\
**Post date:** [April 4, 2018, 6:34am UTC](https://discuss.dgraph.io/t/tweaks-for-k8s-1-9-5/2412/1 "2018-04-04T06:34:09Z")

</div>

I’ve struggled through getting a dgraph running on an on-premise K8S cluster (version 1.9.5) and there are number of tweaks. Most importantly, I needed the set the hostPort property on the StatefulSet for the dgraph server replicas. Have others run into this?

Essentially, it looks something like this:

```auto
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: dgraph-server
spec:
  serviceName: "dgraph-server"
  replicas: 3
  selector:
    matchLabels:
      app: dgraph-server
  template:
    metadata:
      labels:
        app: dgraph-server
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - dgraph-server
              topologyKey: kubernetes.io/hostname
      containers:
      - name: server
        image: dgraph/dgraph:latest
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 7080
          hostPort: 7080
          name: server-grpc-int
        - containerPort: 8080
          name: server-http
        - containerPort: 9080
          name: server-grpc
        volumeMounts:
        - name: datadir
          mountPath: /dgraph
        command:
          - bash
          - "-c"
          - |
            set -ex
            dgraph server --my=$(hostname -f):7080 --memory_mb=65535 --zero=dgraph-zero-0.dgraph-zero:5080
      terminationGracePeriodSeconds: 60
      volumes:
      - name: datadir
        persistentVolumeClaim:
          claimName: datadir
  updateStrategy:
    type: RollingUpdate
  volumeClaimTemplates:
  - metadata:
      name: datadir
      annotations:
        volume.beta.kubernetes.io/storage-class: dgraph-storage-server
    spec:
      accessModes:
        - "ReadWriteOnce"
      resources:
        requests:
          storage: 100Gi

```

Also, oddly, this service seems to be broken:

```auto
apiVersion: v1
kind: Service
metadata:
  name: dgraph-server-0-http-public
  labels:
    app: dgraph-server
spec:
  type: LoadBalancer
  ports:
  - port: 8080
    targetPort: 8080
    name: server-http
  selector:
    statefulset.kubernetes.io/pod-name: dgraph-server-0

```

I can port-forward the dgraph-server-0 http protocol and everything is fine. This might be an issue with our on-premise k8s deployment too.

---

<div class="post-metadata">

**Author:** ![alexmilowski](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/alexmilowski/32/1246_2.png) [@alexmilowski](https://discuss.dgraph.io/u/alexmilowski)\
**Post date:** [April 4, 2018, 6:07pm UTC](https://discuss.dgraph.io/t/tweaks-for-k8s-1-9-5/2412/2 "2018-04-04T18:07:35Z")

</div>

Another note, you should comment out one of the public dgraph server public services. The service dgraph-server-public and dgraph-server-0-http-public are both load balanced and run on port 8080; may confuse k8s. In my case, the dgraph-server-public service did not function, dgraph-server-0-http-public was silently not created.

An alternate definition for dgraph-server-0-http-public might be:

```auto
apiVersion: v1
kind: Service
metadata:
  name: dgraph-server-0-http-public
  labels:
    app: dgraph-server
spec:
  type: LoadBalancer
  ports:
  - port: 8081
    targetPort: 8080
    name: server-http
  selector:
    statefulset.kubernetes.io/pod-name: dgraph-server-0

```

Note the different port for dgraph-server-0-http-public.

It may be the case that load balanced services and/or ingress to the service will run differently depending on how k8s is deployed, what version you have, and on-premise versus cloud providers.

Certainly having two services, load balanced, on the same port will cause some issues.

---

<div class="post-metadata">

**Author:** ![alexmilowski](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/alexmilowski/32/1246_2.png) [@alexmilowski](https://discuss.dgraph.io/u/alexmilowski)\
**Post date:** [April 10, 2018, 10:33pm UTC](https://discuss.dgraph.io/t/tweaks-for-k8s-1-9-5/2412/3 "2018-04-10T22:33:25Z")

</div>

I have a solution for the hostPort issue - as hostPort should be avoided (see [1]). The Dockerfile for dgraph needs to expose port 7080 so that k8s will make it available internally to the cluster. Right now it only exposes 8080 and 9080.

[1] [k8s - Port 7080 not exposed · Issue #2328 · dgraph-io/dgraph · GitHub](https://github.com/dgraph-io/dgraph/issues/2328)

---

<div class="post-metadata">

**Author:** ![ddibiase](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/ddibiase/32/1157_2.png) [@ddibiase](https://discuss.dgraph.io/u/ddibiase)\
**Post date:** [April 11, 2018, 2:22pm UTC](https://discuss.dgraph.io/t/tweaks-for-k8s-1-9-5/2412/4 "2018-04-11T14:22:48Z")

</div>

I’m setting this up right now as well and will take some of your changes into consideration. Using the same setup with 3 separate AWS instances.

Have you had a situation where your pods remain in ContainerCreating state? Mine looks like this:

```auto
NAME READY STATUS RESTARTS AGE
dgraph-ratel-7b7bc97649-nj7mf 0/1 ContainerCreating 0 15h
dgraph-server-0 0/1 Pending 0 15h
dgraph-zero-0 0/1 Pending 0 15h

```

---

<div class="post-metadata">

**Author:** ![alexmilowski](https://yyz1.discourse-cdn.com/flex007/user_avatar/discuss.dgraph.io/alexmilowski/32/1246_2.png) [@alexmilowski](https://discuss.dgraph.io/u/alexmilowski)\
**Post date:** [April 12, 2018, 8:24pm UTC](https://discuss.dgraph.io/t/tweaks-for-k8s-1-9-5/2412/5 "2018-04-12T20:24:42Z")

</div>

I have not see this. The dgraph-ratel container is just a deployment and independent of the state of the cluster. Since that is still being created after 15 hours, I think something else must be amiss.

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex007/uploads/dgraph/original/2X/2/2b38fdece2abe5814f2e51bee69d1e67fc304b0a.png) [@system](https://discuss.dgraph.io/u/system)\
**Post date:** [May 12, 2018, 8:24pm UTC](https://discuss.dgraph.io/t/tweaks-for-k8s-1-9-5/2412/6 "2018-05-12T20:24:51Z")

</div>

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.