Nedtes

My ideas, thoughts and questions, but mostly just software

Deploy to Kubernetes From Travis

Introduction

When you deploy applications to your kubernetes engine. You’d want it to be done as securely as possible of course. A little issue when you think about that is that you need to grant the deployment job as little permissions as possible and absolutely in a specific namespace for better isolation. In order to do that, we could use Google IAM (In case you are running on GKE). Or we can use the kubernetes native way.

Creating the deployment user

We will need first to create a service account user with specific permissions (based on our deployment script). In a specific namespace, for this example the user will be named travis-echo in the echo namespace. Assuming that our deployment consists of updating Deployment object and Service and Ingress as well as ConfigMaps.

travis-ca.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
apiVersion: v1
kind: ServiceAccount
metadata:
  name: travis-echo
  namespace: echo
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  namespace: echo
  name: travis-deploy-role
rules:
- apiGroups: ["extensions", "apps"]
  resources: ["deployments"]
  verbs: ["get", "create", "update", "patch"]
- apiGroups: [""]
  resources: ["services"]
  verbs: ["get", "create", "update", "patch"]
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get", "create", "update", "patch"]
- apiGroups: ["extensions", "apps"]
  resources: ["ingresses"]
  verbs: ["get", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: travis-role-binding
  namespace: echo
subjects:
- kind: ServiceAccount
  name: travis-echo
  namespace: echo
roleRef:
  kind: Role
  name: travis-deploy-role
  apiGroup: rbac.authorization.k8s.io

We then create this ServiceAccount as apply its permissions:

kubectl apply -f travis-sa.yml

Getting user credentials

Now that the user is created. We need to get its credentials that kubernetes generated to be able to use it.

You can use the following script

The script will output 3 variables, that CA_CRT, USER_TOKEN and CLUSTER_ENDPOINT. Go to your travis settings and set the variables names and their values.

Deploying

Now that we have everything setup for us, the last thing we need to actually deploy our files to kubernets.

The last part of our .travis.yml looks like this:

travis-ca.yml
1
2
3
4
5
deploy:
  provider: script
  script: scripts/k8s-deploy.sh
  on:
    branch: master

And the our deploy script looks like this

k8s-deploy.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
#!/usr/bin/env bash
set -o pipefail
set -o errexit
set -o nounset
# set -o xtrace

echo $CA_CRT | base64 --decode -i > ${HOME}/ca.crt

kubectl config set-cluster our-k8s-cluster --embed-certs=true --server=${CLUSTER_ENDPOINT} --certificate-authority=${HOME}/ca.crt
kubectl config set-credentials travis-echo --token=$SA_TOKEN
kubectl config set-context travis --cluster=$CLUSTER_NAME --user=travis-echo --namespace=echo
kubectl config use-context travis
kubectl config current-context


kubectl apply -f service.yml
kubectl apply -f deployment.yml
kubectl apply -f ingress.yml

function cleanup {
    printf "Cleaning up...\n"
    rm -vf "${HOME}/ca.crt"
    printf "Cleaning done."
}

trap cleanup EXIT

And that’s it. We’re done, we are good to go for our amazing deployments from Travis. If you’re too lazy and want this whole thing automated in one script, feel free to tweet me @kiloreux.