K8s - Add Routes to a Host From Pods

I’m in a situation where I have a Kubernetes cluster on Hetzner, created using a custom Ansible playbook. The playbook is executed only once to build the cluster and nothing more. The set of instructions in the playbook is supposed to be idempotent, but I have persistent volumes in my cluster, and I don’t want to risk disrupting it (also, I’m too lazy to back up my volumes). Network interfaces are managed by the cloud provider’s ISO, and since cloud-init runs only on the first boot, it is not a feasible solution to install the persistent static routes that I need.

So, what should I do to install my routes? Wouldn’t it be nice to manage the static routes I need directly from a pod? Maybe not, it sounds like a ‘hacky’ solution. But let me hack around just for fun.

I put the routes I need into a ConfigMap, just like this:

apiVersion: v1
kind: ConfigMap
metadata:
  name: hc-static-routes
  labels:
    app: host-configs
    component: static-routes
data:
  static-routes: |
    #dest gw dev
    10.0.0.0/8 10.0.0.1 enp7s0
    172.16.0.0/12 10.0.0.1 enp7s0
    192.168.0.0/16 10.0.0.1 enp7s0

And now, the ‘magic’ is handled by a DaemonSet, as I want to install the routes on all of my hosts:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: hc-static-routes
  labels:
    app: host-configs
    component: static-routes
  annotations:
    command: &cmd cat /config/static-routes | while read dest gw dev; do ip r r $dest via $gw dev $dev; done
spec:
  selector:
    matchLabels:
      app: host-configs
      component: static-routes
  template:
    metadata:
      labels:
        app: host-configs
        component: static-routes
    spec:
      hostNetwork: true
      hostPID: true
      initContainers:
      - name: install-routes
        image: praqma/network-multitool:latest
        securityContext:
          privileged: true
        command:
          - nsenter
          - -t
          - "1"
          - -n
          - --
          - bash
          - -c
          - *cmd
        volumeMounts:
          - name: config
            mountPath: /config
            readOnly: true
      containers:
      - name: sleep
        image: registry.k8s.io/pause:3.1
      volumes:
        - name: config
          configMap:
            name: hc-static-routes
  updateStrategy:
    type: RollingUpdate

The hostPid and hostNetwork flags set to true, in conjunction with nsenter, give me the ability to enter the network namespace of the host machine’s init process. From there, I can install the routes I need.

K8s - Add Routes to a Host From Pods

Table of Contents