leaderless.yaml

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: task-owners.future.ideas.protosam
spec:
  group: future.ideas.protosam


  versions:
  - name: v1beta1
    served: true
    storage: true
    
    additionalPrinterColumns:
    - name: Owner
      type: string
      description: "Operator that owns this task"
      jsonPath: .owningOperator

    - name: Age
      type: date
      jsonPath: .metadata.creationTimestamp

    schema:
      openAPIV3Schema:
        type: object
        properties:
          owningOperator:
            type: string
            default: "PENDING"

  scope: Namespaced
  names:
    plural: task-owners
    singular: task-owner
    kind: TaskOwner

---
apiVersion: v1
data:
  run.sh: |-
    #!/bin/sh
    export PYTHONUNBUFFERED=1
    pip3 install -r requirements.txt > /dev/null 2>/dev/null
    python3 operator.py

  requirements.txt: |-
    kubernetes

  operator.py: |-
    #!/usr/bin/env python3
    from kubernetes import client, config, watch
    import secrets
    import socket
    import time

    try:
        config.load_kube_config()
    except:
        # load_kube_config throws if there is no config, but does not document what it throws, so I can't rely on any particular type here
        config.load_incluster_config()

    print("operator.py is running.")

    for event in watch.Watch().stream(client.CustomObjectsApi().list_cluster_custom_object, "future.ideas.protosam", "v1beta1", "task-owners", resource_version=""):
        if event["type"] == "ADDED" and event['object']["owningOperator"] == "PENDING":
            print("Attempting claim")
            event['object']['owningOperator'] = socket.getfqdn()
            try:
                client.CustomObjectsApi().patch_namespaced_custom_object("future.ideas.protosam", "v1beta1", event['object']['metadata']['namespace'], "task-owners", event['object']['metadata']['name'], event['object'])
            except client.exceptions.ApiException:
                print("Failed to claim.")
                continue

            event['object'] = client.CustomObjectsApi().get_namespaced_custom_object("future.ideas.protosam", "v1beta1", event['object']['metadata']['namespace'], "task-owners", event['object']['metadata']['name'])

            print(event['object']['metadata']['name'], "was is claimed by me.")


kind: ConfigMap
metadata:
  name: leaderless-elections

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: leaderless-operator
automountServiceAccountToken: true

---
# minimal permissions required for the service account
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leaderless-operator
rules:
- apiGroups: [""]
  resources: ["task-owners"]
  verbs: ["list", "watch", "patch"]

---
# binding the above cluster role (permissions) to the above service account
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: leaderless-operator-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: leaderless-operator
subjects:
- kind: ServiceAccount
  name: leaderless-operator
  namespace: default

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: leaderless-operator
spec:
  replicas: 3
  selector:
    matchLabels:
      name: leaderless-operator
  template:
    metadata:
      labels:
        name: leaderless-operator
    spec:
      serviceAccountName: leaderless-operator
      automountServiceAccountToken: true
      
      containers:
      - image: python:3
        name: leaderless-operator
        command: [ 'bash', 'run.sh' ]
        workingDir: /usr/src
        volumeMounts:
        - mountPath: /usr/src
          name: source

      volumes:
      - name: source
        configMap:
          defaultMode: 420
          name: leaderless-elections

This is an example of abusing Kubernetes CRDs to have an operator claim work without a leader. In testing, I found that the apiserver has logic built in to prevent multiple updates of the same object source.

You can test this out after deploying leaderless.yaml.

Create a new object:

$ cat <<EOF | kubectl apply -f -
apiVersion: future.ideas.protosam/v1beta1
kind: TaskOwner
metadata:
    name: mytaskname-$(openssl rand -hex 12)
EOF

Check the results.

$ kubectl get task-owners
$ kubectl describe task-owners mytaskname-...

You can also check all the logs.

$ kubectl get pods | awk '/^leaderless/{print "echo "$1"; kubectl logs --tail 10 "$1"; echo ---;"}' | sh

You can also just see who owns the tasks.

$ kubectl get task-owners