Skip to content

Instantly share code, notes, and snippets.

@asmacdo

asmacdo/tutorial.md

Last active July 28, 2020 19:20
Show Gist options
  • Save asmacdo/a41628e1390689e132e24a73483de84f to your computer and use it in GitHub Desktop.
Save asmacdo/a41628e1390689e132e24a73483de84f to your computer and use it in GitHub Desktop.
Download ZIP
Raw
tutorial.md
title linkTitle weight
Ansible Operator Tutorial
Tutorial
3

This tutorial walks operator authors through the entire process of:

  • building a simple Ansible-based operator using the operator sdk scaffolding
  • deploying and using the operator on a cluster.

If you would prefer to try out an existing operator you can skip to the Using the Operator section using a container we've built from the [operator-sdk-samples][TODO/ansible-memcached] repository.

TODO(asmacdo) remove Note to reviewer

  I can remove this part for now, but if we put this in place,
  prospective operator authors can understand and play with a working
  operator without needing to write one.

Prerequisites

  • [docker][docker_tool] version 17.03+.
  • [kubectl][kubectl_tool] version v1.11.3+.
  • [Ansible Operator SDK Installation][ansible-operator-install] v1.0.0+
  • Access to a Kubernetes v1.11.3+ cluster.

Creating an Operator

In this tutorial we will:

  • extend the Kubernetes API with a [Custom Resource Definition][TODO] that allows users to create Memcached resources.
  • create a manager that updates the state of the cluster to the desired state defined by Memcached resources.

Scaffold a New Operator

Begin by generating a new project from a new directory.

# TODO(asmacdo) I Dont think this needs to be in the gopath, even for `make run`?
$ mkdir memcached-operator && cd memcached-operator
$ operator-sdk init --plugins=ansible

Among the files generated by this command is a Kubebuilder PROJECT file. Subsequent operator-sdk commands (and help text) run from the project root read this file and are aware that the project type is Ansible.

# Since this is an Ansible-based project, this help text is Ansible specific.
$ operator-sdk create api -h

Next, we will create a Memcached API.

$ operator-sdk create api --generate-playbook --group cache --version v1alpha1 --kind Memcached --generate-role

The scaffolded operator has the following structure:

  • Memcached Custom Resource Definition, and a sample Memcached resource.
  • A "Manager" that [reconciles][TODO] the state of the cluster to the desired state
    • A reconciler, which is an Ansible Role or Playbook.
    • A watches.yaml file, which connects the Memcached resource to the memcached Ansible Role.

See [scaffolded files reference][TODO] and [watches reference][watches reference TODO] for more detailed information

Modify the Manager

Now we need to provide the reconcile logic, in the form of an Ansible Role, which will run every time a Memcached resource is created, updated, or delete.

Update roles/memecached/tasks/main.yml:

---
- name: start memcached
  community.kubernetes.k8s:
    definition:
      kind: Deployment
      apiVersion: apps/v1
      metadata:
        name: '{{ meta.name }}-memcached'
        namespace: '{{ meta.namespace }}'
      spec:
        replicas: "{{size}}"
        selector:
          matchLabels:
            app: memcached
        template:
          metadata:
            labels:
              app: memcached
          spec:
            containers:
            - name: memcached
              command:
              - memcached
              - -m=64
              - -o
              - modern
              - -v
              image: "docker.io/memcached:1.4.36-alpine"
              ports:
                - containerPort: 11211

This memcached role will:

  • Ensure a memcached Deployment exists
  • Set the Deployment size

It is good practice to set default values for variables used in Ansible Roles, so edit memcached-operator/roles/memecached/defaults/main.yml:

---
# defaults file for Memcached
size: 1

Finally, update the Memcached sample, memcached-operator/config/samples/cache_v1alpha1_memcached.yaml:

apiVersion: cache.example.com/v1alpha1
kind: Memcached
metadata:
  name: memcached-sample
spec:
  size: 3

The key-value pairs in the Custom Resource spec are passed to Ansible as extra variables.

Note: The names of all variables in the spec field are converted to snake_case by the operator before running ansible. For example, serviceAccount in the spec becomes service_account in ansible. You can disable this case conversion by setting the snakeCaseParameters option to false in your watches.yaml. It is recommended that you perform some type validation in Ansible on the variables to ensure that your application is receiving expected input.

Finishing up

All that remains is building and pushing the operator container to your favorite registry.

$ IMG=quay.io/someuser/image:v1.0.0 make docker-build && make docker-push

Note To allow the cluster to pull the image, the repository needs to be set to public.

Using the Operator

This section walks through the steps that operator users will perform to deploy the operator and managed resources.

Select Operator Image

If you've built an operator using the steps above, use your image:

$ export IMG=<yourimage>

OR: If you just want to try out an operator, you can also use our pre-built image.

export IMG=quay.io/operator-sd/ansible-memcached-operator-tutorial:v0.0.0

Install the CRD

To apply the Memcached Kind (CRD):

$ make install

Deploy the Operator:

# IMG environment variable must be set
$ make deploy

Verify that the memcached-operator is up and running:

$ kubectl get deployment
NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
memcached-operator       1         1         1            1           1m

Create Memcached Resource

Create the resource, the operator will do the rest.

kubectl apply -f config/samples/cache_v1alpha1_memcached.yaml

Verify that Memcached pods are created

kubectl get pods

View the Ansible logs

In order to see the logs from a particular you can run:

kubectl logs deployment/kb-controller-manager -c manager

The logs contain the information about the Ansible run and will make it much easier to debug issues within your Ansible tasks. Note that the logs will contain much more detailed information about the Ansible Operator's internals and interface with Kubernetes as well.

Also, you can use the environment variable ANSIBLE_DEBUG_LOGS set as True to check the full Ansible result in the logs in order to be able to debug it.

Example

In the deploy/operator.yaml:

...
- name: ANSIBLE_DEBUG_LOGS
  value: "True"
...

Cleanup

Clean up the resources:

# TODO(asmacdo) the CR should be removed when CRD is removed $ kubectl delete -f config/samples/blahblak
$ make undeploy

Next Steps

For development it may be easier to [run the operator locally][TODO(asmacdo)], which does not require rebuilding the manager image every iteration.

We recommend reading through the our [Ansible development section][TODO(asmacdo)] for tips and tricks.

In this tutorial, the scaffolded watches.yaml could be used as-is, but has additional optional features. See [watches reference][TODO(asmacdo)]

For brevity, some of the scaffolded files were left out of this guide. See [Scaffolding Reference][TODO(asmacdo)]

This example built a namespaced scope operator, but Ansible operators can also be used with cluster-wide scope. See the operator scope documentation.

To version, install, and upgrade your operator, see out [Operator Lifecycle Management integration][TODO(asmacdo)] documentation.

END

TODO(asmacdo) Run Locally, NEW DOC

This method is preferred during the development cycle to speed up deployment and testing.

Note: Ensure that Ansible Runner and Ansible Runner HTTP Plugin is installed or else you will see unexpected errors from Ansible Runner when a Custom Resource is created.

It is also important that the role path referenced in watches.yaml exists on your machine. Since we are normally used to using a container where the Role is put on disk for us, we need to manually copy our role to the configured Ansible Roles path (e.g /etc/ansible/roles.

Run the operator locally with the default Kubernetes config file present at $HOME/.kube/config:

$ operator-sdk run local
INFO[0000] Go Version: go1.10
INFO[0000] Go OS/Arch: darwin/amd64
INFO[0000] operator-sdk Version: 0.0.5+git

Run the operator locally with a provided Kubernetes config file:

$ operator-sdk run local --kubeconfig=config
INFO[0000] Go Version: go1.10
INFO[0000] Go OS/Arch: darwin/amd64
INFO[0000] operator-sdk Version: 0.0.5+git

TODO(asmacdo) OLM new DOC

Deploy your Operator with the Operator Lifecycle Manager (OLM)

OLM will manage creation of most if not all resources required to run your operator, using a bit of setup from other operator-sdk commands. Check out the OLM integration user guide for more information.

Create a Memcached CR

Modify deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml as shown and create a Memcached custom resource:

$ cat deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml
apiVersion: "cache.example.com/v1alpha1"
kind: "Memcached"
metadata:
  name: "example-memcached"
spec:
  size: 3

$ kubectl apply -f deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml

Ensure that the memcached-operator creates the deployment for the CR:

$ kubectl get deployment
NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
memcached-operator       1         1         1            1           2m
example-memcached        3         3         3            3           1m

Check the pods to confirm 3 replicas were created:

$ kubectl get pods
NAME                                  READY     STATUS    RESTARTS   AGE
example-memcached-6fd7c98d8-7dqdr     1/1       Running   0          1m
example-memcached-6fd7c98d8-g5k7v     1/1       Running   0          1m
example-memcached-6fd7c98d8-m7vn7     1/1       Running   0          1m
memcached-operator-7cc7cfdf86-vvjqk   2/2       Running   0          2m

Additional Ansible Debug TODO(asmacdo) mv development-tips

Occasionally while developing additional debug in the Operator logs is nice to have. Using the memcached operator as an example, we can simply add the "ansible.sdk.operatorframework.io/verbosity" annotation to the Custom Resource with the desired verbosity.

apiVersion: "cache.example.com/v1alpha1"
kind: "Memcached"
metadata:
  name: "example-memcached"
  annotations:
    "ansible.sdk.operatorframework.io/verbosity": "4"
spec:
  size: 4

Update the size

Change the spec.size field in the memcached CR from 3 to 4 and apply the change:

$ cat deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml
apiVersion: "cache.example.com/v1alpha1"
kind: "Memcached"
metadata:
  name: "example-memcached"
spec:
  size: 4

$ kubectl apply -f deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml

Confirm that the operator changes the deployment size:

$ kubectl get deployment
NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
example-memcached    4         4         4            4           5m

Cleanup

Clean up the resources:

$ kubectl delete -f deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml
$ kubectl delete -f deploy/operator.yaml
$ kubectl delete -f deploy/role_binding.yaml
$ kubectl delete -f deploy/role.yaml
$ kubectl delete -f deploy/service_account.yaml
$ kubectl delete -f deploy/crds/cache.example.com_memcacheds_crd.yaml

NOTE Additional CR/CRD's can be added to the project by running, for example, the command :operator-sdk add api --api-version=cache.example.com/v1alpha1 --kind=AppService For more information, refer cli doc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment