bcdurden

Simple way to merge yaml files using yq

Unbeknownst to me, yq possesses the ability to merge whole yaml files on top of one another in order. I had been working around this ignorance using scriptlets that would set values as they were discovered while trapsing a file, so I usually avoided doing it altogether. Many of you know I am NOT a fan of complex scripting.

But after doodling around a bit with yq, I found the reduce/ireduce commands and was able to condense down the ordered/layered merging of N yaml files using a very simple yq one liner.

Simple and effective:

This was on yq version 4.44.3, so the command may differ on older versions as I think the ireduce command is more recent.

bcdurden

function create_project() {
  project_name=$1
  URL=$2
  PASSWORD=$3

  code=$(curl -s -o /dev/null -w "%{http_code}" -u "admin:$PASSWORD" -X HEAD $URL/api/v2.0/projects?project_name=${project_name})
  if [[ "$code" -eq 404 ]]; then
    # create project
    code=$(jq '.project_name = "'${project_name}'"' project_template.json | curl -s -o /tmp/result -w "%{http_code}" -u "admin:$PASSWORD" -H 'accept: application/json' -H 'Content-Type: application/json' --data-binary @- -X POST $URL/api/v2.0/projects)
    if [[ "$code" -eq 201 ]]; then

bcdurden

Generating Deterministic Names with Harvester Cloud Credentials in Rancher

One of the issues with automating much of Rancher MCM deployments that is commonly run into by prospective DevOps-focused folks is that Rancher lazily uses guid-based name generation for many of its resources.

When automating Rancher on Harvester, this is compounded a bit because of Harvester's nature: It is a Kubernetes cluster and so it's API is via the kube-apiserver. Rancher offers tight integration options with Harvester that isn't possible on other infrastructures. But the emergent issue is the cloud credential created in Rancher that references a specific Harvester cluster has a generated name. Any automation that depends on this name has to be updated in kind whenever that credential changes (or is initially created).

From an automation standpoint that's a show-stopper in my opinion. Thankfully, Harvester exposes this functionality through its own API. Below is the secret sauce to explain how to do this. I will also expla

bcdurden

RKE2 and Packer

These files are configured in a way so that an Ubuntu cloud-image is modified by downloading the RKE2 install script from upstream as well as installing the qemu-guest-agent. This is done so the Ubuntu image can now function in an airgapped environment as an RKE2 node. Previous methods I've done involved using libguestfs tools and it was a bit clunky due to how it needed to be managed. Packer's QEMU provider fixes that for me.

Unfortunately, Packer's QEMU provider must run locally as there is no Harvester provider that would run these commands on a remote Harvester cluster to save us the dependency problem. Perhaps in the future we can explore that level of sophistication, but for now this works great.

There is a post-install provisioner that uploads the resulting image to Harvester using a VirtualMachineImage CR template. If you do not wish to upload to Harvester, feel free to comment out that section in spec.pkr.hcl, it is located towards the bottom and starts with the lines `post-p

bcdurden

Removing the CPU reservtion in the Harvester/Longhorn default configuration

The values to change are guaranteed-engine-manager-cpu and guaranteed-replica-manager-cpu within the longhorn-default-setting ConfigMap in the longhorn-system namespace.

Manually Editing this is annoying because the value is an embedded yaml string within a ConfigMap. This can be done with a two-command copy-pasta. Ensure your kube-context is pointed at your harvester cluster. This may not work within the kubectl shell in the Harvester UI as I don't know if it has jq installed.

VALUES=$(kubectl get cm longhorn-default-setting -n longhorn-system -o jsonpath="{.data['default-setting\.yaml']}" | while read key value; do
if [ $key = "guaranteed-engine-manager-cpu:" ]; then 
    printf "$key 0\n" 
else 

bcdurden

Keybase proof

I hereby claim:

• I am bcdurden on github.
• I am bcdurden (https://keybase.io/bcdurden) on keybase.
• I have a public key ASC0_2i_s1KXykoV7EV9HSQem5nRiLtQhQadHvt9F1neDAo

To claim this, I am signing this object: