pwillis-els

Heap dumping a Java process running in AWS ECS

Note: this guide is designed for AWS ECS services, but starting from Step 4 is functionally equivalent to any Docker container on a Linux host.

Step 1. Look up ECS service

1. Log into the AWS Console using the appropriate AWS account
2. Navigate to AWS ECS service clusters (https://console.aws.amazon.com/ecs/home)
3. Make sure you are in the correct region, if not, switch to the correct region (second drop-down menu in top right corner)
4. Select the correct cluster (ex: https://console.aws.amazon.com/ecs/home?region=us-east-1#/clusters//services)
5. In the Services tab, In the 'Filter in this page' text box, type the name of the service

pwillis-els

#!/bin/sh
# attach_ebs.sh - Attach an EBS volume to an EC2 instance.
# Copyright (C) 2020 Peter Willis <peterwwillis+github@gmail.dotcom>
# 
# This script is designed to create and mount a single EBS volume based on its tag:Name
# in order to implement persistent storage. If there is more than one EBS volume
# with the same tag, this script will fail.
# 
# Order of operations:
#   1. Detect EBS volume based on "tag:Name" "$TAG_NAME"

pwillis-els

Using profiles, assume-role, and Federated SAML authentication with AWS CLI

Let's say you use a Federated authentication method for AWS (like ADFS), and by default you have access to multiple roles and accounts. You want to be able to easily switch between accounts, roles, and even assume a second role after assuming a first one. The following guide explains how this works using [aws-adfs][1] and the [AWS CLI][2].

Background info about profiles

Profiles are how AWS CLI configures the settings for individual credentials, and allows you to switch between them. You can specify a profile either by passing the --profile NAME option to AWS CLI, or with an environment variable AWS_PROFILE=name.

pwillis-els

node('node-label') {
    stage('Checkout') {
        scm checkout
    }

    stage('Tag') {
        sh 'git tag my-tag'
        sh 'sh git-credential-env.sh --install'

        withCredentials([[

pwillis-els

My development environment

I use a couple tools to make it easier for me to get work done:

• cliv. This installs all my typical Ops tools that aren't packaged by my Linux distribution. This also allows me to switch versions of any tool at any time, either by specifying a particular version of a tool, or by pinning a version in a .COMMAND-versions file. No need for tfenv.

• terraformsh. This wrapper for Terraform makes it much easier to manage lots of environments and run common Terraform commands. It's simpler than TerraGrunt and still allows me

pwillis-els

Jenkins: A DevOps Anti-Pattern

Jenkins is the WordPress of CI/CD. Designed in another era, it creates more problems than it needs to and is more complex than it needs to be. But because it’s free and user-friendly, it is ubiquitous and perennial, like a weed. Every year, people will try to use it, unaware of the problems it will create.

The "tl;dr" is that Jenkins was not designed to be used like modern Cloud-native DevOps-friendly software. You can "make it work", in the same sense that you can make pigs fly.... but they're really not designed to fly.

What is Jenkins?

Jenkins is an “automation server”. Basically it’s software that can continuously run automated tasks for you. It has a friendly web-based user interface, and because it’s written in Java, you can run it on any computer. And it has a lot of plugins to add features to do whatever you need.

pwillis-els

Making a Changelog by hand with Git

If you need to create a ChangeLog for a repository, there's a lot of software out there than can help you generate one. But most of it works by having your Git commits include certain information. What if you want to make a ChangeLog for software whose commit messages are not uniform?

Basically, just use git log to compare commits to your existing ChangeLog, and format the Changelog entry for a new release (using the Keep A Changelog format).

First add a shortcut to format Git logs to only show you the changes:

pwillis-els

Kubernetes Administration Best Practice

Reliability

Networking Reliability

• AWS VPC Container Network Interface (CNI) for Kubernetes has an inherent limit on number of pods per instance, due to using one ENI per pod per instance. The workaround is to use Calico CNI. The Calico CNI can be deployed in EKS to run alongside the VPC CNI, providing Kubernetes Network Policies support.

Manifest Reliability

• Lint your K8s manifests as part of a CI process.
• https://www.kubeval.com/

pwillis-els

This is an overview of the different options for multi-tenancy in Kubernetes as of November 2021.

Kubernetes Multi-Tentancy SIG (https://github.com/kubernetes-sigs/multi-tenancy)

• Multi-Tenancy Benchmarks (https://github.com/kubernetes-sigs/multi-tenancy/tree/master/benchmarks)
• Hierarchical Namespace Controller (https://github.com/kubernetes-sigs/hierarchical-namespaces)
  • "You can create additional namespaces under your team's namespace, even if you don't have cluster-level permission to create namespaces, and easily apply policies like RBAC and Network Policies across all namespaces in your team"
• VirtualCluster (https://github.com/kubernetes-sigs/cluster-api-provider-nested/tree/main/virtualcluster)
• *"each tenant is assigned a dedicated tenant control plane, which is a upstream Kubernetes distribution.

pwillis-els

Simple Atomic File Locking in Linux

If you have access to a traditional programming language, there are many methods1 to use2 locks in linux3. However, we don't necessarily have access to those methods within a shell script. In addition, using locks over different kinds of filesystems (such as NFS) can also have inconsistencies and bugs.

What if you just want a very simple form of locking that works on all filesystems? The answer is Maildir locking. The way Qmail / Maildir works is specific to mail files, so I'll break it down in a more general way below. You also don't have to strictly follow this method; the general idea can be modified.