Skip to content

Instantly share code, notes, and snippets.

View mrballcb's full-sized avatar

Todd Lyons mrballcb

12 followers · 7 following
View GitHub Profile
@mrballcb
mrballcb / gist:9fb358d23b2c303837f3b35461159819
Last active July 2, 2019 16:34
terraform aws_api_gateway_method_response documentation error
$ TF_LOG=debug terraform plan
2019/07/02 09:33:53 [INFO] Terraform version: 0.12.3
2019/07/02 09:33:53 [INFO] Go runtime version: go1.12.4
2019/07/02 09:33:53 [INFO] CLI args: []string{"/usr/local/bin/terraform", "plan"}
2019/07/02 09:33:53 [DEBUG] Attempting to open CLI config file: /Users/tlyons/.terraformrc
2019/07/02 09:33:53 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019/07/02 09:33:53 [INFO] CLI command args: []string{"plan"}
Error: Invalid resource name
@mrballcb
mrballcb / coreos_systemd_units.txt
Last active June 6, 2019 15:25
CoreOS ami various modifications using kops
# The CoreOS ami is opinionated (IMHO, and is not a bad thing). There are two opinions it projects that are of interest:
# 1) automatically download updates, apply, and reboot nodes (with locking in etcd to prevent too many from doing it at once)
# 2) enable selinux by default, but in permissive mode.
# The first can cause problems if your app is not totally written with cloud concepts in mind.
# The second can cause problems if you have a really large EBS volume with a lot of files getting mounted by Kube because the mount
# enables selinux, which then performs a recursive relabeling of the volume when it mounts. You should always leave selinux on, but
# if you have a specific case where this is a problem, evaluate your options and your liability.
# You can define both of these in your kops cluster definition or in a specific instanceGroup definition.
@mrballcb
mrballcb / rolling-updates-2x-IG.txt
Created April 19, 2019 23:16
kops 2X IG rolling-update (from ReactiveOps medium.com article)
I don't advise to just blindly copy and paste. Try things one line at a time.
Make sure you understand what they are supposed to be doing.
0. # Set versions
kops get cluster
CLUSTER=full_cluster_name
kops edit cluster --name $CLUSTER # set kube version
kops get ig --name $CLUSTER
kops edit ig --name $CLUSTER first_master # if updating ami
kops edit ig --name $CLUSTER second_master # if updating ami
@mrballcb
mrballcb / Jenkinsfile
Created March 14, 2019 14:47
Jenkinsfile for building docker images
def OUR_DOCKER = "docker.example.com"
// Requires leading slash, no trailing slash
def PROJECT_URL = "/logging/logstash"
def pod_label = "buildpod-${env.JOB_NAME.reverse().take(35).reverse()}-${env.BUILD_NUMBER}".replace('_', '-').replace('/', '-')
podTemplate(
label: pod_label,
containers: [
containerTemplate(name: 'docker-builder', image: "${OUR_DOCKER}/helpers/docker-builder", ttyEnabled: true, command: 'cat')
@mrballcb
mrballcb / gist:c1a8ff4132224e654e85aad80f3a0fec
Created March 6, 2019 14:59
Using Fluentd to convert Kubernetes Jenkins dynamic builder label to save ElasticSearch indexes
I started with the fluentd-elasticsearch docker image at:
https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch
I added to the Gemfile:
gem 'fluent-plugin-record-modifier', '~>1.1.0'
and built my own docker image.
Then I made a helm chart from the official (but now deprecated) fluentd-elasticsearch helm chart at:
https://github.com/helm/charts/tree/master/stable/fluentd-elasticsearch
@mrballcb
mrballcb / kops_update.log
Created August 30, 2018 18:11
kops update, no kubedns modifications
I0830 09:47:18.103466 91357 factory.go:68] state store s3://BUCKET
I0830 09:47:18.860757 91357 s3context.go:198] Checking default bucket encryption "BUCKET"
I0830 09:47:18.860808 91357 s3context.go:203] Calling S3 GetBucketEncryption Bucket="BUCKET"
I0830 09:47:19.340191 91357 s3context.go:210] Unable to read bucket encryption policy: will encrypt using AES256
I0830 09:47:19.340210 91357 s3context.go:182] Found bucket "BUCKET" in region "us-east-1" with default encryption set to false
I0830 09:47:19.340226 91357 s3fs.go:219] Reading file "s3://BUCKET/foo/config"
I0830 09:47:19.474990 91357 s3fs.go:256] Listing objects in S3 bucket "BUCKET" with prefix "foo/instancegroup/"
I0830 09:47:19.597008 91357 s3fs.go:284] Listed files in s3://BUCKET/foo/instancegroup: [s3://BUCKET/foo/instancegroup/bastions s3://BUCKET/foo/instancegroup/debian_nodes s3://BUCKET/foo/instancegroup/master-us-east-1c s3://BUCKET/foo/instancegroup/master-us-east-1d s3://BUCKET/foo/instancegroup/master-us-east-1e s3://BUCKET/
@mrballcb
mrballcb / daemonset.yaml
Created August 20, 2018 11:21
Helm daemonset yaml template for pre-pulling images.
apiVersion: apps/v1beta2
kind: DaemonSet
metadata:
name: {{ .Release.Name }}-prepull
labels:
app: {{ template "name" . }}
env: {{ .Values.environment }}
chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
@mrballcb
mrballcb / Jenkinsfile
Last active June 18, 2019 16:10
Sample Jenkinsfile - There are external library functions used here but not shown
def label = "buildpod.${env.JOB_NAME.replace('%2F','_').reverse().take(38).reverse()}.${env.BUILD_NUMBER}".replace('-', '_').replace('/', '_')
def buildCloudName = "foo.cloud"
def flatBranchName = BRANCH_NAME.replace("/", "-")
def CHECKOUTPREFIX = "CHART"
def CHARTPATH = "${CHECKOUTPREFIX}/chart"
podTemplate(
label: label,
cloud: buildCloudName,
@mrballcb
mrballcb / gist:6733184bb4d5aebd8841e746f00a0030
Created January 19, 2018 06:17
kops rolling-update --interactive, answered no
~/repos/kubernetes-kops tlyons(⎈ |iad1:default) (test_1_8_0_interactive_cli) $ $KOPS rolling-update cluster sjc1.cloud.net --interactive --yes --node-interval 3m0s --force --instance-group nodes
NAME STATUS NEEDUPDATE READY MIN MAX NODES
nodes Ready 0 2 2 20 2
I0118 22:11:38.598151 49094 instancegroups.go:158] Draining the node: "ip-10-1-61-33.us-west-1.compute.internal".
node "ip-10-1-61-33.us-west-1.compute.internal" cordoned
node "ip-10-1-61-33.us-west-1.compute.internal" cordoned
WARNING: Ignoring DaemonSet-managed pods: canal-19pmc; Deleting pods not managed by ReplicationController, ReplicaSet, Job, DaemonSet or StatefulSet: kube-proxy-ip-10-1-61-33.us-west-1.compute.internal
node "ip-10-1-61-33.us-west-1.compute.internal" drained
I0118 22:13:08.908090 49094 instancegroups.go:275] Stopping instance "i-04176bbac40527d44", node "ip-10-1-61-33.us-west-1.compute.internal", in group "nodes.sjc1.cloud.net".
I0118 22:16:09.202630 49094 instancegroups.go:189] Validating the c
@mrballcb
mrballcb / gist:7899624679cc45c16e6d396ece9e688e
Created January 19, 2018 06:10
kops rolling-update --interactive, default (not yes, no, nor Alwaysyes)
~/repos/kubernetes-kops tlyons(⎈ |iad1:default) (test_1_8_0_interactive_cli) $ $KOPS rolling-update cluster sjc1.cloud.net --interactive --yes --node-interval 3m0s
NAME STATUS NEEDUPDATE READY MIN MAX NODES
master-us-west-1a-1 Ready 0 1 1 1 1
master-us-west-1a-2 Ready 0 1 1 1 1
master-us-west-1b-1 Ready 0 1 1 1 1
nodes NeedsUpdate 2 0 2 20 2
I0118 21:58:37.724453 47095 instancegroups.go:158] Draining the node: "ip-10-1-61-118.us-west-1.compute.internal".
node "ip-10-1-61-118.us-west-1.compute.internal" cordoned
node "ip-10-1-61-118.us-west-1.compute.internal" cordoned
WARNING: Deleting pods not managed by ReplicationController, ReplicaSet, Job, DaemonSet or StatefulSet: kube-proxy-ip-10-1-61-118.us-west-1.compute.internal; Ignoring DaemonSet-managed pods: canal-40x89