Skip to content

Instantly share code, notes, and snippets.

@mowings
mowings / status.yml
Created Jun 21, 2020
evicted kubernetes pod status
View status.yml
status:
phase: Failed
message: 'The node was low on resource: ephemeral-storage. Container resque-web was using 72Ki, which exceeds its request of 0. Container resque-worker was using 60Ki, which exceeds its request of 0. Container nginx was using 88Ki, which exceeds its request of 0. Container app was using 72Ki, which exceeds its request of 0. '
reason: Evicted
startTime: '2020-06-19T13:14:22Z'
@mowings
mowings / database.yml
Last active Jun 21, 2020
postgres tcp keepalive setting in rails. May be needed for kubernetes rails deployments, especially sneakers
View database.yml
production:
url: <%= DATABASE_URL %>
variables:
tcp_keepalives_idle: 300
tcp_keepalives_interval: 20
tcp_keepalives_count: 20
@mowings
mowings / wait.sh
Created May 18, 2020
bash one-line wait
View wait.sh
while sleep 15; do echo Sleeping...; done
@mowings
mowings / collectd.conf
Created May 1, 2020
General collectd.conf config
View collectd.conf
FQDNLookup false
Interval 10
Timeout 2
ReadThreads 5
WriteThreads 5
LoadPlugin cpu
LoadPlugin memory
LoadPlugin load
LoadPlugin disk
@mowings
mowings / addkeys.yml
Created Jan 27, 2020
Ansible -- add a bunch of public keys in a directory to a remote user
View addkeys.yml
- name: Add keys to hosts
hosts: all
become: true
tasks:
- name: add keys
authorized_key:
user: ubuntu
state: present
key: "{{item}}"
with_file:
@mowings
mowings / readme.md
Created Jan 24, 2020
fio disk latency benchmark
View readme.md
fio --name=sequential_write_iops_test --filename=testfile --filesize=10G --time_based --ramp_time=2s \
   --runtime=1m --ioengine=libaio --direct=1 --verify=0 --randrepeat=0 --bs=4k \
   --rw=write --group_reporting --iodepth=2 --numjobs=8
@mowings
mowings / etcd_data_dir.md
Last active Jan 24, 2020
Change etcd data dir on running kubernetes master node
View etcd_data_dir.md

This works as of kubernetes 1.16.4

I use a symlink rather than changing the config in /etc/kubernetes/manifests. You can do it either way.

First get the id of the running etcd container:

# docker ps | grep etcd
b006a4f993d0 .... # Your id will be different, obv.
@mowings
mowings / readme.md
Last active Jan 20, 2020
kuberetes etcd metrics or api calls with curl
View readme.md
curl --cacert /etc/kubernetes/pki/etcd/ca.crt \
    --cert /etc/kubernetes/pki/etcd/peer.crt \
    --key /etc/kubernetes/pki/etcd/peer.key \
    https://localhost:2379/metrics
@mowings
mowings / readme.txt
Last active Jan 3, 2020
Remove all kubernetes iptables rules/chains
View readme.txt
iptables-save | grep -v KUBE | iptables-restore # maybe
@mowings
mowings / readme.md
Last active Dec 31, 2019
Add a new kubernetes master node to an existing HA cluster
View readme.md

In this example, we alreadyhave an HA cluster using an API proxy at 10.98.83.105:6443

On a working master node, generate and upload new control plane certs. These will be good for two hours. Take note of the cert key:

sudo kubeadm init phase upload-certs --upload-certs
 
[upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
[upload-certs] Using certificate key:
50a7a77ddad29d95f8a7fb3249ebea01ec50e38d9aae64a8d8d3669276b71783
You can’t perform that action at this time.