shcallaway

Datadog's reserved log attributes are confusing as heck. It's not clear what each attribute does, so you can’t predict or understand what will happen when you create a mapping. Allow me to demonstrate.

I got this logline from my Datadog S3 archive bucket. It gives you a sense of what logs look like after going through Datadog's opaque transformations.

{
    "_id": "AW5Hc8y8FxIBf2udiA1a", // Log ID generated by Datadog
    "attributes": { // Key-values from the original JSON logline are moved under "attributes"
        "@timestamp": "2019-11-07T19:59:59.804Z",
        "@version": "1",

shcallaway

You can view paginated, colorized JSON in your terminal by piping JSON to jq -C and less -R:

curl https://jsonplaceholder.typicode.com/posts | jq -C | less -R

shcallaway

#!/bin/sh

set -e

# =============================
# ========= Homebrew ==========
# =============================

echo "Running Homebrew..."

shcallaway

#!/bin/bash

TOTAL=$(cat $1 | wc -l)
TOTAL_W_TRACE_ID=$(cat input.csv | awk 'BEGIN { FS = "," } ; { print $5 }' | grep . | wc -l)

echo "Total:               $TOTAL"
echo "Total with trace_id: $TOTAL_W_TRACE_ID"
echo "To calculate a percentage, plug these numbers into something that can do floating point arithmetic."

shcallaway

#!/bin/sh

# Source: https://medium.com/@tannhauser.sphinx/bash-kubernetes-script-to-list-all-available-resources-subresources-c65a5c2c1173

_list=($(kubectl get --raw / |grep "^    \"/api"|sed 's/[",]//g')); 
for _api in ${_list[@]}; do
  _aruyo=$(kubectl get --raw ${_api} | jq .resources); 
  if [ "x${_aruyo}" != "xnull" ]; then 
    echo; 
    echo "===${_api}==="; 

shcallaway

Print a list of Helm releases w/ their statuses. This command references the ConfigMaps directly -- Helm's underlying storage mechanism -- which makes it good for debugging Helm state.

kubectl get cm -o json | jq '.items[] | select(.metadata.labels.OWNER=="TILLER") | .metadata.labels.NAME + " - " + .metadata.labels.STATUS'

shcallaway

If you have some migration jobs in Kubernetes, you can re-run them with this one-liner:

kubectl get jobs | grep migrations | awk '{print $1}' | xargs -I {} sh -c "kubectl get job {} -o json | jq 'del(.spec.selector)' | jq 'del(.spec.template.metadata.labels)' | kubectl replace --force -f -"

Based on this Stack Overflow answer.

shcallaway

#!/bin/bash

echo "Kubernetes Context: $1"
echo "-- LOADBALANCERS --"

LOADBALANCERS=`kubectl get svc --context $1 | grep LoadBalancer | awk '{print $1}'`

for LOADBALANCER in $LOADBALANCERS; do
  echo $LOADBALANCER
  PRIVATE=`kubectl get svc --context $1 $LOADBALANCER -o jsonpath='{.metadata.annotations.service\.beta\.kubernetes\.io\/aws-load-balancer-internal}'`

shcallaway

It's somewhat confusing to provide S3 access to an IAM role from another AWS account. The thing to remember is this: you need to provide access from both sides. In other words, you need to set the bucket resource policy and attach a regular IAM policy to the role.

It's kind of like how, when you want to go to your friend's house, you need permission from both moms. :)

See "Resource-based policies and IAM policies".

shcallaway

kubectl get po -o json | jq '.items[] | select(.metadata.name | test("my-pod")) | .status.podIP'