Skip to content

Instantly share code, notes, and snippets.

View joemiller's full-sized avatar

joe miller joemiller

View GitHub Profile

Loki log ingestion issue

Loki stops ingesting logs from promtail. The error messages are a stream HTTP 499 errors in the gateway component which is based on nginx. Ex:

10.194.148.169 - - [17/Feb/2022:21:08:28 +0000]  499 "POST /loki/api/v1/push HTTP/1.1" 0 "-" "promtail/" "-"
10.194.90.195 - - [17/Feb/2022:21:08:28 +0000]  499 "POST /loki/api/v1/push HTTP/1.1" 0 "-" "promtail/" "-"
@joemiller
joemiller / he-dns-update.sh
Last active September 4, 2024 18:46
script for updating dynamic DNS records on he.net (hurricane electric)
#!/bin/bash
#
# Script for updating DNS records on Hurricane Electirc's DNS system (https://dns.he.net).
#
# The record will be updated with the IP address that originates the request.
#
# Usage
# -----
#
# Create config file `/etc/he-dns-update.conf`:
@joemiller
joemiller / Dockerfile
Last active August 26, 2024 07:39
convert RFC 3164 syslog messages to RFC 5424 for ingestion into Loki/promtail
FROM balabit/syslog-ng:3.35.1
COPY syslog-ng.conf /etc/syslog-ng/syslog-ng.conf
@joemiller
joemiller / mk-test-certs.sh
Created December 29, 2015 22:32
helper script for making a new CA and signing client (leaf) certs, including making java keystore (JKS) files. useful in creating test fixtures
#!/bin/sh
# helper script for making a new CA and signing client (leaf) certs, including making java keystore (JKS) files. useful in creating test fixtures
#
# Example:
# ./mk-test-certs.sh
# ==> Creating new CA: certs/ca.key, certs/ca.crt
# Generating a 2048 bit RSA private key
# ....................................................+++
# ...............................+++
# writing new private key to 'certs/ca.key'
@joemiller
joemiller / netpps.sh
Last active January 12, 2024 15:39
shell: quick linux scripts for showing network bandwidth or packets-per-second
#!/bin/bash
if [ -z "$1" ]; then
echo
echo usage: $0 network-interface
echo
echo e.g. $0 eth0
echo
echo shows packets-per-second
@joemiller
joemiller / gke-np-ip-utilization.rb
Last active January 6, 2024 01:05
GKE node-pool pod IP CIDR range usage calc script
#!/usr/bin/env ruby
#
# Show GKE per-node-pool pod IP CIDR range utilization.
#
# Usage:
# gcloud container clusters describe <cluster-name> --location <loc> --format=json | ./gke-np-ip-utilization.rb
require 'json'
require 'ipaddr'
@joemiller
joemiller / raid_ephemeral.sh
Last active October 23, 2023 21:53
detect all ephemeral disks on EC2 then stripe together in a raid-0 vol mounted at /mnt
#!/bin/bash
#
# this script will attempt to detect any ephemeral drives on an EC2 node and create a RAID-0 stripe
# mounted at /mnt. It should be run early on the first boot of the system.
#
# Beware, This script is NOT fully idempotent.
#
METADATA_URL_BASE="http://169.254.169.254/2012-01-12"
@joemiller
joemiller / 71.patch
Created October 20, 2022 18:55
openbsd 7.1 kernel patch to stop a GFE L6F ACPI interrupt storm
--- /usr/src/sys/dev/acpi/acpi.c.orig Mon Oct 3 16:26:55 2022
+++ /usr/src/sys/dev/acpi/acpi.c Mon Oct 3 16:30:29 2022
@@ -2269,6 +2269,18 @@
{
struct aml_node *node = arg;
uint8_t mask, en;
+
+ /* bad bios. mask/ignore the GPE _L6F (0x6f) interrupt */
+ if (gpe == 0x6f && (sc->gpe_table[gpe].flags & GPE_LEVEL)) {
+ static unsigned short i;
@joemiller
joemiller / auto_shutdown.service
Last active May 12, 2023 15:44
bash script + systemd units for auto-shutting down an idle machine (no active ssh logins)
[Unit]
Description=Auto shutdown service
[Service]
Type=oneshot
ExecStart=/auto_shutdown.sh
@joemiller
joemiller / README.md
Last active April 7, 2023 15:13
Buildkite OIDC JWT example with Vault auth

Get JWT from inside of a running BK job:

curl -s -X POST -H "Authorization: Token ${BUILDKITE_AGENT_ACCESS_TOKEN}" \
  "${BUILDKITE_AGENT_ENDPOINT:-https://agent.buildkite.com/v3}/jobs/${BUILDKITE_JOB_ID}/oidc/tokens" \
  --data '{"audience":"vault"}'

In a future, currently (2022/11/17) unreleased version of buildkite-agent you can run this instead: