Skip to content

Instantly share code, notes, and snippets.

Avatar
💭
Adding tremendous value

Steven Acreman sacreman

💭
Adding tremendous value
  • ThunderOps
  • United Kingdom
View GitHub Profile
@sacreman
sacreman / prometheus.yml
Last active Feb 4, 2022
Prometheus configuration to scrape Kubernetes outside the cluster
View prometheus.yml
# Prometheus configuration to scrape Kubernetes outside the cluster
# Change master_ip and api_password to match your master server address and admin password
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
# metrics for the prometheus server
- job_name: 'prometheus'
@sacreman
sacreman / tsdb_blog.md
Last active Oct 12, 2021
Time Series DB Smack Down
View tsdb_blog.md

Databases are a crazy topic and it seems everyone has an opinion. The trouble is that opinions are like belly buttons. Just because everyone has one it doesn’t mean they are useful for anything.

Time series databases (TSDB’s) in particular always provoke the usual “have you tried X”, where X is some obscure project with 50 commits back in 2009. Invariably, if X is something a bit more mainstream then yes, it probably has been played with. It’s probably good at certain things and bad at others like all software.

With all of the above in mind I decided to pen a magnum opus of my own opinions. Something I can point the HaveYouTriedX’ers at next time they make an appearance. So here it is..

My Top10 TSDB’s:

  1. DalmatinerDB (no surprise here)
  2. InfluxDB
View ddb_benchmark.md

Write Performance Benchmark

This document will allow anyone to verify the benchmark result of writing 2 - 3 million metrics per second into DalmatinerDB. This is a single node benchmark to keep things simple and easily comparable between time series databases that don't support clustering.

We will setup 2 Haggar servers to generate metrics and fire them at a single node DalmatinerDB server as per this diagram.

dalmatiner benchmark

You can expect near linear performance results as a DalmatinerDB cluster is horizontally scaled.

@sacreman
sacreman / kubernetes-dashboard.json
Created Aug 6, 2017
Kubernetes Dashboard for Grafana
View kubernetes-dashboard.json
{
"annotations": {
"list": []
},
"description": "Monitors Kubernetes cluster using Prometheus. Shows overall cluster CPU / Memory / Filesystem usage as well as individual pod, containers, systemd services statistics. Uses cAdvisor metrics only.",
"editable": true,
"gnetId": 315,
"graphTooltip": 0,
"hideControls": false,
"id": 2,
@sacreman
sacreman / k8sbackup.sh
Last active Jul 12, 2019
Backup all K8s config to a timestamped location
View k8sbackup.sh
#!/bin/bash -e
BACKUP_DIR="/var/tmp/k8sbackup/$(date +%s)"
echo "Backing up cluster to ${BACKUP_DIR}"
NAMESPACES=$(kubectl get ns -o jsonpath={.items[*].metadata.name})
RESOURCETYPES="${RESOURCETYPES:-"ingress deployment configmap secret svc rc ds networkpolicy statefulset cronjob pvc"}"
GLOBALRESOURCES="${GLOBALRESOURCES:-"namespace storageclass clusterrole clusterrolebinding customresourcedefinition"}"
mkdir -p ${BACKUP_DIR}
@sacreman
sacreman / aks.txt
Created Oct 23, 2018
Dolos Results 2018-10-23
View aks.txt
2018-10-22 08:39:01 INFO Starting test
2018-10-22 08:39:01 INFO Creating Resource Group
2018-10-22 08:39:03 INFO Creating the AKS cluster
2018-10-22 08:52:22 INFO Getting cluster credentials
2018-10-22 08:52:23 INFO Get Nodes
2018-10-22 08:52:26 INFO b'NAME STATUS ROLES AGE VERSION\naks-nodepool1-18093422-0 Ready agent 3m v1.9.11\n'
2018-10-22 08:52:26 INFO Applying Deployment
2018-10-22 08:52:32 INFO b'deployment.apps/azure-vote-back created\nservice/azure-vote-back created\ndeployment.apps/azure-vote-front created\nservice/azure-vote-front created\n'
2018-10-22 08:52:32 INFO Getting external IP
2018-10-22 08:56:18 INFO Getting web contents from 104.41.139.254
View dalmatiner.md

DalmatinerDB Installation Guide (Linux)

These instructions outline how to install DalmatinerDB on a single Linux x86_64 physical server or virtual machine. Scaling out will be covered in a future document. This setup guide also covers configuring CAdvisor and Telegraf to send in monitoring data and Grafana to build dashboards.

Here's how everything connects together:

dalmatiner architecture

Create a VM

View riak collectd config
<URL "http://localhost:8098/stats">
Instance "riak"
<Key "converge_delay_last">
Type "gauge""
</Key>
<Key "converge_delay_max">
Type "gauge""
</Key>
<Key "converge_delay_mean">
Type "gauge""
@sacreman
sacreman / upgrade_python.sh
Created Nov 19, 2014
Centos 6 upgrade to Python 2.7
View upgrade_python.sh
#!/usr/bin/env bash
# install build tools
sudo yum install make automake gcc gcc-c++ kernel-devel git-core -y
# install python 2.7 and change default python symlink
sudo yum install python27-devel -y
sudo rm /usr/bin/python
sudo ln -s /usr/bin/python2.7 /usr/bin/python
View gist:4453493
global
log 127.0.0.1 local2 info
pidfile /var/run/haproxy.pid
stats socket /var/run/haproxy.stat mode 600 level admin
#debug
defaults
mode http
log global