Skip to content

Instantly share code, notes, and snippets.

View JPvRiel's full-sized avatar

Jean-Pierre van Riel JPvRiel

5 followers · 6 following
  • South Africa, Johannesburg
View GitHub Profile
@JPvRiel
JPvRiel / 0_reuse_code.js
Created August 19, 2016 20:11
Here are some things you can do with Gists in GistBox.
// Use Gists to store code you would like to remember later on
console.log(window); // log the "window" object to the console
@JPvRiel
JPvRiel / apt_pinning_priorities.md
Last active April 22, 2024 19:42
Apt package pinning and priorities
@JPvRiel
JPvRiel / remove-docker-containers-and-untaged-images.md
Last active February 14, 2023 13:36 — forked from ngpestelos/remove-docker-containers.md
How to remove unused docker containers and images (cleanup)

Delete all containers

$ docker ps -q -a | xargs docker rm
  • -q prints only the container IDs
  • -a prints all containers

Notice that it uses xargs to issue a remove container command for each container ID

@JPvRiel
JPvRiel / ubunut_network_manager_docker_dns_workaround.md
Last active July 18, 2022 23:09
Ubuntu, NetworkManager and Docker DNS workaround

Docker issues are frequently logged for DNS resolution in containers because it doens't inhert or get values for DNS from NetworkManager which leverages a built in dnsmasq to inteligently manage DNS.

Perminant workarround

sudo bash -c "echo listen-address=$(ip -4 addr show dev docker0 | grep -oP '(?<=inet\s)\d+(\.\d+){3}') > /etc/NetworkManager/dnsmasq.d/docker-bridge"
sudo systemctl reload NetworkManager
sudo bash -c 'echo -e "{\n\t\"dns\": [\"$(ip -4 addr show dev docker0 | grep -oP "(?<=inet\s)\d+(\.\d+){3}")\"]\n}" > /etc/docker/daemon.json'
sudo systemctl restart docker
@JPvRiel
JPvRiel / bash_string_substitution.md
Last active October 10, 2016 23:10
Bash String Substitution

Syntax

${varName/Pattern/Replacement}

Given a variable

$ val="test"
@JPvRiel
JPvRiel / bash_regex_match_groups.md
Last active October 16, 2023 15:28
Bash regular expression match with groups including example to parse http_proxy environment variable

The newer versions of bash (>= 3.0) include a regex operator =~

Simple example

$ re='t(es)t'
$ [[ "test" =~ $re ]]
$ echo $?
0
$ echo ${BASH_REMATCH[1]}
es
@JPvRiel
JPvRiel / bash_replace_newlines_string_substitution.md
Last active June 15, 2023 00:24
Replacing newlines with commas (or other text) using pure bash string substitution (not awk).

In this example

  • Multiline input can be caputured by spreading openening and closing quote accross lines
  • Newlines can be replaced using built-in bash text substitution
  • The main trick is knowing that $'\n' is the way to specify the newling within the subsitution
  • Note, echo output of a var without quotation replaces newlines with spaces
$ t='1
> 2
@JPvRiel
JPvRiel / ubuntu_mirror_selection.md
Last active April 11, 2023 00:49
Ubuntu mirror selection and package downloads for South Africa using nmap and curl to manually test latency and throughput

A quick set of notes looking into Ubuntu mirror locations for South Africa from my home fiber (Vumatel). The examples should apply in general.

Debian Package Info and Downloads

There are two phases

  1. Getting lists of packages
  2. The actual package download

When testing performance of package list info downloads via upt-get update, clear previous package info downloaded. If this isn't done, cached package info could skew results.

@JPvRiel
JPvRiel / linux_memory_control_to_avoid_swap_thrashing.md
Created November 7, 2016 22:29
Notes on linux memory management options to prioritize and control memory access using older ulimits, newer cgroups and overcommit policy settings. Mostly as an attempt to keep a desktop environment responsive and avoid swap thrashing under high memory pressure.

Overview

Some notes about:

  • Explaining why current day Linux memory swap thrashing still happens (as of 2016).
  • Mitigating "stop the world" type thrashing issues on a Linux workstation when it's under high memory pressure and where responsiveness is more important than process completion.
  • Prioritizing and limiting memory use.
  • Older ulimit versus newer CGroup options.

These notes assume some basic background knowledge about memory management, ulimits and cgroups.

@JPvRiel
JPvRiel / ssllabs_cert_test_web_scrape.py
Created November 27, 2016 21:11
A web scraping example using python. It used ssllabs.com to test a list of web sites and scraped the results. This was only useful back in 2015 before they published an API
# Python script to test ssl via ssllabs service
# - input: space or newline delimited set of hostnames to test
# - process: sends request, polls till results are complete
# - output: saves html page
# - output: prints summary results per site in csv
# - output format: <fqdn/hostname>,<http status code>,<overall rating>,<certficate score %>,<protocol support score %>,<key exchange score %>,<cipher strength score %>
# python libs to use
import csv
from lxml import html