Skip to content

Instantly share code, notes, and snippets.

View JPvRiel's full-sized avatar

Jean-Pierre van Riel JPvRiel

5 followers · 6 following
  • South Africa, Johannesburg
View GitHub Profile
@JPvRiel
JPvRiel / deb_apt_dpkg_locked.md
Created February 16, 2017 10:37
Troubleshoot "Could not get lock /var/lib/dpkg/lock" errors when trying to update or install debian packages

Troubleshoot "Could not get lock /var/lib/dpkg/lock" errors when trying to update or install debian packages

Error Message

E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable)
E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
@JPvRiel
JPvRiel / ubunut_network_manager_docker_dns_workaround.md
Last active July 18, 2022 23:09
Ubuntu, NetworkManager and Docker DNS workaround

Docker issues are frequently logged for DNS resolution in containers because it doens't inhert or get values for DNS from NetworkManager which leverages a built in dnsmasq to inteligently manage DNS.

Perminant workarround

sudo bash -c "echo listen-address=$(ip -4 addr show dev docker0 | grep -oP '(?<=inet\s)\d+(\.\d+){3}') > /etc/NetworkManager/dnsmasq.d/docker-bridge"
sudo systemctl reload NetworkManager
sudo bash -c 'echo -e "{\n\t\"dns\": [\"$(ip -4 addr show dev docker0 | grep -oP "(?<=inet\s)\d+(\.\d+){3}")\"]\n}" > /etc/docker/daemon.json'
sudo systemctl restart docker
@JPvRiel
JPvRiel / README.md
Last active October 10, 2022 17:27
Decode syslog priority

Decode syslog priority

Well-formed syslog message headers start with the facility and severity encoded as the priority in angle brackets, e.g. <179>.... As per RFC5424:

The Priority value is calculated by first multiplying the Facility number by 8 and then adding the numerical value of the Severity.

The python files in this gist are a demo to help showcase how to decode the priority. E.g.:

pri = 179
@JPvRiel
JPvRiel / Convert old weakly encrypted SSH private keys to the newer PKCS8 format.md
Last active November 24, 2022 14:05
Convert Old Weakly Encrypted SSH Private Keys to the Newer PKCS8 Format

Convert Old Weakly Encrypted SSH Private Keys to the Newer PKCS8 Format

The default password-based encryption for openSSH RSA private keys (generated with ssh-keygen -t rsa) has inadequate protection against brute forcing due to a weak Password-Based Key Derivation Function (PBKDF) because it used OpenSSL's key derivation with just one round of MD5.

TL;DR: Rather generate new keys with ssh-keygen -t ed25519 which by default should use the more secure PBKDF with the PKCS8 format. RSA key generation defaults to the weak PBKDF stored in the PEM format.

However, if you still need RSA to work with older Unix systems, network devices, and other systems' SSH servers are not yet upgraded with Ed25519 support, this provides info to avoid the less secure default of saving private keys in the legacy PEM format when generating RSA keys. Backward compatibility is not that much of a concern since only the SSH client needs a reasonably modern version of SSH.

Identify if the old PEM format is in use

@JPvRiel
JPvRiel / remove-docker-containers-and-untaged-images.md
Last active February 14, 2023 13:36 — forked from ngpestelos/remove-docker-containers.md
How to remove unused docker containers and images (cleanup)

Delete all containers

$ docker ps -q -a | xargs docker rm
  • -q prints only the container IDs
  • -a prints all containers

Notice that it uses xargs to issue a remove container command for each container ID

@JPvRiel
JPvRiel / ubuntu_mirror_selection.md
Last active April 11, 2023 00:49
Ubuntu mirror selection and package downloads for South Africa using nmap and curl to manually test latency and throughput

A quick set of notes looking into Ubuntu mirror locations for South Africa from my home fiber (Vumatel). The examples should apply in general.

Debian Package Info and Downloads

There are two phases

  1. Getting lists of packages
  2. The actual package download

When testing performance of package list info downloads via upt-get update, clear previous package info downloaded. If this isn't done, cached package info could skew results.

@JPvRiel
JPvRiel / gnome_proxy_to_env.md
Last active May 24, 2023 12:15
a shell wrapper to pull org.gnome.proxy settings into env, e.g. http_proxy, which is useful for .desktop files

Why?

Useful for:

  • updating current terminal's shell env proxy settings, or
  • wrapping exec in .desktop files to inject env proxy settings

GNOME proxy settings should normally get propergated into the shell (bash) environment via gnome-terminal, e.g. http_proxy and no_proxy. However:

  • I noticed that applications exectuted via .desktop entries sometimes don't work. This happens when an app ignores org.gnome.proxy settings but can often use proxy env vars from the shell.
  • If you're proxy settings change, existing shell processes will still have the old proxy enviroment variables, so this can help refresh them whithout having to create a new shell.
@JPvRiel
JPvRiel / bash_replace_newlines_string_substitution.md
Last active June 15, 2023 00:24
Replacing newlines with commas (or other text) using pure bash string substitution (not awk).

In this example

  • Multiline input can be caputured by spreading openening and closing quote accross lines
  • Newlines can be replaced using built-in bash text substitution
  • The main trick is knowing that $'\n' is the way to specify the newling within the subsitution
  • Note, echo output of a var without quotation replaces newlines with spaces
$ t='1
> 2
@JPvRiel
JPvRiel / bash_history_to_syslog.md
Last active August 29, 2023 09:11
Notes on (ab)using bash history to record commands to syslog

Logging bash history to syslog

Overview

Bash history was a convenience feature to help a user recall previous commands and not intended to meet any security requirements.

The Linux audit system (or alternate kernel level audit OS facility) is a more robust way to ensure user and process log events are recorded.

Security issues with bash history files and $BASH_COMMAND

@JPvRiel
JPvRiel / bash_regex_match_groups.md
Last active October 16, 2023 15:28
Bash regular expression match with groups including example to parse http_proxy environment variable

The newer versions of bash (>= 3.0) include a regex operator =~

Simple example

$ re='t(es)t'
$ [[ "test" =~ $re ]]
$ echo $?
0
$ echo ${BASH_REMATCH[1]}
es