Skip to content

Instantly share code, notes, and snippets.

View JPvRiel's full-sized avatar

Jean-Pierre van Riel JPvRiel

  • South Africa, Johannesburg
View GitHub Profile
@JPvRiel
JPvRiel / apt_pinning_priorities.md
Last active April 22, 2024 19:42
Apt package pinning and priorities
@JPvRiel
JPvRiel / SysRq magic keys.md
Created January 9, 2019 02:26
SysRq magic keys

sysrq magic keys

This is useful for trying to recover from an unresponsive system. However, by default, Ubuntu disables the key to gain raw keyboard access.\

Sometimes part of the issue is that X/Wayland display servers have crashed due to bad graphics drivers and keyboard input isn't processed anymore, whereby changing to another console, e.g. ctrl + alt + F1 doesn't work. Since the default disables keyboard control, alt + sysrq + r, can't help salvage the situation. Hence the suggestion to at least enable system request raw keyboard control when trying to recover a system.

Ubuntu doesn't enable control of keyboard by default

Ubuntu 16.04 LTS default value:

@JPvRiel
JPvRiel / linux_memory_control_to_avoid_swap_thrashing.md
Created November 7, 2016 22:29
Notes on linux memory management options to prioritize and control memory access using older ulimits, newer cgroups and overcommit policy settings. Mostly as an attempt to keep a desktop environment responsive and avoid swap thrashing under high memory pressure.

Overview

Some notes about:

  • Explaining why current day Linux memory swap thrashing still happens (as of 2016).
  • Mitigating "stop the world" type thrashing issues on a Linux workstation when it's under high memory pressure and where responsiveness is more important than process completion.
  • Prioritizing and limiting memory use.
  • Older ulimit versus newer CGroup options.

These notes assume some basic background knowledge about memory management, ulimits and cgroups.

@JPvRiel
JPvRiel / journalctl_enable_persistent_storage.md
Last active January 24, 2024 20:23
Enable persistent storage for the systemd journal log

Enable persistent storage for the systemd journal log

Overview

The assumed default setting in /etc/systemd/journald.conf is Storage=auto which implies that systemd journaling will only persist the journal if the expected storage location is available. Otherwise, the journal data is stored in memory and lost between reboots. On Ubuntu 16.04, /var/log/journal does not exist by default. Create it to keep and query events from previous boots.

Considerations:

  • Syslog still provides the persistant log records for Ubuntu 16.04, so enabling persistant systemd journal logging does cause a level of duplicaiton.
  • There are sane defaults:
@JPvRiel
JPvRiel / bash_regex_match_groups.md
Last active October 16, 2023 15:28
Bash regular expression match with groups including example to parse http_proxy environment variable

The newer versions of bash (>= 3.0) include a regex operator =~

Simple example

$ re='t(es)t'
$ [[ "test" =~ $re ]]
$ echo $?
0
$ echo ${BASH_REMATCH[1]}
es
@JPvRiel
JPvRiel / bash_history_to_syslog.md
Last active August 29, 2023 09:11
Notes on (ab)using bash history to record commands to syslog

Logging bash history to syslog

Overview

Bash history was a convenience feature to help a user recall previous commands and not intended to meet any security requirements.

The Linux audit system (or alternate kernel level audit OS facility) is a more robust way to ensure user and process log events are recorded.

Security issues with bash history files and $BASH_COMMAND

@JPvRiel
JPvRiel / bash_replace_newlines_string_substitution.md
Last active June 15, 2023 00:24
Replacing newlines with commas (or other text) using pure bash string substitution (not awk).

In this example

  • Multiline input can be caputured by spreading openening and closing quote accross lines
  • Newlines can be replaced using built-in bash text substitution
  • The main trick is knowing that $'\n' is the way to specify the newling within the subsitution
  • Note, echo output of a var without quotation replaces newlines with spaces
$ t='1
> 2
@JPvRiel
JPvRiel / gnome_proxy_to_env.md
Last active May 24, 2023 12:15
a shell wrapper to pull org.gnome.proxy settings into env, e.g. http_proxy, which is useful for .desktop files

Why?

Useful for:

  • updating current terminal's shell env proxy settings, or
  • wrapping exec in .desktop files to inject env proxy settings

GNOME proxy settings should normally get propergated into the shell (bash) environment via gnome-terminal, e.g. http_proxy and no_proxy. However:

  • I noticed that applications exectuted via .desktop entries sometimes don't work. This happens when an app ignores org.gnome.proxy settings but can often use proxy env vars from the shell.
  • If you're proxy settings change, existing shell processes will still have the old proxy enviroment variables, so this can help refresh them whithout having to create a new shell.
@JPvRiel
JPvRiel / ubuntu_mirror_selection.md
Last active April 11, 2023 00:49
Ubuntu mirror selection and package downloads for South Africa using nmap and curl to manually test latency and throughput

A quick set of notes looking into Ubuntu mirror locations for South Africa from my home fiber (Vumatel). The examples should apply in general.

Debian Package Info and Downloads

There are two phases

  1. Getting lists of packages
  2. The actual package download

When testing performance of package list info downloads via upt-get update, clear previous package info downloaded. If this isn't done, cached package info could skew results.

@JPvRiel
JPvRiel / remove-docker-containers-and-untaged-images.md
Last active February 14, 2023 13:36 — forked from ngpestelos/remove-docker-containers.md
How to remove unused docker containers and images (cleanup)

Delete all containers

$ docker ps -q -a | xargs docker rm
  • -q prints only the container IDs
  • -a prints all containers

Notice that it uses xargs to issue a remove container command for each container ID