Skip to content

Instantly share code, notes, and snippets.



Last active Jul 27, 2021
What would you like to do?
Various oneliners from various languages/tools



  • Basic for loop to iterate over lines in a file:
for pkg in $(cat pkgs.txt); do sudo apt purge "$pkg" -y; done
  • More complex for loop using if statement to control actions, clean up output etc:
for node in $(cat nodes.txt); do echo "Node: ${node}"; ssh -q -t "$node" 'if [[ $(lsblk | grep -i lvm) ]]; then sudo apt install mdadm -y; fi'; done
  • Checking very busy log files for their contents without having your console hang (as opposed to using tail -f):
watch -n 0.5 sudo tail /var/log/named/queries.log
  • Alternative conditional logic in for loop iterating over array variable:
# Declare 'nodes' variable separately or prepend to loop and separate with semicolon
for node in "${nodes[@]}"; do ping -c 2 -W 0.1 "$node" > /dev/null && echo "OK: ${node}" || echo "NOT OK: ${node}"; done
  • Use while loop to iterate over lines in a file:
    • Avoids calls to cat as is the case with the for loop example
# Using 'madison' command rather than 'policy' seems to be slightly faster
while read pkg; do if [[ $(apt-cache madison "$pkg") ]]; then echo "OK: ${pkg} exists in some repo"; else echo "NOT OK: ${pkg} doesn't exist in any repo"; fi; done < pkgs.txt
  • Match lines into an array:
types=($(grep -oE 'pattern' input.txt))
  • Grab block of text between two patterns:
sed -n '/pattern1/,/pattern2/p' input.txt
  • Just see octal permissions for a file or directory:
stat -c '%a' /etc/passwd
  • Grab last character from string:
  • Parse file list from output of grep into subsequent commands:
grep -rl '\-\- MARK \-\-' /var/log/* | while read line; do echo "Working with file '${line}'"; grep MARK "$line" | tail -n1; done
  • Include lines before and after a grep match:
grep -B 3 -A 3 -i "hv_fcopy" /var/log/messages
  • Find all unique directories in listed directories that contain files modified 10 minutes ago since the command was ran:
ls | xargs -I {} find {} -type f -mmin -10 | cut -d "/" -f2 | sort -u
  • Find all files in the current directories that were modified at least a minute ago, are larger than 500MB, and long list them:
find . -type f -mmin -1 -size +500M -exec ls -lsh {} \;
  • Find all files in the current directories that were modified at least a day ago, are larger than 2GB, and empty their contents:
find . -type f -mtime -1 -size +2G -exec bash -c 'echo > {}' \;
  • Run arbitrary command against a list of directories:
ls | xargs -I {} git -C {} pull
  • Place the following somewhere in your shell script for easier step-by-step debugging; move ahead with Enter key:
set -x
trap read debug
  • Change timezone interactively:
dpkg-reconfigure tzdata
  • Search binary file that looks like text while ignoring case:
grep -ai "end:" /var/log/syslog
  • Count time, calls, and errors for each system call when performing a directory listing:
strace -c ls test/
  • Add to script to determine which line number the execution is at:
echo "DEBUG: ${LINENO}"
  • Remove duplicated lines from a file without messing up the order:
awk '!visited[$0]++' your_file > deduplicated_file
  • Run local script on a remote endpoint using SSH:
ssh -q <username>@<endpoint> "sudo bash -s" <
  • Create new directory and change right into it:
# Oneliner
mkdir new_directory && cd $_

# Alias
mkcd () {
  mkdir "$1"
  cd "$1"
  • Recall argument to last used command:
Alt + .
  • Get SSH key fingerprint:
# SHA-256
ssh-keygen -lf ~/.ssh/

# MD5
ssh-keygen -E md5 -lf ~/.ssh/
  • Find broken symbolic links in current directory:
find . -xtype l
  • Bulk fix relative symbolic links:
find . -lname '<relative-to-source target>*' -exec sh -c 'ln -sfn "<new relative-to-source target>/$(basename $0)" $0' {} \;
  • Run remote script on remote endpoint using SSH:
ssh -q <username>@<endpoint> './location/to/script'
  • Run local script on remote endpoint using SSH:
ssh -q <username>@<endpoint> 'sudo bash -s' < ./location/to/local/script
  • Create ISO from directory without truncating long names (-l) and by not replacing hyphens with underscores -iso-level 4:
genisoimage -o data.iso -iso-level 4 -R -l data/
  • List ISO file contents without having to mount it:
isoinfo -l -i data.iso
  • Simple colouring for log files, both static and running output:
cat test.log | perl -pe 's/^\[\*\].*/\e[0;36m$&\e[0m/g; s/^\[\+\].*/\e[0;32m$&\e[0m/g; s/^\[\!\].*/\e[0;31m$&\e[0m/g'
  • In situations such as these a Bash environment variable will suppress warnings that clog output:
export PYTHONWARNINGS='ignore'
  • Remove last column in string based on delimiter:
$ string='my_underscored_string_12345'
$ echo "$string" | rev | cut -d '_' -f 2- | rev
  • Prefix aliased command with backslash to avoid triggering alias:
$ halt -p
$ alias halt
alias halt='echo "REALLY!?"'
$ \halt -p
Connection to closed by remote host.
  • Pretty print CSV files:
function pretty_csv {
    perl -pe 's/((?<=,)|(?<=^)),/ ,/g;' "$@" | column -t -s, | less  -F -S -X -K

$ pretty_csv data.csv
$ pretty_csv < data.csv
$ sort data.csv | pretty_csv
  • Pretty print TSV files:
function pretty_tsv {
    perl -pe 's/((?<=\t)|(?<=^))\t/ \t/g;' "$@" | column -t -s $'\t' | less  -F -S -X -K

$ pretty_tsv data.tsv
$ pretty_tsv < data.tsv
$ sort data.tsv | pretty_tsv
  • Diff two files and save unified output to file:
diff -u file1 file2 > files.diff
  • Show build information for cloud-based image:
$ cat /etc/cloud/ 
build_name: server
serial: 20201211.1
  • Show top disk usage and exclude certain directories under root:
du -Sh / --exclude=/{proc,sys,dev,var} | sort -rh | head -n 10
  • It's possible to use the built-in : as a short-hand for an infinite loop:
while :; do "looping"; done
  • Re-execute a shell to 'unsource' variables and aliases:
exec /bin/bash
  • Use binary version of time instead of shell built-in to get access to more information:
$(which time) --verbose echo "test"
  • Use perf stat to easily perform repeated executions of a command and measure it in various ways:
perf stat --null --repeat 5 --table echo "test"
  • Visual editing in Vim can be used to delete any number of lines:
Shift+V from point of cursor and press D key to delete
  • Search and replace in Vim:
  • Change nested key value in an array of JSON objects:
.parameters.vmObjects.value |= map(if .vmName == "router" then .moduleSnapshot = "fixed" else . end)
  • Use indirect references to use dynamic variable names:
for host in "${hosts[@]}"; do
  declare "parent_disk_${host}=$parent_disk"

for host in "${hosts[@]}"; do
  echo "${!parent_disk}"
  • Test terminal's colors:
msgcat --color=test
  • Disable search highlight in Vim:
  • Bulk rename files in place:
find . -type f -name '<file name>' -execdir mv {} "description.txt" \;
  • Encode with Base64 on a single line:
echo "text" | base64 -w 0
  • Convert PEM to single-line:
awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' combined.pem
  • Install requirements for Poetry using existing requirements.txt:
cat requirements.txt | xargs poetry add
  • Direct standard output to a file in a directory that might not yet exist:
echo "something" | install -D /dev/stdin directory/file.txt


  • Push existing repository to new remote:
git remote add <name of new remote> <HTTPS or SSH URL>
git push <name of new remote> master
  • Pretty-print branch graph:
git log --all --decorate --oneline --graph
  • Move master back X amount of commits:
git checkout master
git reset --hard <old_commit_id>
git push -f origin master
  • Replay changes on master to some other branch:
# Beware of blindly accepting any incoming changes in favor of your own
git checkout master
git pull
git checkout different_branch
git rebase -Xtheirs master
git push --force
  • Show changed files in specified commit hash:
git diff-tree --no-commit-id --name-only <commit hash>
  • Create patch file from diff:
git diff file.json > file.patch
  • Create patch file from commit:
git show <commit hash> > commit.patch
  • Apply patch file:
git apply commit.patch
  • Bulk create patch files from individual files when running git diff in a repository:
OLDIFS=$IFS; IFS=';' blocks=$(git diff | sed -n '/diff/,/(diff|$)/ {/diff / s/^/\;/; p}'); for block in ${blocks#;}; do echo "$block" > $(echo "$block" | head -n 1 | rev | cut -d "/" -f 1 | rev).patch; done; IFS=$OLDIFS
  • Show diff of stashed hunk:
git stash show -p [stash@{N}]
  • Bulk create separate stashes of every changed file with a message equaling the filename:
git status -s | cut -d " " -f 3 | xargs -I {} git stash push {} -m "{}"
  • Pop every entry from the stash back to the working tree:
git stash list | cut -d ":" -f 1 | xargs -I {} git stash pop
  • Move unpushed commits to a new branch:
# Pull latest changes from 'origin/master' if haven't already
git checkout -b new_branch
git checkout master
git reset --hard origin/master
  • Copy commit to current branch:
git cherry-pick <commit hash>
  • Undo pushed commit that nobody has yet pulled:
git reset HEAD^ --hard
git push --force origin
  • View history of specific function in file:
git log -L :<function>:<file>
  • Speed up Git for larger repositories:
git config feature.manyFiles 1


  • Parse lines from an arguments file to separate parameters for building:
podman build -t foo $(while IFS= read -r line; do args+="--build-arg ${line} "; done < <(cat .arg); echo "$args"; unset args) .
  • Remove all 'exited' containers:
podman rm $(podman ps --all -q -f status=exited)
  • Build and run container based on Dockerfile in current context:
podman build -t foo . && podman run --rm -it foo
  • Prune everything that shouldn't exist anymore without any confirmation:
podman system prune -a -f
  • Remove all images except latest:
podman images | grep -v "latest" | tail -n +2 | awk '{ print $3 }' | xargs --no-run-if-empty podman rmi
  • Possible improvement when executing RUN within a Dockerfile:
    • Benefit is that when a specific line fails, then the error message is much more concise as opposed to the standard method of using ampersands
RUN set -eu; \
    python3 -m venv venv; \
    venv/bin/pip install -r requirements.txt; \
    venv/bin/pip install -r requirements-dev.txt; \
    echo 'Venv creation + requirements installation: OK';
  • Remove dangling <none>:<none> images:
docker rmi $(docker images -f "dangling=true" -q)


  • Run role against arbitrary host:
# Note the comma after the IP or FQDN
# Additional hosts can be added by supplying '-i' parameter with more arguments (comma at the end only if count == 1)
ansible-playbook -i '<IP or FQDN>,' -u '<user name>' --extra-vars 'ansible_winrm_server_cert_validation=ignore ansible_connection=winrm ansible_winrm_transport=credssp ansible_password=<password>' --tags '<tag value for a role>' playbook.yml
  • Run ad-hoc command against arbitrary host:
# Replace the final 'all' with a more precise host pattern if you passed more than one IP or FQDN to the initial list (comma at the end only if count == 1)
ansible -i '<IP or FQDN>,' -u '<user name>' --extra-vars 'ansible_winrm_server_cert_validation=ignore ansible_connection=winrm ansible_winrm_transport=credssp ansible_password=<password>' -m 'win_shell' -a 'ipconfig' 'all'
  • Add timing information to playbook execution output:
$ cat ansible.cfg
callback_whitelist = profile_tasks
  • Make verbose output more readable by using YAML instead of JSON:
  • Debug variables without running entire playbook:
ansible -m debug <host> -a "var=hostvars[inventory_hostname].<variable>"


  • Test simple journal scraping configuration:
$ cat /etc/promtail/config.yml
  http_listen_port: 9080
  grpc_listen_port: 0

  filename: /var/log/positions.yaml

  - url: https://<token>@<host>/loki/api/v1/push

  - job_name: journal
      max_age: 12h
        job: systemd-journal
        static_label: label-value
      - source_labels: ['__journal__systemd_unit']
        target_label: 'unit'

$ promtail --dry-run --config.file /etc/promtail/config.yml
2021-05-20T03:48:54     {job="systemd-journal", static_label="label-value"}     Stopped target Default.
2021-05-20T03:48:54     {job="systemd-journal", static_label="label-value"}     Stopped target Basic System.
2021-05-20T03:48:54     {job="systemd-journal", static_label="label-value"}     Stopped target Timers.
2021-05-20T03:48:54     {job="systemd-journal", static_label="label-value"}     Stopped target Paths.
2021-05-20T03:48:54     {job="systemd-journal", static_label="label-value"}     Stopped target Sockets.


  • Get today's date and time where time is set to midnight:
  • Show list of logged on users:
query user /server:$SERVER
  • Log off user by specifying session ID:
logoff <session ID>
  • Reload a local module:
Import-Module -Name .\module.psd1 -Force
  • Pretty-print minified JSON:
$String | ConvertFrom-Json | ConvertTo-Json -Depth 100
  • Convert from Base64:
  • Convert string to boolean:
  • Resolve FQDN:


  • Colored prompt with Git branch for terminal (e.g. ConEmu):
PS1='${debian_chroot:+($debian_chroot)}\[\033[01;35m\] \w\[\033[01;33m\]$(__git_ps1)\[\033[01;m\] > '


  • Query in JMESPath website: [?tags.currently-used == 'False' || tags.currently_used == 'False'].name
    • Same query in jpterm: [?tags."currently-used" == 'False' || tags."currently_used" == 'False']
    • Same query in console: "[?tags.\"currently-used\" == 'False' || tags.\"currently_used\" == 'False'].name"

This comment has been minimized.

Copy link

@andreasvirkus andreasvirkus commented Aug 13, 2018

Strip leading slash from path

echo "/usr/home/www/site/app/vendor" \| sed -e 's/^\///'

Strip trailing slash from path

echo "/usr/home/www/site/app/vendor/" \| sed -e 's#/$##'

This comment has been minimized.

Copy link
Owner Author

@usrme usrme commented Apr 7, 2021

@andreasvirkus, better late that never, but it's way easier to use shell parameter expansion:

  1. strip leading slash:
$ str="/usr/home/www/site/app/vendor/"
$ echo "${str#/}"
  1. strip trailing slash:
$ str="/usr/home/www/site/app/vendor/"
$ echo "${str%/}"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment