Commands covered:
ls, cd, pwd, find,
touch, mkdir, mv, cp, ln, rm, shred,
chmod, chown, chgrp,
file, iconv, xxd, hexdump,
tar, zip, unzip,
less, cat, tail, head, grep, sed, cut, sort,
jq, xmllint,
halt, poweroff, shutdown, reboot,
netstat, arp, dig, nslookup, traceroute, ping, route, ifconfig, ss,
curl,
du, df, mount, umount, fdisk,
top, htop, ps, kill,
exiftools,id3info,
xdotool,
ffmpeg,
fuck,figlet,toilet,lolcat,cowsay
Use ls to list directory contents, cd to traverse through directories and pwd to find out which directory you’re in.
$ ls -lah
total 61M
drwx------ 57 stasim users 4.0K Apr 5 09:30 .
drwxr-xr-x 3 root root 4.0K Jan 2 15:51 ..
-rw-r--r-- 1 stasim users 1.3K Mar 29 09:39 .bashrc
drwxr-xr-x 47 stasim users 4.0K Mar 28 16:59 .config
drwxr-xr-x 2 stasim users 4.0K Mar 15 09:20 Desktop
drwxr-xr-x 5 stasim users 4.0K Mar 29 14:00 Documents
drwxr-xr-x 11 stasim users 4.0K Apr 4 10:04 Downloads
$ cd Downloads
$ pwd
/home/stasim/Downloads
The find command can be used to find items (files, directories) by name, type, depth and such.
$ find -name '*Zip*.java'
./blah/src/main/java/cz/blah/registration/SkZipCodes.java
$ find . -mindepth 2 -maxdepth 2 -type d -ls
By default the touch command updates the modification time of a file, or creates the file if it does not exist. The mkdir command can be used with flag -p to create the path if needed.
$ ls -l
total 0
-rw-r--r-- 1 simon users 0 Apr 1 08:00 README.md
$ touch LICENCE.md README.md
$ mkdir some_directory1
$ mkdir -p deeper/structure/some_directory2
$ ls -l
total 8
drwxr-xr-x 3 simon users 4096 Apr 4 09:37 deeper
-rw-r--r-- 1 simon users 0 Apr 4 09:37 LICENCE.md
-rw-r--r-- 1 simon users 0 Apr 4 09:37 README.md
drwxr-xr-x 2 simon users 4096 Apr 4 09:37 some_directory1
All moving, copying, and creating symlinks have the same syntax:
$ mv existing_thing new_thing
$ cp existing_thing new_thing
$ ln -s existing_thing new_thing
When removing, be careful what you wish for. The shred commands deletes the file securely by overwriting it with garbage first.
$ rm -rf thing_to_remove
$ shred -u secret_thing
The chmod command is used to change permissions for files and directories. You can either specify the number combination such as 600 which specifies which combination of permissions should be set for user/group/other, or specify which permission to give to or take away from a user/group/others, such as g+w to give a write permission to the owning group. To apply permissions recursively to subdirectories, use -R.
$ chmod g+w -R shared_dir
$ chmod 600 ~/.ssh/id_rsa
Commands chown and chgrp have the same syntax and can be used to change the owner-user and owner-group of a file.
$ chown -R simon /opt/tomcat
$ chgrp -R users /opt/tomcat
You can use the following two commands to add a user to a group. Make sure to use the -a flag (appends instead of replacing).
$ usermod -a -G groupName simon
$ gpasswd -a stasim docker
Use the file command to learn what charset a file is using and iconv to convert a file from one encoding to another.
$ file -i proh6.csv
proh6.csv: text/plain; charset=utf-8
$ iconv -f windows-1250 -t iso-8859–1 -o fixed.csv original.csv
Commands xxd and hexdump show the hex dump of a file (or whatever you pipe to it).
$ xxd kody_bank_CR.csv
00000000: efbb bf4b c3b3 6420 706c 6174 6562 6ec3 ...K..d platebn.
00000010: ad68 6f20 7374 796b 753b 506f 736b 7974 .ho styku;Poskyt
00000020: 6f76 6174 656c 2070 6c61 7465 626e c3ad ovatel platebn..
00000030: 6368 2073 6c75 c5be 6562 3b42 4943 206b ch slu..eb;BIC k
00000040: c3b3 6420 2853 5749 4654 293b 5379 7374 ..d (SWIFT);Syst
00000050: c3a9 6d20 4345 5254 4953 0d0a 3031 3030 ..m CERTIS..0100
00000060: 3b4b 6f6d 6572 c48d 6ec3 ad20 6261 6e6b ;Komer..n.. bank
00000070: 612c 2061 2e73 2e3b 4b4f 4d42 435a 5050 a, a.s.;KOMBCZPP
00000080: 3b41 0d0a 3033 3030 3bc4 8c65 736b 6f73 ;A..0300;..eskos
00000090: 6c6f 7665 6e73 6bc3 a120 6f62 6368 6f64 lovensk.. obchod
000000a0: 6ec3 ad20 6261 6e6b 612c 2061 2e73 2e3b n.. banka, a.s.;
The syntax for tar is infamously impossible to remember. Use c to create an archive or x to extract it, chose z if you want to use gzip (de)compression, use f to specify the archive being created or extracted.
$ tar czf /backup/simon.tar.gz /home/simon
$ tar xzf /backup/simon.tar.gz
Unfortunately tar does not handle .zip files at all, so you’ve got to use zip and unzip to do that.
$ zip myfiles.zip file1 file2 file3
$ zip -r myfiles.zip directory1
$ unzip myfiles.zip
Less is a terminal pager that allows you to navigate through text files or whatever you pipe to it. Use less with -R flag to display raw characters, that is to interpret the ANSI colors.
$ less -R catalina.out
For cat command, the -n flag counts the number of lines and prepends the line number to each line.
$ cat colors.sh
printf " "
for b in 0 1 2 3 4 5 6 7; do printf " 4${b}m "; done
echo
for f in "" 30 31 32 33 34 35 36 37; do
for s in "" "1;"; do
printf "%4sm" "${s}${f}"
printf " \033[%sm%s\033[0m" "$s$f" "gYw "
for b in 0 1 2 3 4 5 6 7; do
printf " \033[4%s;%sm%s\033[0m" "$b" "$s$f" " gYw "
done
echo
done
done
$ cat -n colors.sh
1 printf " "
2 for b in 0 1 2 3 4 5 6 7; do printf " 4${b}m "; done
3 echo
4 for f in "" 30 31 32 33 34 35 36 37; do
5 for s in "" "1;"; do
6 printf "%4sm" "${s}${f}"
7 printf " \033[%sm%s\033[0m" "$s$f" "gYw "
8 for b in 0 1 2 3 4 5 6 7; do
9 printf " \033[4%s;%sm%s\03...
10 done
11 echo
12 done
13 done
For head and tail, the -n flag says how many lines should it display from the beginning or end of file (or piped input) respectively. If left out, it will show 10 lines, which is the default.
$ head -n 2 colors.sh
printf " "
for b in 0 1 2 3 4 5 6 7; do printf " 4${b}m "; done
$ tail -n 2 colors.sh
done
done
$ tail colors.sh
for f in "" 30 31 32 33 34 35 36 37; do
for s in "" "1;"; do
printf "%4sm" "${s}${f}"
printf " \033[%sm%s\033[0m" "$s$f" "gYw "
for b in 0 1 2 3 4 5 6 7; do
printf " \033[4%s;%sm%s\033[0m" "$b" "$s$f" " gYw "
done
echo
done
done
To monitor log files and such, use the tail command with -f flag to follow new lines as the file grows. You can also use the shorthand tailf that does the same trick or tail -F which works for rotating logs as well.
$ tail -f catalina.out
Definitely one of the most used commands, grep is used to filter lines piped to it based on whether they match the given pattern. Use -v flag to reverse the filter, -x to have an exact match or -i for the filter to be case-insensitive. Use -e if the search term is a regexp.
$ ls
prague.html prague.html.txt prague.txt trondheim.html trondheim.txt
$ ls | grep prague
prague.html prague.html.txt prague.txt
$ ls | grep -v html
prague.txt trondheim.txt
$ ls | grep prague.html
prague.html prague.html.txt
$ ls | grep -x prague.html
prague.html
$ ls | grep praguE
$ ls | grep -i praguE
prague.html prague.html.txt prague.txt
Use cut with -d and -f flags to cut the input into columns by delimiter specified by -d and columns specified by -f. To specify column, you can enumerate them or specify ranges. For example -f1,3–5,8- will show first, third to fifth and eight and further columns.
$ cut -d':' -f1 /etc/passwd
root
bin
daemon
mail
ftp
...
Use cut with -c instead of -f to do the same with characters instead of fields.
$ cut -c1-5 /etc/passwd
root:
bin:x
daemo
mail:
ftp:x
Use sort to sort input lexicographically. Use -r flag to reverse the order. To sort numbers, use -n or -h (for human readable numbers such as 3K and 1M). Use -M to sort by month names (JAN to DEC).
$ cut -d':' -f1 /etc/passwd | sort
avahi
bin
colord
daemon
dbus
Use sed to replace by regular expressions.
$ cat names
Stastny,Simon
Poirot,Hercules
$ sed 's/^\(.*\),\(.*\)$/\2\ \1/g' names
Simon Stastny
Hercules Poirot
You can use jq to extract data from json using a selector or do some other cool tricks. Further reading: https://shapeshed.com/jq-json
$ echo '{"name":"simon","born":{"city":"prague" }}' | jq .born.city
"prague"
$ echo '["simon", "world"]' | jq 'map("hello " + .)'
[
"hello simon",
"hello world"
]
Use xmllint to format xml or select using an xpath expression. Specify file or let is read from standard input (remember to use dash instead of filename).
$ cat colors.html | xmllint --format -
<?xml version="1.0"?>
<!DOCTYPE html>
<html>
<head>
<title/>
...
$ cat colors.html | xmllint --xpath "//div[@id='term']" -
<div id="term">
<font color="#282C34">#282C34</font>
<font color="#2C2F33">#2C2F33</font>
<font color="#56B6C2">#56B6C2</font>
...
Pipe something to source-highlight to get it highlighted.
cat xml.xml | source-highlight -s xml -f esc
You can also use a source highlighter to color source code and then display it with less -R. Just put this in your .bashrc (remember to install source-highlight and set the path right):
export LESSOPEN="| /usr/bin/src-hilite-lesspipe.sh %s"
export LESS=" -R "
Use xclip to copy/paste from/to standard input/output.
$ uptime | xclip -selection c
$ xclip -o -selection c
17:15:00 up 7:55, 1 user, load average: 0.63, 0.74, 1.12
Use watch to run a command in regular intervals (which can be specified using the -n flag (default is 2 seconds).
$ watch -n 5 'docker ps'
To run a command in background, append &. To run a command that will not die after you log out or close the terminal, use nohup (which stands for no hang up).
$ nohup dostuff &
When piping standard output to a program that does not work with standard IO, but with arguments, you can use xargs to redirect standard input as arguments of a program. This example deletes all filed in a directory by piping the output of ls (directory listing) as arguments to rm.
$ ls | rm
rm: missing operand
Try 'rm --help' for more information.
$ ls | xargs rm
While clear clears your terminal, you are still able to scroll up. To completely reset the terminal, use reset.
All of these can be used to “power off” your computer:
$ halt
$ shutdown -H now
$ poweroff
$ shutdown -p now
Check this great answer on stack exchange (https://www.binarytides.com/linux-command-shutdown-reboot-restart-system/) to learn the technical differences, check this article on Binary Tides (https://unix.stackexchange.com/questions/205464/whats-the-difference-between-poweroff-and-halt) to see how to use the shutdown command.
To restart your computer, use either:
$ reboot
$ shutdown -r now
Tools dig and nslookup help you look up IP address for a given hostname
$ nslookup google.com
Server: 127.0.0.53
Address: 127.0.0.53#53
Non-authoritative answer:
Name: google.com
Address: 142.251.36.110
Name: google.com
Address: 2a00:1450:4014:80b::200e
$ dig google.com
; <<>> DiG 9.18.10 <<>> google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63744
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 9
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;google.com. IN A
;; ANSWER SECTION:
google.com. 277 IN A 142.251.36.110
;; AUTHORITY SECTION:
google.com. 277 IN NS ns2.google.com.
google.com. 277 IN NS ns4.google.com.
google.com. 277 IN NS ns3.google.com.
google.com. 277 IN NS ns1.google.com.
;; ADDITIONAL SECTION:
ns2.google.com. 277 IN AAAA 2001:4860:4802:34::a
ns3.google.com. 277 IN AAAA 2001:4860:4802:36::a
ns4.google.com. 277 IN A 216.239.38.10
ns1.google.com. 277 IN AAAA 2001:4860:4802:32::a
ns4.google.com. 277 IN AAAA 2001:4860:4802:38::a
ns2.google.com. 277 IN A 216.239.34.10
ns3.google.com. 277 IN A 216.239.36.10
ns1.google.com. 277 IN A 216.239.32.10
;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
;; WHEN: Fri Mar 03 18:17:41 CET 2023
;; MSG SIZE rcvd: 303
The netstat command shows all programs listening on ports...
$ sudo netstat -tlpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:36053 0.0.0.0:* LISTEN 2315/java
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 322/cupsd
tcp 0 0 127.0.0.1:42877 0.0.0.0:* LISTEN 2315/java
tcp 0 0 127.0.0.1:6942 0.0.0.0:* LISTEN 2315/java
tcp 0 0 127.0.0.1:63342 0.0.0.0:* LISTEN 2315/java
tcp6 0 0 ::1:631 :::* LISTEN 322/cupsd
tcp6 0 0 :::61306 :::* LISTEN 2542/java
tcp6 0 0 :::40905 :::* LISTEN 2542/java
tcp6 0 0 :::8080 :::* LISTEN 11126/node
...or open connections...
$ netstat -ant
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:36053 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:42877 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:6942 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:63342 0.0.0.0:* LISTEN
tcp 327 0 10.50.51.34:34416 94.31.29.138:443 ESTABLISHED
tcp 0 0 10.50.51.34:40730 64.233.167.189:443 ESTABLISHED
tcp 0 0 127.0.0.1:36053 127.0.0.1:43152 ESTABLISHED
tcp 0 0 10.50.51.34:46880 13.32.113.58:443 ESTABLISHED
tcp 0 0 10.50.51.34:34812 34.192.63.110:443 ESTABLISHED
tcp 1 0 10.50.51.34:45532 10.240.122.21:80 CLOSE_WAIT
...or a routing table (same can be done with route command)...
$ netstat -r
Kernel IP routing table
Destination Gateway Genmask Flags Window irtt Iface
default 10.50.51.1 0.0.0.0 UG 0 0 enp0s31f6
default 10.50.7.1 0.0.0.0 UG 0 0 wlp4s0
10.50.7.0 0.0.0.0 255.255.255.0 U 0 0 wlp4s0
10.50.51.0 0.0.0.0 255.255.255.0 U 0 0 enp0s31f6
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 docker0
Or you can use ip to get route to particular address
$ ip route get 142.251.36.110
142.251.36.110 via 192.168.1.1 dev enp0s31f6 src 192.168.1.9 uid 1000
Then traceroute to trace the route
$ traceroute to 142.251.36.110 (142.251.36.110), 30 hops max, 60 byte packets
1 _gateway (192.168.1.1) 0.960 ms 4.947 ms 4.918 ms
2 10.38.199.1 (10.38.199.1) 4.881 ms 4.839 ms 4.802 ms
3 10.38.6.250 (10.38.6.250) 138.311 ms 138.275 ms 138.236 ms
4 10.38.254.252 (10.38.254.252) 138.171 ms 138.110 ms 138.066 ms
5 core.ttc2.jmnetworks.cz (185.91.168.27) 6.988 ms 7.234 ms 7.453 ms
6 google.peering.cz (91.213.211.170) 7.124 ms 3.609 ms 2.260 ms
7 108.170.245.33 (108.170.245.33) 2.429 ms 2.643 ms 2.560 ms
8 142.251.224.125 (142.251.224.125) 2.746 ms 142.251.224.123 (142.251.224.123) 4.182 ms 142.251.224.125 (142.251.224.125) 4.392 ms
9 prg03s11-in-f14.1e100.net (142.251.36.110) 4.644 ms 4.567 ms 4.473 ms
And you can use ping to measure ping to a given IP address (as long as it supports ICMP)
ping 8.8.8.8
While arp shows you hardware adresses
$ arp
Address HWtype HWaddress Flags Mask Iface
10.50.51.1 ether 5c:f3:fc:4e:21:d8 C enp0s31f6
10.50.51.4 ether 5c:f3:fc:4e:21:d8 C enp0s31f6
10.50.7.87 ether 0c:8b:fd:cb:b2:ea C wlp4s0
And iwgetid can print information about the network ou are connected to
$ iwgetid -r
feg-employee
create a curl_format.txt file with the following contents
time_namelookup: %{time_namelookup}s\n
time_connect: %{time_connect}s\n
time_appconnect: %{time_appconnect}s\n
time_pretransfer: %{time_pretransfer}s\n
time_redirect: %{time_redirect}s\n
time_starttransfer: %{time_starttransfer}s\n
----------\n
time_total: %{time_total}s\n
And pass it to the curl command to get timing details about the request and response.
curl -w "@tools/curl_format.txt" -o /dev/null -s https://blabla
$ df -h
Filesystem Size Used Avail Use% Mounted on
dev 5.6G 0 5.6G 0% /dev
run 5.6G 1.5M 5.6G 1% /run
/dev/sda2 114G 44G 65G 41% /
tmpfs 5.6G 179M 5.5G 4% /dev/shm
tmpfs 5.6G 0 5.6G 0% /sys/fs/cgroup
/dev/sda1 243M 47M 181M 21% /boot
tmpfs 5.6G 228M 5.4G 4% /tmp
tmpfs 1.2G 16K 1.2G 1% /run/user/120
tmpfs 1.2G 1.8M 1.2G 1% /run/user/1000
$ du -sh workspace/
2.9G workspace/
Use mount to list mounted filesystems and with arguments device and mountpoint (where in the system is the filesystem going to be accesible) to mount filesystems from devices. Use umount to unmount the filesystems.
$ mount -t ext4
/dev/sda2 on / type ext4 (rw,noatime,discard,data=ordered)
/dev/sda1 on /boot type ext4 (rw,noatime,discard,data=ordered)
$ mount -t vfat /dev/sdb1 /media/flashdisk
$ umount /media/flashdisk
Use fdisk with the -l flag to list disks and partitions. Use it without the flag and with the device as an argument to enter the command mode where you can change the partition table.
$ sudo fdisk -l
[sudo] password for stasim:
Disk /dev/sda: 119.2 GiB, 128035676160 bytes, 250069680 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x99edec50
Device Boot Start End Sectors Size Id Type
/dev/sda1 * 2048 524287 522240 255M 83 Linux
/dev/sda2 524288 243124223 242599936 115.7G 83 Linux
/dev/sda3 243124224 250069679 6945456 3.3G f W95 Ext'd
/dev/sda5 243126272 250069679 6943408 3.3G 82 Linux swap
$ sudo fdisk /dev/sdc
Use top or htop to monitor CPU and memory usage by processes. Use ps to see which processes are running on your system. Use kill -9 and PID as an argument to kill a process.
$ ps
PID TTY TIME CMD
17864 pts/1 00:00:00 bash
18137 pts/1 00:00:00 watch
18148 pts/1 00:00:00 ps
$ kill -9 18137
[1]+ Killed nohup watch 'ps'
exiftool 20210606085350_IMG_3032.JPG
exiftool "-DateTimeOriginal=2021:06:06 08:53:50.69+02:00" 20210606085350_IMG_3032.JPG
id3info 09_gratulacni_cranberries.mp3
find . -iname '*.mp3' -exec id3v2 --TIT3 "Updated tag!" '{}' \;
ffmpeg -i input.mp4 -b 800k output.mp4
xdotool getmouselocation
xdotool mousemove 3500 750
xdotool click 1
xdotool key ctrl+alt+n
xdotool type foobar
And even in case you screw up typing pretty much any command, swearing helps if you have thefuck package installed.
$ git push
fatal: The current branch f/L3-14 has no upstream branch.
To push the current branch and set the remote as upstream, use
git push --set-upstream origin f/L3-14
$ fuck
git push --set-upstream origin feature/L3-14 [enter/↑/↓/ctrl+c]
Total 0 (delta 0), reused 0 (delta 0)
To ssh://******************************/repo/git/WEB/livethree
* [new branch] f/L3-14 -> f/L3-14
Branch f/L3-14 set up to track remote branch f/L3-14 from origin.
watch -n 1 --color 'ls -la | \
grep dc-live3api-5-vx9jl.hprof | \
sed "s/.*simon\(.*\)Oct.*/\1/g" | \
figlet -f ~/Downloads/3d.flf | \
lolcat'
Enjoy.