Skip to content

Instantly share code, notes, and snippets.

@jonnyjava

jonnyjava/Getting started with linux.md

Last active February 21, 2023 10:49
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jonnyjava/f0a473c2b546b36cf0a7 to your computer and use it in GitHub Desktop.
Save jonnyjava/f0a473c2b546b36cf0a7 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Getting started with linux.md

GETTING STARTED WITH LINUX

1 BASIC COMMANDS

Return to top

To chain more than one command use ; or &&

  • cal
  • date
  • df show free space of each partition
    • -h show sizes in human readable format
  • free show free space on disk
    • -h show sizes in human readable format
  • filefilename shows the MIMEtype of a file
  • ls
  • -a show hiddens
  • -d show only directories
  • -F classify (if it ends with a / is a directory) l alone does the same
  • -h show sizes in human readable format
  • -l long format
  • -r reverse order
  • -S sort by size
  • -t sort by time
  • type display the command type
  • which display executable location
  • mancommand display help for the command
  • aproposcommand list of man pages for given command
  • whatiscommand display short command description
  • infocommand the same of man
  • alias list all the aliases
  • alias alias_name='command chain' There are not spaces around =
  • unalias alias_name destroy alias
  • history shows the list of the most recent commands with an history number, !history_number executes that command
  • alt+> show the first command in history
  • alt+< show the last command in history

1.1 Where stuff is located

Return to top

/ The system root

/bin System binaries

/boot Linux kernel

/dev List of connected devices

/etc System configuration files and scripts

/home Each users has his folder here

/lib System shared libraries

/lost +found recovery files when crash happens

/media Mounting point of external devices

/opt Optional software installed by users

/proc Kernel stuff

/sbin System binaries

/tmp Temporary files

/usr Users stuff

/usr/bin, /usr/lib, /usr/local, /usr/sbin Stuff belonging to user's programs

/usr/share, /usr/share/doc Shared user datas and user programs libs

/var Databases and logs are all here

2 PERMISSIONS

Return to top

There are 10 symbols. The first one is for the type the number (d/L/c/b). The number 2,3 and 4 are the owner's permissions, the 5, 6 and 7 are for the group, 8, 9 and 10 are the permissions of the rest of the world. r means read, w write, x execute.

Value Symbols
0 ---
1 --x
2 -w-
3 -wx
4 r--
5 r-x
6 rw-
7 rwx

Another way to change permission is the symbolic notation:

  • u user
  • g group
  • o world
  • a all (u+g+o)
  • + add permission
  • - remove permission
  • = set permission

2.1 Commands

Return to top

  • id show the user identity and his groups
  • chmod change mode
  • umask set default permissions
  • su run as another user (for that session of the shell)
  • su -c command execute a single command as another user (the same of sudo)
  • sudo run as another user (for single command)
  • chgrp change group
  • chown change owner.
  • chown ownername [:[array of groups]] arguments permits to set user and groups in one shot
  • passwd

3 FILES AND FOLDERS MANIPULATION

Return to top

3.1 Wildcards

Return to top

* means any char

? one char

[expression] set of chars [!expression] the ! means exclude

Expression can be ranges or classes of chars. Classes are:

  • [:alnum:] a-z A-Z 0-9
  • [:alpha:] a-z A-Z
  • [:digit:] 0-9
  • [:lower:] a-z
  • [:upper:] A-Z

3.2 Commands

Return to top

mkdir create directory ln create symbolic link

cp copy

  • -a copy owner and permissions too
  • -i interactive copy (otherwise it s overwrites silently)
  • -r recursive copy
  • -u update (copy or overwrite only new or younger)
  • -v verbose

mv move or rename

  • -i interactive (otherwise it s overwrites silently)
  • -r recursive
  • -u update (only new or younger)

rm remove files or directories

  • -i interactive (otherwise it removes without confirmation)
  • -r recursive
  • -f force
  • -v verbose

Remember! There is no UNDO for rm!

4 REDIRECTIONS

Return to top

  • cat concats params in output destination
  • sort sort lines in output
  • uniq remove duplicates from output
  • -d only duplicates
  • grep any string or regexp apply that as filter
  • -i ignore case
  • -v reverse matching
  • wc count words, lines and bytes of the output
  • head print from the beginning of the output
    • -n specify number of lines (default 10)
  • tail print from the end of the output
    • -n specify number of lines (default 10)
  • tee get from specified input and write into specified output
  • > write from the beginning
  • >> append here
  • 2> write to standar error output
  • &> destination Redirects file (STDERR) and (STDOUT) to destination
  • &>/dev/null Redirects file (STDERR) and (STDOUT) to `/dev/null´. This basically means to ignore output.

5 THE SHELL

Return to top

5.1 Expressions

Return to top

$((expression)) where expression is any arithmetic calculation

{range_start..range_end} prints all the combinations (it works with reversed ranges too)

An uppercased name preceded by $ is a system variable

Is possible to use double quotes in a command to ignore whitespaces. What is contained between single quotes is considered plain text \ to escape a single caracter \b is a backspace \n is a newline \r is the carriage return \t is a tab

5.2 Keyboard tricks

Return to top

  • ctrl+l clear screen
  • ctrl+t transpose actual char with previous one
  • alt+t transponse actual word with previous one
  • alt+l lowercase from cursor to end
  • alt+u uppercase from cursor to end
  • ctrl+k erase line
  • ctrl+u erase from cursor to the beginning of the line
  • alt+d erase from the cursor to the end of the line
  • alt+backspace erase last word
  • double tab display all the possible completions

6 PROCESSES AND MEMORY

Return to top

  • ps snapshot of running processes. R means process running, S sleeping, D unstoppable sleep, T stopped, Z zombie, N low priority, < high priority process

  • aux shows USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND

  • top display running tasks in real time and 5 lines containing the summary

    1. uptime, number of users and average load for queued processes
    2. task situations
    3. CPU state
    4. amout of KU in use
    5. SWAP (virtual memory) in use

  • jobs display running jobs

  • bg place a job in background

  • fgl place a job in foreground

  • kill send a signal to a process

    • 1 HUP hangup (the dial connection)
    • 2 INT interrupt (as ctrl+c)
    • 9 KILL kill the process without performing any cleanups
    • 15 TERM terminate process
    • 18 CONT resume stopped process
    • 19 STOP pause process
    • 3 QUIT
    • 11 SEGV segmentation (memory) violation
    • 20 TSTP terminal stop
    • 28 WINCH window resize/change

  • killall Send a signal to all processes running any of the specified commands. If no signal name is specified, SIGTERM is sent.

  • lshw List hardware

  • -html Outputs the device tree as an HTML page.

  • -xml Outputs the device tree as an XML tree.

  • -json Outputs the device tree as a JSON object

  • -short Outputs the device tree showing hardware paths

  • -businfo Outputs the device list showing bus information, detailing SCSI, USB, IDE and PCI addresses.

  • -C type of hardware, for example memory

  • shutdown shuts down the system

  • pstree show processes as parent child tree

  • vmstat Report virtual memory statistics: processes, memory, paging, block IO, traps, and cpu activity.

  • xload opens a windo with system stats

  • tload draws in the termina a graph with system stats

7 CONFIGURATION AND ENVIRONMENT

Return to top

  • set set shell options
  • export put in the environment
  • alias
  • printenv shows all the system variables Some interesting variables: EDITOR, HOME, PAGER, SHELL, LANG, PATH (where executables are searched), PS1 (prompt string), TERM, PWD, TZ, USER

The system configuration is established by the following files in this order:/etc/procfile, ~/.bash_profile, ~/bash_login, ~/profiler. If the user is not logged in the shell starts loading /etc/.bashrc and ~/.bashrc.

To reload the environment source .bashrc.

7.1 Customize the prompt

Return to top

echo $PS1 to show the actual prompt config (generally stored in .bashrc).

In the output is possible to find \w for the working directory and \h for the full hostname.

7.1.1 Colors

Return to top

Each color number is included between [033[ and ] and followed by a m.
Black 0;30 Dark Gray 1;30
Blue 0;34 Light Blue 1;34
Green 0;32 Light Green 1;32
Cyan 0;36 Light Cyan 1;36
Red 0;31 Light Red 1;31
Purple 0;35 Light Purple 1;35
Brown 0;33 Yellow 1;33
Light Gray 0;37 White 1;37

8 PACKAGE MANAGEMENT

Return to top

Packages of different systems are, obvioulsly, not compatibles; for example deb and rpm. All systems use 2 kind of tools: low and high level management tools.

Low level tools are for install and remove packages into or from the system.

High level tools are to perform metadata searches and dependency resolution.

For Ubuntu the low level one is dpkg and the high level are. apt-get and aptitute.

dpkg --install packagefile install the given package without performing the dependecy resolution.

dpkg --list list all the installed packages.

dpkg --status packagename show the package status

dpkg -r packagename removes the package

dpkg --search packagename tells who installed that package

apt-cache search packagename (or string) looks for a repo for that package (or containing one with the given string).

apt-get install packagename install the package and performs the dependecy resolution.

apt-get remove removes the package

apt-get update and apt-get upgrade perform a global update pr upgrade

apt-cache show packagename show infos about the given package

9 MEDIA STORAGE

Return to top

/etc/fstab contains ext3 hard disk partitions. It shows:

  • file system
  • mount point
  • file system type
  • options Mount options of access to the device/partition
  • dump frequency of backups; if is set to 0 dum is disabled
  • pass Controls the order in which fsck checks the device/partition for errors at boot time. The root device should be 1. Other partitions should be 2, or 0 to disable checking.

9.1 Commands

Return to top

ls /dev lists all the connected devices

  • mount if executed without params shows actuals mounted filesystems
  • -tdevicename (or isoname)mounting point mounts the filesystem. Example sudo mount -o loop /<path>/<filename>.iso /mnt/iso to mount an ISO image as it was an external cd.
  • unmount
  • fdisk /dev/devicename formats that device
  • fsck Check and repair
  • mkfs create a file system
  • dd write info device
  • wodim burn cd
  • md5sum calculate MD5 checksum

10 NETWORKS

Return to top

  • ping
  • traceroute servername shows the first 30 hopt to reach the specified server. * * * means DNS lookup failure.
  • -n display IPs instead of servernames
  • -w waiting time
  • netstat shows netweork settings and statistics
  • -k display the kernel routing table
  • ftp
  • wget url to download it
  • -O name to change the name of the file when downloaded
  • -i textfile containing a list of url to download them all in one shot
  • wget --ftp-user=USERNAME --ftp-password=PASSWORD URL to download from an FTP with authentication
  • --user-agent=AGENT URL to fake an user aged as mozilla or chrome
  • -r recursively
  • -H convert links to local version
  • -level=NUMBER level of recursion. 0 means infinite
  • ssh user@host to log into a remote host
  • ssh host command runs a comand on a remote host
  • scp
  • scp username@remotehost.edu:foobar.txt /some/local/directory Copy the file "foobar.txt" from a remote host to the local host
  • scp foobar.txt username@remotehost.edu:/some/remote/directory Copy the file "foobar.txt" from the local host to a remote host
  • scp -r foo username@remotehost.edu:/some/remote/directory/bar Copy the directory "foo" from the local host to a remote host's directory "bar"
  • scp username@rh1.edu:/some/remote/directory/foobar.txt username@rh2.edu:/some/remote/directory/ Copy the file "foobar.txt" from remote host "rh1.edu" to remote host "rh2.edu"
  • scp foo.txt bar.txt username@remotehost.edu:~ Copying the files "foo.txt" and "bar.txt" from the local host to your home directory on the remote host
  • scp -P 2264 foobar.txt username@remotehost.edu:/some/remote/directory Copy the file "foobar.txt" from the local host to a remote host using port 2264
  • scp username@remotehost.edu:/some/remote/directory/\{a,b,c\} . Copy multiple files from the remote host to your current directory on the local host
  • scp username@remotehost.edu:~/\{foo.txt,bar.txt\}.

11 SEARCHING FOR FILES

Return to top

  • locate Find by name into system db. The internal system db is updated daily but it can be manually updated with updatedb
  • find accepts some options
  • -name
  • -size
  • -empty
  • -groupname
  • -newer or -cnewer filename as time reference
  • -nogroup No group corresponds to file's numeric group ID.
  • -nouser No user corresponds to file's numeric user ID.
  • -type Supported. POSIX specifies b, c, d, l, p, f and s. GNU find also supports D, representing a Door, where the OS provides these.

And can perform some action on the resulting search

  • -delete

  • -ls

  • -print

  • -quit exit immediatly without doing anything else

  • -exec execute the command without asking. it accepts an array of params

  • -ok Like -exec but ask the user first. If the user agrees, run the command.

  • xargscommand take the input and use it as param for the specified command

  • --null or -0 Input items are terminated by a null character instead of by whitespace; the quotes and backslash are not special (every character is taken literally).

  • touch Update the access and modification times of each file to the current time. A file that does not exist is created empty, unless -c is supplied.

  • -c do not create anything

  • statfilename or dirname or filesystem display properties

12 ARCHIVING AND BACKUP

Return to top

  • gzip filename compress and replace
  • -d is the same of unzip
  • gunzip filename uncompress and replace
  • -l For each compressed file list compressed size, uncompressed size, ratio, uncompressed_name
  • -t Check the compressed file integrity
  • -r recursively
  • -#(0-9), --fast --best velocity VS more compression
  • bzip The same of gzip but using a different algorithm
  • tar Makes a single file out of multiple files or extract from an archive. Example: tar -zxvf data.tar.gz -C location
  • -c create
  • -x extract
  • -r append to the end of an archive
  • -t list the content
  • -f specify the archive name
  • -z Uncompress the resulting archive with gzip command.
  • -C Change directory to the specified location
  • zip join and compress Files
  • unzip location filename
  • rsync [OPTION] SOURCE [DEST]
  • -a create an unique archive
  • -r recursive
  • -u skip files that are newer on the receiver
  • -t preserve times
  • -p preserve permissions
  • -g preserve group
  • -o preserve owner (super-user only)
  • -n perform a trial run with no changes made
  • --delete delete files which no loger exists on source
  • --exclude regexp exclude matches from syncronization

13 REGULAR EXPRESSIONS

Return to top

grep means global regular expression print

  • -i ignore case
  • -v invert match (aka exclude)
  • -c count the number of matches
  • -l print the name of the file containing the match instead of the match itself
  • -L filenames that not match
  • -n print the line number where the match occurr
  • -h suppress filenames in the output
  • ^ string starting with
  • $ string ending with

13.1 Ranges

Return to top

  • [chars] Specified chars
  • [^chars] Exclude specified chars
  • [start-end] like a-k or 0-9
  • [:alnum:] a-Z and 0-9
  • [:word:] Alphanumeric and _
  • [:alpha:] a-Z
  • [:blank:] Spaces and tabs
  • [:cntrl:] Ascii chars from 0 to 127
  • [:digit:] 0-9
  • [:graph:] Any visible char
  • [:lower:] Lowercase letters
  • [:punct:] Punctuaction chars
  • [:print:] graph range and space
  • [:upper:] Uppercase chars
  • [:xdigit:] Chars used in exadecimal numbers

13.2 Counters

Return to top

  • ? More than 1
  • + at least 1
  • {n,m} at least n to maximum m

14 TEXT PROCESSING

Return to top

  • cat concatenate
  • -A display non printing chars too
  • -n display line numbers
  • -s suppress multiple blank lines
  • sort sort lines. Is possible specify an offset to a column. Example sort -k 3-7 means sort by the seventh char of the third field
  • -b ignore leading blanks
  • -f ignore case
  • -m merge
  • -n numeric sort
  • -o output to a file
  • -r reverse
  • -t specify column separator
  • uniq Report or omit repeated lines. With no options, matching lines are merged to the first occurrence.
  • -c prefix lines by the number of occurrences
  • -d only print duplicate lines, one for each group
  • -f Number avoid comparing the first N fields
  • -i ignore case
  • -s Number avoid comparing the first N characters
  • cut remove sections from file. Example cut -d ':' -f filename
  • -c select only these characters
  • -d delimiter Use specified delimiter instead of TAB for field delimiter. It sould go between '
  • -f field list select only these fields
  • --complement reverse extraction
  • paste Copy content from origin to destination
  • join Join lines of two files on a common field.
  • comm Compare 2 sorted files. With no options, produce three-column output. Column one contains lines unique to FILE1, column two contains lines unique to FILE2, and column three contains lines common to both files. Files must be sorted. Example comm <(sort firstfile) <(sort secondfile)
  • -1 Suppress column 1 (lines unique to FILE1)
  • -2 Suppress column 2 (lines unique to FILE2)
  • -3 Suppress column 3 (lines that appear in both files)
  • --check-order Check that the input is correctly sorted, even if all input lines are pairable
  • diff Compare 2 sorted files showing differences
  • -c Show context (display the first file before and after the second one)
  • -u Compact difference (git diff style). Without, equal lines are grouped
  • -a Treat all as text
  • -r Recursively compare any subdirectories found
  • patch receives a file containing diff output and apply those difference to a given target.
  • tr Convert input changing a set of chars with another one
  • -s replace each input sequence of a repeated character that is listed in SET1 with a single occurrence of that character
  • sed Stream Edit
  • aspell Check spelling interactively and replace errors,
  • --dont-backup to avoid the creation of a .bak file as backup
  • --add-html-check=<list>, --rem-html-check=<list> Add or remove a list of HTML attributes to always check.
  • --add-html-skip=<list>, --rem-html-skip=<list> Add or remove a list of HTML attributes to always skip while spell checking

15 FORMATTING OUTPUT

Return to top

  • nl numebr lines of files
  • -b style use style for numbering body lines. Styles can be:
  • -a All
  • -t Non blank
  • -n None
  • -p Number only lines matching regexp
  • -f Number footer
  • -h Number header
  • -i number Step increment
  • -n Number format
  • -ln Left justified
  • -rn Right justified
  • -rz Right justified with leading zeros
  • -p Do not reset page number
  • -s Add a string after the number
  • -v First line number on each logical page
  • -w number Use the given number for line numbers
  • fold Break lines at specified width
  • -w Specify width (default is 80)
  • -s Respect words spaces
  • fmt Text formatter
  • pri Paginate for printing
  • printf Print a string following the specified format
    • %d as decimal
    • %f as float
    • %o integer as octal
    • %s as string
    • %x as hexadecimal
    • %y as hexadecimal in upcase
    • %% print a single %
  • groff and family are a group of text editors as LATEX

16 BASH SCRIPTING BASICS

Return to top

Every script must start with #!/bin/bash

Every script must have read and execute permission

Is a good practice to use the long form for commands in scripts

variable=value must be written without spaces around the =.

$variable echoes the variable value.

Variables can't start start with numbers, spaces and puntuation signs are not allowed.

declare -r UPPERCASED_NAME=VALUE creates a constant.

{enclose variable name} to avoid confusion in case of composed names.

function name {code} is a funcion declaration. Function are called by name and they must be declared before their call.

Local variabled inside a function are declared with LOCAL.

A script executed with the -x flag will output all the executed lines.

set -x (and set +x) can be used to trace only a section of the script

TIME command returns the amount of time spent executing the given command.

{} and () can be used to group a set of commands.

< () sends the output of the group to its parent.

trap [commands] [signals] Creates a listener to the system signals to catch them (for example keyboard abort sequences) and execute the specified command. Example trap "echo 'killing signal'" SIGINT SIGTERM.

sleepseconds stops the execution for the amount of given seconds

waitPID Waits for a process to finish it execution.

16.1 Flow control

Return to top

IF [$var=value];THEN
ELSIF
ELSE
FI

When a command executes, it returns 0 if it success or any other value up to 255 if it fails.

The exit command return the execution status

To execute an expression, can be used the command test or the more powerful [[ expression ]]. For integers the command is (( expression )).

Parenthesis can be used to group expression but they must be escaped.

16.1.1 Strings Expressions evaluation

Return to top

  • -n string length > 0
  • -z string length = 0
  • == equal
  • != not equal
  • < or > given 2 strings, check if the first string orders before (or after) the second one

16.1.2 Integers Expressions evaluation

Return to top

  • eq equal
  • ne not equal
  • le less than or equal
  • lt less than
  • ge greater than or equal
  • gt greater

16.1.3 Logical operators

Return to top

Logical operators can be used between commands too.

  • -a for string is written as [[&&]] and for integers ((&&)). It means and.
  • -o for string is written as [[||]] and for integers ((||)). It means or.
  • ! for string is written as [[!]] and for integers ((!)). It means not.

16.2 Reading inputs

Return to top

read varname (withour $)

read varname1, varname2, varname3, varname4 ...

If less values are provided, the rest will be set as empty string. Space is used to separate input assignments. The default receiver is the variable REPLY.

The read command can't be piped because pipes create subshells, each with its own REPLY variable.

The read command supprts the following options:

  • -a assign input to an array
  • -ddelimiter declare a string delimiter which indicates the end of the input. (the default is the new line char)
  • -e the input can be edited before submitting
  • -nnumber read only X chars
  • -p display a prompt
  • -v raw mode. it means that \ is not an escape char but is traten as simple text
  • -s silent mode
  • -tseconds timeout
  • -u read from file

The IFS (internal field separator) command can be used to change the separator between columns (which by default is space). Example IFS=":" The commands << and <<< stay for here document and here string, everything follows these commands is considered all input until read delimiter is found, no matter what it contains or how large it could be.

16.2.1 Positionals params

Return to top

$0...$9 by default, ${10}... from numbers with more than one digit.

16.3 Flow control

Return to top

WHILE [expression]; DO
code
DONE

UNTIL [expression]; DO
code
DONE

They can be used to read files too.

WHILE [expression]; DO
code
DONE < file.txt

CASE input IN
option1)
;;
option....)
;;
ESAC

;; means _break. The option) can be a values or a regexp or a set (like [:alpha:])

FOR var IN [words]; DO
code
DONE

FOR ((i=0;i<5;i=i+1));DO
code
DONE

16.4 Variables

Return to top

  • variable=value Assignment
  • let var2=var1 operation Set var2 with the result of the operation over var1.
  • echo $variable Echoes the variable value
  • $variable and ${variable:+default} Execute the variable values as command, if it is null does nothing
  • ${variable:-default} Execute the variable values as command, if it is null execute the default value as command
  • {$variable:=default} Execute the variable values as command, if it is null set the variable to the default value and executes it.
  • {$variable:?default} Execute the variable values as command, if it is null return the default value as error message
  • echo ${!prefix*} return a list of variables beginning with the give prefix.

16.5 Strings

Return to top

  • ${#string} String length
  • ${string:position:length} Extracts substring -${string#pattern} Removes the leading shortest match from string -${string##pattern} Removes the leading largest match from string -${string%pattern} Removes the starting shortest match from string -${string%%pattern} Removes the starting largest match from string -${value_to_find/string/replacing_value} replace value to find with replacing value. The search accepts:
  • // global replace
  • /# replace at the beginning
  • /% replace at the end

16.6 Numbers

Return to top

Numbers are in decimal base by default. Numbers beginning with 0 are considered octals, numbers beginning with 0x are considered hexadecimals.

Assignments are the common ones: =, +=, -=, *=, /=, %=, ++, --

bc Is the bash decimal calculator. It could be used with the here doc notation. Example: bc -l <<< '10/3', without -l the output is rounded to the nearest integer. Alternatively, scale=number can be used to specify the number of decimals: bc <<< 'scale=2; 10/3'

16.7 Arrays

Return to top

There is no maximum limit to the size of an array, nor any requirement that member variables be indexed or assigned contiguously. Arrays are zero-based: the first element is indexed with the number 0.Return to top

  • declare -a ARRAYNAME Instantiate an array.
  • ARRAY[index]=value Assign the value to the specified index.
  • * and @ Mean all the keys of the array.
  • #ARRAY Return the array length
  • #array[@] Returns all the keys of the given array
  • #array[key] Return the value of the given key
  • += Push a new value into the array
  • unset deletes the array ot the element at the given key
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment