Skip to content

Instantly share code, notes, and snippets.

@nathaniel-miller

nathaniel-miller/00 basic linux and shell.md

Last active December 31, 2015 06:43
Show Gist options
  • Save nathaniel-miller/52169079265694be3299 to your computer and use it in GitHub Desktop.
Save nathaniel-miller/52169079265694be3299 to your computer and use it in GitHub Desktop.
Download ZIP
Basic Linux and Shell
Raw
00 basic linux and shell.md

The Rails Prime stack runs on a Unix stack, which is to say that all production hardware (and most people's personal workstations) are some variant of that system. As such, you'll need to understand some basic Unix utilities to control your development environment.

Raw
01 absolute vs relative paths
# 1) Demonstrate a few commands showing the difference between navigating to an absolute path versus a relative path.
dukeoflaser:~/workspace $ less /etc/passwd #view file via absolute path
dukeoflaser:~/workspace $ less ../../../etc/passwd #view file via relative path
dukeoflaser:/ $ ls /usr/local/heroku/vendor/gems #list contents of directory via absolute path
dukeoflaser:/ $ ls usr/local/heroku/vendor/gems #list contents of directory via relative path
dukeoflaser:/ $ ls ./usr/local/heroku/vendor/gems #list contents of directory via relative path
dukeoflaser:~/workspace $ cd /home #change directory via absolute path
dukeoflaser:/home $
dukeoflaser:~/workspace $ cd ../../../home #change directory via relative path
dukeoflaser:/home $
# 2) Navigate to home using the tilde and then list your working directory to verify the path you've arrived at.
dukeoflaser:/home $ cd ~
dukeoflaser:~ $ pwd
/home/ubuntu
# 3) Now navigate upward two directories and check your current directory again.
dukeoflaser:~ $ cd ../..
dukeoflaser:/ $ pwd
/
Raw
02 everything is a file
# 1) Name some 'files' in a standard unix filesystem that aren't very file-like.
dukeoflaser:/ $ ls -l proc/kcore #This is a character device.
crw-rw-rw- 1 root root 1, 3 Nov 11 03:01 proc/kcore #It is a representation of the memory in your computer.
dukeoflaser:/ $ ls -l proc/mounts #This is a symbolic link.
lrwxrwxrwx 1 root root ... proc/mounts -> self/mounts #It contains a referencing path to another file or directory.
dukeoflaser:/ $ ls -l #This is a directory.
dr-xr-xr-x 10741 root root 0 Nov 11 03:01 proc/
# 2) Identify some places where we use file descriptors for non-file content as part of the development process
# (hint: sockets are a good example here).
#A file descriptor is an integer associated with any open file.
#Those files could be, among other things, a network connection, a terminal, or your keyboard/mouse/monitor.
#When you communicate with another program over the Internet you have to go through a file descriptor.
# 3) Postgres writes a file when it starts up to record some information about itself, called a 'pid file'.
# Find this file and display its contents.
dukeoflaser:~ $ cd ../../etc/postgresql/9.3/main
dukeoflaser:/etc/postgresql/9.3/main $ ls
... postgresql.conf ...
dukeoflaser:/etc/postgresql/9.3/main $ less postgresql.conf
...
#------------------------------------------------------------------------------
# FILE LOCATIONS
#------------------------------------------------------------------------------
...
# If external_pid_file is not explicitly set, no extra PID file is written.
external_pid_file = '/var/run/postgresql/9.3-main.pid' # write an extra PID file
...
dukeoflaser:/etc/postgresql/9.3/main $ less /var/run/postgresql/9.3-main.pid
5129 #Well, that was anti-climactic. :-/
Raw
03 path and the environment
# 1) Demonstrate the command to view all environment variables, and then use grep to find the one called EDITOR.
# If it's not set, modify your initialization scripts to set it to 'vim'.
dukeoflaser:~/workspace $ env
...
dukeoflaser:~/workspace $ env | grep EDITOR
dukeoflaser:~/workspace $ #Hmmmmm. Nothing there. :-/
#If you'd like to set the bash environment variables permanently,
#add your settings to the initialization file located in the home
#directory of your account $HOME/.bash_profile.
#SOURCE: http://bash.cyberciti.biz/guide/Startup_scripts
#NOTE: .bash_profile is now simply called .profile
dukeoflaser:~/workspace $ ls -a ~
./ .bash_aliases .bash_logout .c9/ .gemrc .gitignore .hgignore .node-gyp/ .nvm/ .ssh/ workspace/
../ .bash_history .bashrc .gem/ .gitconfig .gnupg/ .hgrc .npm/ .profile lib/
#Though placing your aliases and shell functions in your .profile will work,
#it is not considered good form.
#There is a separate file named .bashrc that is intended to be used for such things.
#SOURCE: http://linuxcommand.org/wss0020.php
dukeoflaser:~/workspace $ cd
dukeoflaser:~ $ echo 'export EDITOR="vim"' >> .bashrc #Append the output of <cmd> to <file>
#When a script is run using source it runs within the existing shell,
#any variables created or modified by the script will remain available after the script completes.
#SOURCE: http://ss64.com/bash/source.html
dukeoflaser:~ $ source .bashrc
dukeoflaser:~ $ env | grep EDITOR
EDITOR=vim
# 2) Demonstrate a command that echoes the value of the PATH environment variable.
dukeoflaser:~ $ echo $PATH
/home/ubuntu/.nvm/versions/node/v4.1.1/bin: ... /usr/local/rvm/bin
# 3) Demonstrate the command that will tell you the full path of the script used when you execute 'ruby'
dukeoflaser:~ $ which ruby
/usr/local/rvm/rubies/ruby-2.2.1/bin/ruby
Raw
04 transferring files - vagrant
# 1) Create a text file with your email username.
iMac:~ Nathaniel$ touch dukeoflaser.txt
iMac:~ Nathaniel$ echo 'Email: dukeoflaser@gmail.com' >> dukeoflaser.txt
# 1b) Now SCP it to your VM and put it into the /tmp directory.
#Set up a Vagrant VM.
# ...
#default: SSH address: 127.0.0.1:2222
#default: SSH username: vagrant
#default: SSH auth method: private key
#default: Warning: Connection timeout. Retrying...
#...
#default: Vagrant insecure key detected. Vagrant will automatically replace
#default: this with a newly generated keypair for better security.
#...
#default: Inserting generated public key within guest...
#default: Removing insecure key from the guest if it's present...
#default: Key inserted! Disconnecting and reconnecting using new SSH key...
# ==> default: Machine booted and ready!
iMac:~ Nathaniel$ vagrant ssh-config
Host default
HostName 127.0.0.1
User vagrant
Port 2222
...
#Unable to SSH into the box as I do not know the password.
iMac:~ Nathaniel$ ssh vagrant@127.0.0.1
Password:
Permission denied (publickey,keyboard-interactive).
#Same goes for SCP.
iMac:~ Nathaniel$ scp dukeoflaser.txt vagrant@127.0.0.1:/tmp
Password:
Permission denied (publickey,keyboard-interactive).
lost connection
#Using the private network IP didn't work either.
iMac:~ Nathaniel$ ssh -p 2222 vagrant@192.168.33.10
ssh: connect to host 192.168.33.10 port 2222: Operation timed out
#I could still SSH into the box the 'Vagrant Way', however.
iMac:~ Nathaniel$ vagrant ssh
Welcome to your Vagrant-built virtual machine.
vagrant@precise32:~$
#Placing a Vagrantfile inside a folder on your host allows
#any parallel files/folders to be accessed from within the VM.
iMac:~ Nathaniel$ ls
... Vagrantfile ... dukeoflaser.txt
#Instead of 'iMac:~ Nathaniel$ scp dukeoflaser.txt vagrant@127.0.0.1:/tmp'
#I did the following:
iMac:~ Nathaniel$ vagrant ssh
vagrant@precise32:~$ ls ../../vagrant
... dukeoflaser.txt ...
vagrant@precise32:~$ cp ../../vagrant/dukeoflaser.txt ../../tmp
vagrant@precise32:~$ ls ../../tmp
dukeoflaser.txt
# 2) Delete your local copy,
iMac:~ Nathaniel$ rm dukeoflaser.txt
iMac:~ Nathaniel$ cat dukeoflaster.txt
cat: dukeoflaster.txt: No such file or directory
# 2b) ...and then copy the file from the remote box back to your local machine.
#As previously stated, I couldn't SCP without a password.
iMac:~ Nathaniel$ scp vagrant@127.0.0.1:dukeoflaser.txt /tmp
Password:
Permission denied (publickey,keyboard-interactive).
#Instead of 'iMac:~ Nathaniel$ scp vagrant@127.0.0.1:/tmp/dukeoflaser.txt ../Nathaniel'
#I did the following:
iMac:~ Nathaniel$ vagrant ssh
vagrant@precise32:~$ cp ../../tmp/dukeoflaser.txt ../../vagrant
#Now it's back
iMac:~ Nathaniel$ cat dukeoflaser.txt
Email: dukeoflaser@gmail.com
Raw
04.1 transferring files - vagrant
#THIS IS AN ALTERNATIVE SOLUTION TO THE PROBLEMS I WAS HAVING WHILE USING THE VAGRANT VM.
# 1) Create a text file with your email username.
iMac:~ Nathaniel$ touch dukeoflaser.txt
iMac:~ Nathaniel$ echo 'Email: dukeoflaser@gmail.com' >> dukeoflaser.txt
# 1b) Now SCP it to your VM and put it into the /tmp directory.
#An alternate solution to the problems I was having
#was to install the 'vagrant-scp' plugin.
iMac:~ Nathaniel$ vagrant plugin install vagrant-scp
Installing the 'vagrant-scp' plugin. This can take a few minutes...
Installed the plugin 'vagrant-scp (0.5.4)'!
#The VM /tmp directory is empty
vagrant@precise32:/tmp$ ls
#Used the plugin to SCP into the VM
iMac:~ Nathaniel$ vagrant scp dukeoflaser.txt /tmp
Warning: Permanently added '[127.0.0.1]:2222' (RSA) to the list of known hosts.
dukeoflaser.txt
#Worked like a charm
vagrant@precise32:/tmp$ cat dukeoflaser.txt
Email: dukeoflaser@gmail.com
# 2) Delete your local copy, and then copy the file from the remote box back to your local machine.
iMac:~ Nathaniel$ rm dukeoflaser.txt
iMac:~ Nathaniel$ cat dukeoflaster.txt
cat: dukeoflaster.txt: No such file or directory
#Copying file back from VM
iMac:~ Nathaniel$ vagrant scp default:/tmp/dukeoflaser.txt ../Nathaniel
Warning: Permanently added '[127.0.0.1]:2222' (RSA) to the list of known hosts.
dukeoflaser.txt
iMac:~ Nathaniel$ cat dukeoflaser.txt
Email: dukeoflaser@gmail.com
Raw
04.2 transferring files - vagrant
# 1) Create a text file with your email username.
iMac:~ Nathaniel$ touch dukeoflaser.txt
iMac:~ Nathaniel$ echo 'Email: dukeoflaser@gmail.com' >> dukeoflaser.txt
# 1b) Now SCP it to your VM and put it into the /tmp directory.
#I discovered that 'vagrant' works as the password.
iMac:~ Nathaniel$ ssh -p 2222 vagrant@127.0.0.1
vagrant@127.0.0.1's password: #entered 'vagrant'
Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic-pae i686)
vagrant@precise32:~$
#I can now SCP conventionally.
iMac:~ Nathaniel$ scp -P 2222 dukeoflaser.txt vagrant@127.0.0.1:/tmp
vagrant@127.0.0.1's password: #entered 'vagrant'
dukeoflaser.txt
...
vagrant@precise32:/tmp$ cat dukeoflaser.txt
Email: dukeoflaser@gmail.com
# 2) Delete your local copy,
iMac:~ Nathaniel$ rm dukeoflaser.txt
iMac:~ Nathaniel$ cat dukeoflaster.txt
cat: dukeoflaster.txt: No such file or directory
# 2b) ...and then copy the file from the remote box back to your local machine.
iMac:~ Nathaniel$ scp -P 2222 vagrant@127.0.0.1:/tmp/dukeoflaser.txt ../Nathaniel
vagrant@127.0.0.1's password: #entered 'vagrant'
dukeoflaser.txt
iMac:~ Nathaniel$ cat dukeoflaser.txt
Email: dukeoflaser@gmail.com
Raw
05 git just git
# 1) Go run through Try Git to get up to speed.
...
# 2) Fork the exercises repo to get ready for the other exercises.
#Fork the original repo to:
#https://github.com/dukeoflaser/level_up_exercises
#Clone repo
$ git clone https://github.com/dukeoflaser/level_up_exercises workspace
Initialized empty Git repository in /home/ubuntu/workspace/.git/
...
Branch master set up to track remote branch master from origin.
Already on 'master'
dukeoflaser:~/workspace (master) $
# 3) Create a topic branch called 'just_git'...
dukeoflaser:~/workspace (master) $ git checkout -b just_git
Switched to branch 'just_git'
# 3b) ...add a new file...
dukeoflaser:~/workspace (just_git) $ touch git_just_git.txt
dukeoflaser:~/workspace (just_git) $ echo 'Git. Just Git' >> git_just_git.txt
dukeoflaser:~/workspace (just_git) $ git add git_just_git.txt
dukeoflaser:~/workspace (just_git) $ git status
...
dukeoflaser:~/workspace (just_git) $ git commit -m 'Add simple text file'
[just_git 27bbb30] Add simple text file
1 file changed, 1 insertion(+)
create mode 100644 git_just_git.txt
# 3c) ...and push it to github.
dukeoflaser:~/workspace (just_git) $ git push -u origin just_git
Username for 'https://github.com': dukeoflaser
Password for 'https://dukeoflaser@github.com':
...
dukeoflaser:~/workspace (just_git) $
# 3d) Issue a pull request against the upstream repo.
#Go to github and press the 'pull request' button, etc.
# 4) Add some more text and set the author to '_partner'.
dukeoflaser:~/workspace (just_git) $ echo "This line was written by Nathaniel's twin, Timothy." >> git_just_git.txt
dukeoflaser:~/workspace (just_git) $ git add -A
dukeoflaser:~/workspace (just_git) $ git commit -m "Have partner add a line of text to git_just_git.txt" --author="_partner <partner@partner.com>"
dukeoflaser:~/workspace (just_git) $ git log
commit c9d347a1c535ca708e4072cfc630d74d0d00670e
Author: _partner <partner@partner.com>
Date: Tue Nov 17 13:18:18 2015 +0000
Have partner add a line of text to git_just_git.txt
commit 27bbb30c599e24fba322fdeb71a7caabc5ffba36
...
# 4b) Push it to github...
dukeoflaser:~/workspace (just_git) $ git push -u origin just_git
Username for 'https://github.com': dukeoflaser
Password for 'https://dukeoflaser@github.com':
...
# 4c) ...and issue a pull request against the upstream repo.
#Go to github and press the 'pull request' button, etc.
#Note: Initial pull request granted after second commit made,
#so partner commit was included already.
dukeoflaser:~/workspace (just_git) $ git checkout master
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
dukeoflaser:~/workspace (master) $
Raw
06 permissions
# 1) Use touch to create a new file in the /tmp directory.
dukeoflaser:/tmp $ touch new_file
# 1b) What permissions do user / group / and world have to that file by default?
dukeoflaser:/tmp $ ls -l new_file
-rw-r--r-- 1 ubuntu ubuntu 0 Nov 18 02:51 new_file
#User has read and write permissions
#Group has read permissions
#World has read permissions
# 2) Change ownership of that file to the root user and root group.
dukeoflaser:/tmp $ chown root:root new_file
chown: changing ownership of ‘new_file’: Operation not permitted
dukeoflaser:/tmp $ sudo chown root:root new_file
dukeoflaser:/tmp $ ls -l new_file
-rw-r--r-- 1 root root 0 Nov 18 02:51 new_file
# 3) Use chmod to grant read and execute permissions to everyone for that file using the 'absolute' (Octal) syntax.
dukeoflaser:/tmp $ chmod 0555 new_file
chmod: changing permissions of ‘new_file’: Operation not permitted
dukeoflaser:/tmp $ sudo chmod 0555 new_file
dukeoflaser:/tmp $ ls -l new_file
-r-xr-xr-x 1 root root 0 Nov 18 02:51 new_file*
#The Octal syntax is the approach used when setting file and directory permissions programmatically.
# 4) Remove the ability for the root group to write the file. Use the 'symbolic' syntax to remove that permission.
dukeoflaser:/tmp $ sudo chmod g-w new_file
dukeoflaser:/tmp $ ls -l new_file
-r-xr-xr-x 1 root root 0 Nov 18 02:51 new_file* #The group write permission was never set in the first place.
#The Symbolic syntax is used to add or remove permissions relative to the existing permissions on that object.
Raw
07 sudo
#Why sudo?
#Users use their own password, not the root password if they MUST access root privilages.
#Sudo gives users SOME root privilages, but not all of them.
#Operating as a root user opens up the possibility of a cataclismic error.
#Having to type sudo before each command acts as a safeguard against forgetful root usage.
# 1) Name an example of a command where sudo is necessary to run a command.
dukeoflaser:/etc $ cat sudoers
cat: sudoers: Permission denied
dukeoflaser:/etc $ sudo cat sudoers
#
# This file MUST be edited with the 'visudo' command as root.
#
# Please consider adding local content in /etc/sudoers.d/ instead of
# directly modifying this file.
#
# See the man page for details on how to write a sudoers file.
...
#includedir /etc/sudoers.d
#Installing anything major (developer tools) are almost always going to require access of folders outside your home folder ~/
#...or even when you ARE in your home folder.
dukeoflaser:~/workspace (master) $ service postgresql start
chmod: changing permissions of '/var/run/postgresql': Operation not permitted
* Starting PostgreSQL 9.3 database server
* Error: You must run this program as the cluster owner (postgres) or root
...fail!
dukeoflaser:~/workspace (master) $ sudo service postgresql start
* Starting PostgreSQL 9.3 database server
...done.
# 2) Name a few examples where use of sudo can get you into trouble.
#The sudo command can get you into trouble when it's used in conjunction with a command that either:
# a) allows the execution of a second command...
dukeoflaser:~/workspace (master) $ touch sample_file.txt
dukeoflaser:~/workspace (master) $ whoami
ubuntu
dukeoflaser:~/workspace (master) $ sudo find . -name sample_file.txt -exec bash \;
dukeoflaser:~/workspace (master) $ whoami
root
#The same command that previously requried the sudo command no longer does.
dukeoflaser:~/workspace (master) $ service postgresql start
* Starting PostgreSQL 9.3 database server
...done.
# b) or spawns a new shell.
dukeoflaser:~/workspace (master) $ whoami
ubuntu
dukeoflaser:~/workspace (master) $ sudo bash
dukeoflaser:~/workspace (master) $ whoami
root
#Time to be careful!
#Any command can be issued with root user privileges, endangering the system.
dukeoflaser:~/workspace (master) $ exit
exit
dukeoflaser:~/workspace (master) $ whoami
ubuntu
#Back to safe ground.
Raw
08 system resources
# 1) Demonstrate the command to check what the remaining disk space is on your laptop.
# Make sure to use the flag to make the output more readable.
Mactop:/ Nathaniel$ df -h
Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk0s2 465Gi 333Gi 131Gi 72% 87435599 34451143 72% /
...
/dev/disk1s3 1.4Ti 428Gi 969Gi 31% 112293495 253958372 31% /Volumes/The Tempest
/dev/disk2s2 596Gi 306Gi 290Gi 52% 80122619 76076367 51% /Volumes/The Swan
# 2) Navigate to the your home directory...
Mactop:/ Nathaniel$ cd ~
# 2b) ...and demonstrate the command to see how much space that directory and each subdirectory takes on disk.
Mactop:~ Nathaniel$ du -h | less
8.0K ./.adobe
...
...
...
8.8G ./VirtualBox VMs
248G .
(END)
# 3) Demonstrate the command to see how much memory is currently being consumed on your laptop.
Mactop:~ Nathaniel$ htop
1 [|||||| 6.4%] Tasks: 209 total, 1 running
2 [||||||| 9.6%] Load average: 1.41 1.38 1.39
Mem[|||||||||||||||||||||||||||||||||||||||||||||||||||| 4999/7936MB] Uptime: 01:49:02
Swp[ 0/0MB]
PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command
860 Nathanie 31 0 2385M 3252 0 R 1.0 0.0 0:00.00 htop
1 root 0 0 0 0 0 0.0 0.0 0:00.00 (launchd)
...
F1Help F2Setup F3SearchF4InvertF5Tree F6SortByF7Nice -F8Nice +F9Kill F10Quit
# 3b) Sort the running processes to see which is consuming the most resources.
#To access the sort menu press ">". Use the UP/DOWN arrows to navigate, and ENTER to select.
#Sort by
#PID
#USER
#PRI
#NI
#VIRT
#RES
#SHR
#S
#CPU%
#MEM%
#TIME+
#Command
# 4) Using that same command, run a few processor-intensive tasks on your laptop.
# Observe the change to your processor usage for a few minutes.
# Read and interpret the CPU load numbers for the system.
#For htop to return proper CPU% and MEM% on Max OS X it needs root privileges. So...
Mactop:/ Nathaniel$ sudo htop
Password:
#BEFORE
1 [||||||||| 12.0%] Tasks: 222 total, 1 running
2 [||||||| 9.6%] Load average: 1.76 1.59 1.44
Mem[|||||||||||||||||||||||||||||||||||||||||||||||||||||||||5702/7936MB] Uptime: 02:23:15
Swp[ 0/0MB]
#I loaded up about 15 different applications, including a CPU heavy audio project in Logic Pro,
#a large guitar patch in Axe-Edit, Skype, Adobe Photoshop, iTunes, and some others.
#AFTER
1 [||||||||||||||||||||||||||||||||||||||||||| 60.8%] Tasks: 255 total, 2 running
2 [||||||||||||||||||||||||||||||||||||||| 55.6%] Load average: 26.15 15.06 7.50
Mem[|||||||||||||||||||||||||||||||||||||||||||||||||||||||||6908/7936MB] Uptime: 02:49:04
Swp[ 0/0MB]
PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command
1958 Nathanie 97 0 1245M 213M 0 S 41.0 2.7 0:01.48 /Applications/Logic Pro.app/Contents/MacOS/Logic Pro
2056 Nathanie 97 0 922M 191M 0 R 36.0 2.4 0:00.24 /Applications/Axe-Edit.app/Contents/MacOS/Axe-Edit
...
2062 Nathanie 63 0 3828M 214M 0 U 3.0 2.7 0:00.09 /Applications/Skype.app/Contents/MacOS/Skype
...
#The immediate load average jumped signifantly from 1.76, which is appropriate for a dual core machine,
#to a devestating 26.15 unit long line up, which stopped my audio project in it's tracks.
#The longer average load times were also well beyond acceptable boundaries.
Raw
09 ports
# 1) Demonstrate the command to show the list of currently listening ports on your laptop.
#Using Mac OS X
Mactop:~ Nathaniel$ lsof -i | grep LISTEN
SpotifyWe 399 Nathaniel 6u IPv4 0xb31406b3c6357379 0t0 TCP localhost:4370 (LISTEN)
SpotifyWe 399 Nathaniel 7u IPv4 0xb31406b3c51c7379 0t0 TCP localhost:4380 (LISTEN)
netsessio 400 Nathaniel 9u IPv4 0xb31406b3c5941769 0t0 TCP localhost:9421 (LISTEN)
Dropbox 446 Nathaniel 34u IPv4 0xb31406b3c7b3a519 0t0 TCP *:17500 (LISTEN)
Dropbox 446 Nathaniel 44u IPv4 0xb31406b3c58b6c49 0t0 TCP localhost:17600 (LISTEN)
Dropbox 446 Nathaniel 49u IPv4 0xb31406b3c7b381d9 0t0 TCP localhost:17603 (LISTEN)
GitHub 2008 Nathaniel 15u IPv4 0xb31406b3c630c379 0t0 TCP localhost:25035 (LISTEN)
GitHub 2008 Nathaniel 16u IPv6 0xb31406b3bc4e2bc9 0t0 TCP localhost:25035 (LISTEN)
VBoxHeadl 2700 Nathaniel 18u IPv4 0xb31406b3c5aed379 0t0 TCP localhost:rockwell-csp2 (LISTEN)
#Using Ubuntu
dukeoflaser:~ $ netstat -l
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp6 0 0 [::]:ssh [::]:* LISTEN
Active UNIX domain sockets (only servers)
Proto RefCnt Flags Type State I-Node Path
unix 2 [ ACC ] STREAM LISTENING 192996609 /home/ubuntu/.c9/bridge.socket
unix 2 [ ACC ] STREAM LISTENING 192996622 /home/ubuntu/.c9/2157596/collab.sock
unix 2 [ ACC ] STREAM LISTENING 133498232 /tmp/tmux-1000/cloud91.8
# 2) Do it again on a Linux VM, but make sure to include the flags to display the applications that are listening.
vagrant@precise32:~$ sudo netstat -lp #need root permissions for -p info
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 *:33354 *:* LISTEN 563/rpc.statd
tcp 0 0 *:sunrpc *:* LISTEN 515/rpcbind
...
Active UNIX domain sockets (only servers)
Proto RefCnt Flags Type State I-Node PID/Program name Path
unix 2 [ ACC ] SEQPACKET LISTENING 6937 295/udevd /run/udev/control
unix 2 [ ACC ] STREAM LISTENING 6761 1/init @/com/ubuntu/upstart
unix 2 [ ACC ] STREAM LISTENING 7508 515/rpcbind /run/rpcbind.sock
unix 2 [ ACC ] STREAM LISTENING 7284 439/dbus-daemon /var/run/dbus/system_
# 2b) Find the application port and PID that run sshd.
vagrant@precise32:~$ sudo netstat -pant | grep sshd
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 606/sshd
tcp 0 0 10.0.2.15:22 10.0.2.2:63160 ESTABLISHED 1288/sshd: vagrant
tcp6 0 0 :::22 :::* LISTEN 606/sshd
#SSHD runs on Port 22
#SSHD is being used by two Process Identifiers (PIDs) - 606 and 1288
# 3) List the default ports for ssh, scp, http, https and postgres.
#SSH 22
#SCP 22
#HTTP 80
#HTTPS 443
#PostgreSQL 5432
Raw
10 fg bg kill ps
# If you run the command `ruby -e 'sleep 1 while true'`,
# you'll start a ruby process that never completes.
# 1) Try it out and verify that the command never returns. Use the keyboard command to kill it.
dukeoflaser:~ $ ruby -e 'sleep 1 while true'
^C-e:1:in `sleep': Interrupt
from -e:1:in `<main>'
# 2) Try it again, but append the character that causes the process to run in the background.
dukeoflaser:~ $ ruby -e 'sleep 1 while true' &
[1] 7954
# 3) Use the terminal command to bring it back to the foreground...
dukeoflaser:~ $ fg
ruby -e 'sleep 1 while true'
# 3b) ...then use the two keyboard commands to return it to the background.
^Z
[1]+ Stopped ruby -e 'sleep 1 while true'
dukeoflaser:~ $ bg
[1]+ ruby -e 'sleep 1 while true' &
# 3c) Yes this is a bit pointless, stick with me here.
# 4) List the processes on your system and find the PID that corresponds to that ruby script.
dukeoflaser:~ $ ps
PID TTY TIME CMD
19 pts/0 00:00:00 bash
47 pts/0 00:00:00 bash
7954 pts/0 00:00:00 ruby
7968 pts/0 00:00:00 ps
# 5) Since we don't want to repeat that debacle above,
# use the terminal command to rudely kill the ruby process based on its PID.
dukeoflaser:~ $ kill 7954
[1]+ Terminated ruby -e 'sleep 1 while true'
Raw
11 start and stop services
# 1) On a Linux development VM, use init scripts to restart sshd. Y'know, just in case.
#Note: According to: http://unix.stackexchange.com/questions/127886/how-can-i-restart-the-ssh-daemon-on-ubuntu,
#Ubuntu calls the service ssh, not sshd.
vagrant@precise32:~$ /etc/init.d/ssh restart
...
start-stop-daemon: warning: failed to kill 606: Operation not permitted
vagrant@precise32:~$ sudo /etc/init.d/ssh restart
Rather than invoking init scripts through /etc/init.d, use the service(8)
utility, e.g. service ssh restart
vagrant@precise32:~$ sudo service ssh restart
ssh stop/waiting
ssh start/running, process 1979
# Let's pretend you just changed your nginx config, and want to reload the changes without stopping the service altogether.
# 2) Invoke the init script without any arguments to see valid choices.
vagrant@precise32:~$ /etc/init.d/nginx
* Usage: /etc/init.d/nginx {start|stop|reload|force-reload|restart|try-restart|status}
# 2b) Now run the correct init command to parse and load the config without stopping the server.
#reload: cause the configuration of the service to be reloaded without actually stopping and restarting the service.
#SOURCE: http://refspecs.linuxbase.org/LSB_3.0.0/LSB-generic/LSB-generic/iniscrptact.html
vagrant@precise32:~$ sudo service nginx reload
Raw
12 monitoring
# 1) Name some resources that you might monitor for a production host.
#Web server software such as Apache
#Server hardware
#Databases such as MySQL or PostgreSQL
#Operating systems issues
#Storage/memory
#Processing power/usage
# 2) There's a big problem with most monitoring:
#a momentary spike in resource usage isn't actually a problem.
#Some software, like the 'god' gem, tries to overcome this limitation.
#Can you think of some strategies to solve this problem?
#Depending on what 'temporary' is, a time limit could be set.
#If resource usage continues to surpasses a threshold beyond that time limit,
#the offending process(es) would be restarted.
#Perhaps a longer load average should decide if there truly is a problem.
#If a resource spike is detected, have that particular resource/process
#'sampled' to see if it continues to spike.
#If so, have a threshold set for how many resource spikes are acceptable in
#a given time frame before the situation is considered problematic.
#Make sure there is plenty of headroom in the system to just deal with resource spikes.
Raw
13 proxies
**** **** ***
** ***** *********
**** ****** ** ******* **
*** **********
* ***** ***** *** *** **
******* ** * ***
|/------ -------\ * * *
**** | |=| :===****
| O | | O | }| *
|---- | ---- | |**
| |___ |\/ *
| |
"Let me Google that..." - \ ----- |
\ |
-__ -- -/
www.google.com
|
|
|
\ /
BROWSER: "Psst, DNS... What's google.com's IP?"
|
|
|
\ /
DNS: "It's 64.233.160.0"
|
|
|
\ /
BROWSER: "Cool, thanks."
*sends a letter to IP address asking for the page*
|
|
|
\ /
CUSTOMS AGENT ASM: "Let me see that!"
*cuts open with knife and dumps on floor*
"Oh. Ya, hmm...looks fine. Nvm."
|
|
|
\ /
LOAD BALANCER #1: "Wow! I got a letter! Why is it so mangled...?"
*squints*
"From what I can tell this guy wants a page.
I think both Matt and Larry have this page,
but Matt is too busy serving that girl,
and Larry is drunk...maybe Dave has it."
"HEY DAVE!!! YOU GOT THIS PAGE???"
|
|
|
\ /
DAVID WEBB: "Yes LB #1, I do."
"I actually have three copies of that page in three different file cabinets,
but my secretary has the keys. Hang on one sec....
Excuse me, Hilda? Can you fetch me this page please?"
|
|
|
\ /
HILDA N. GINX: "Yes, Mr. Webb."
"One of those cabinets is stuck but the other two are ok."
*gets page out of the second cabinet*
*gives to DAVID WEBB*
|
|
|
\ /
DAVID WEBB: *gives to LB*
|
|
|
\ /
LB: *gives to CUSTOM AGENT ASM*
|
|
|
\ /
CUSTOM AGENT ASM: *looks at it suspiciously, then x-rays it*
*finally sends it back to browser*
|
|
|
\ /
BROWSER: *shows page to user*
**** **** ***
** ***** *********
**** ****** ** ******* **
*** **********
* ***** ***** *** *** **
******* ** * ***
|/------ -------\ * * *
**** | |=| :===****
| O | | O | }| *
|---- | ---- | |**
| |___ |\/ *
| |
"Hmmm. I guess yogurt - \ ----- |
DOES go bad... \ |
...eventually". -__ -- -/
Moral of the story: Proxies provide multiple points of failure.
ASM, LB, DAVE, the file cabinets...they all had someone who could take their place if they went down.
In fact, a few coworkers were already down (Larry, I'm looking at you), but the page was retrieved anyway.
Additionally, our user could use another proxy for privacy's sake,
in case he didn't want that Customs Agent to know his home address.
Raw
14 date and time
# 1) Demonstrate the command to reset your time using an external NTP server.
#ntpdate - set the date and time via NTP
#Disclaimer: The functionality of this [ntpdate] program is now available in the ntpd program.
vagrant@precise32:~$ sudo service ntp stop
* Stopping NTP server ntpd [ OK ]
vagrant@precise32:~$ sudo ntpd -gq
sudo service ntntpd: time set +0.198630s
vagrant@precise32:~$ sudo service ntp start
* Starting NTP server ntpd [ OK ]
vagrant@precise32:~$ date '+%A %W %Y %X'
Friday 46 2015 11:42:22 PM
Raw
15 iptables
# 1) See what rules are currently loaded in the firewall.
vagrant@precise32:~$ sudo iptables -L -v
Chain INPUT (policy ACCEPT 79 packets, 4428 bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 41 packets, 3068 bytes)
pkts bytes target prot opt in out source destination
# 2) Explain why a whitelist approach is better than a blacklist approach for firewalls
#Whitelisting is inherently safer than Blacklisting. Why?
#Rather than defaulting to letting EVERYONE through and
#having to pick and choose our baddies (and potentially missing some),
#we can just assume everything is potentially harmful and
#only give access to those we know actually need (and deserve) it.
Raw
15.1 nmap
# 1) Scan http://scanme.nmap.org/ and tell me which ports are currently listening for connections remotely.
vagrant@precise32:~$ nmap scanme.nmap.org
Starting Nmap 5.21 ( http://nmap.org ) at 2015-11-22 17:12 UTC
Nmap scan report for scanme.nmap.org (45.33.32.156)
Host is up (0.053s latency).
Not shown: 992 closed ports
PORT STATE SERVICE
19/tcp filtered chargen
22/tcp open ssh
25/tcp filtered smtp
80/tcp open http
135/tcp filtered msrpc
139/tcp filtered netbios-ssn
445/tcp filtered microsoft-ds
31337/tcp open Elite
Nmap done: 1 IP address (1 host up) scanned in 2.31 seconds
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment