Skip to content

Instantly share code, notes, and snippets.

@rajatmohanty

rajatmohanty/FM-fullstack.md

Forked from Oliver-ke/FM-fullstack.md
Created April 18, 2020 17:39
Show Gist options
  • Save rajatmohanty/afe3d673ae03c269b36f788bf443ab80 to your computer and use it in GitHub Desktop.
Save rajatmohanty/afe3d673ae03c269b36f788bf443ab80 to your computer and use it in GitHub Desktop.
Download ZIP
Note on Full Stack development at Frontend Masters
Raw
FM-fullstack.md

Note on Full Stack development at Frontendmasters.com

note: exclude the leading $ for commands, that just shows it a bash script and is meant to be run in a terminal

Full Stack => Knowing everything up from the front to the back

Frontend can be

  • Cars, -television, -browser, -etc

backend can be

  • API's, -platform, -Database -Security,

Full Stack: fullstack therefore is someone who can build an application from start to finish.

The command Line

Why the command line

  • not all servers have a GUI
  • speed
  • consistency
  • GUI are optionated
  • work everywhere
  • automate

Some common commands

  • $ cd - change directory
  • $ pwd - show the current working directory
  • $ ls - list all the files
  • $ cd - change directory
  • $ mkdri - use to make a directory
  • $ rmdir - removes a directory
  • $ cat - show file content
  • $ man - command manual
  • $ less - show file content type by page
  • $ rm - remove file
  • $ echo - repeat input

The shell

the shell is a command interpreter to interface with system While the terminal like powershell, cmd, bash are used to run shell application shell => application => Kernel to know your shell use $ echo $0

How does the internet work

The internet is a bunch of computers talking to each other sending and receiving information. www, ftp, torent, p2p

  • A series of globally interconnected devices
the Intranet => this is a private internet, inaccessable from the outside

this computers talk and identify each other using an IP, Mac address using protocol such as TCP and UDP Try pinging google with $ ping google.com

DNS => domain name protocol, to translate domain name to ip addresses

DNS resolves to the closest server. Domains e.g block.oliver.com .com => top level domain (tld) oliver.com => domain blog.oliver.com => subdomain

Get the routes(list number of hops) $traceroute google.com

Packet => a little bit of information you can transmit
  • it has senders and receivers address
  • information, protocol, meta data

VIM

VIM stands for Vi improved

  • this allows you edit file on terminal
  • found on every server

VIM modes

  • insert mode,
  • Command Mode,
  • Last line Mode

in VI

  • you cant use your mouse
  • just the keyboard
  • understand the commands

Getting to the modes

  • insert mode => i
  • Command mode => ESC
  • Last line mode => :
How to quit VIM

$ ESC :q! To use VIM on terminal => $ vim

Vim Commands:
  • $vi <filename> => this create a file with the given name or opens an existing one
  • $:w => Write to a file
  • $:wq => Write to a file and quite
  • $set number => to add line numbering to editor
  • $set nonumber => hide line numbering to ediot

Servers

A server wait, and responds to request

  • any computer can be a server, phone, computer, resbery pi, microcontroller
  • anything that responds to request can be regarded as a server HTTPS runs on port: 443
  • servers can be stacked to respond to many network request
  • servers usually live in data centers with huge security and low down time
  • elastic computing => scalling at time needed and using only the resource you need
Buying a virtual private server (VPS) on digitalocean
  1. Register for an account
  2. create a droptlet, a droptlet is a form of VPS on digital ocean
  3. select an image, image is copy of an operating system
  4. Select a server authentication method (SSH)
Operating system (Linux)
  • kernal -> this is the layer that talks to the hardware
  • Utilities -> this are tiny little application that do generally one thing,
  • they can interact with the kernal and other application in the system
Secure Socket Shell (SSH):

A more secure way of loggin in to a server comprises of PRIVATE and PUBLIC key pairs

  • private key stays on your computer
  • while Public key stays on the server While logging in everything is encrypted with the private key and only the corresponding public key can decrypt it. $ cd ~/.ssh => change directory to your ssh directory $ ssh-keygen => Generate a new ssh keys $ ls | grep tieme => this is a search command after piping using | then grep and a name to search for timemechine <= private key timemechine.pub <= public key
SSH into the server

$ ssh root@your_ip - this command enables you to ssh into the server $ ssh -i <privatekeyname> root@your_ip - ssh by specifying the private key you wish to use, you can add -v for debuggin

After logging in the # infront means we logged in as a super user $ whoami - this tells you your current user type

Buy a domain

DNS Records A records => this maps names to IP addresses CNAME => this maps name to name blog.oliver.com => CNAME => oliver.com oliver.com => A => 192.05.21 $ dig <domain name> - you can use this to get information about a given domain

Setting dns on server

NS => name servers, this is what your domain provider should point to setup the server

  1. Update software
  2. Create a new user
  3. Make that user a super user
  4. Enable login for new user
  5. Disable root login

Commands

$ apt update => update software $ apt upgrade => upgrade software $ adduser $USERNAME => add a new user $ usermod -aG sudo $USERNAME => Add user to "sudo" group $ su $USERNAME => Switch user $ sudo cat /var/log/auth.log => check sudo access by viewing auth logs $ sudo tail -f /var/log/auth.log => keeps following the log file $ mkdir -p ~/.ssh => make an ssh directory $ vi ~/.ssh/authorized_keys => create authorized_keys file and paste public key

Disable root user

$ chmod 644 ~/.ssh/authorized_keys => change file permissions $ sudo vi /etc/ssh/sshd_config => Disable root login $ sudo service sshd restart => Restart ssh daemon

Nginx

  • Reverse proxy
  • web server
  • proxy server
  • caching server $ sudo apt install nginx => installs nginx $ sudo service nginx start => start nginx Application Layers DOMAIN => IP => NGINX => WEB APPLICATION Nginx config $ sudo less /etc/nginx/sites-available/default $ ssh -i timemachine oliver@oliver-ke.codes - server ssh example $ ssh -o ServerAliveInterval=30 -i timemechine oliver@oliver-ke.codes - with disconnection interval time

note: whatever you wish to do with nginx, you might find an article for it

Create and edit index.html $sudo vi /var//www/html/index.html => Nginx is a web server, that does the management of trafficking

Installing and setting up node

$ sudo apt install nodejs npm $ sudo apt install git

Application Architecture -> the structure and order of your application plays a huge role in development

$sudo chown -R $USER:$USER /var/www - change ownership of ww directory to current user $ mkdir /var/www/app - create application directory $cd /var/www/app && git init initialize your app directory as a git repository

Proxy traffic from Nginx to express server
location / {
	proxy_pass URL_TO_PROXY_TO;
}

e.g URL_TO_ROXY_TO: http://127.0.0.1:3000;

Restart nginx $sudo service nginx reload

Using process manager to

  • Keeps you application running
  • Handles errors and restart
  • can handle logging and clustering Steps
  1. $sudo npm i -g pm2 - install pm2
  2. $ pm2 start app.js - start pm2
  3. pm2 startup - setup auto restart

Some quick note on pm2

  • PM2 is a Production Process Manager for Node.js applications
  • with a built-in Load Balancer.

Start and Daemonize any application: $ pm2 start app.js

Load Balance 4 instances of api.js: $ pm2 start api.js -i 4

Monitor in production $ pm2 monitor

Make pm2 auto-boot at server restart: $ pm2 startup

To go further checkout: [http://pm2.io/]

Turning off the server: $sudo poweroff

$pm2 status - view pm2 service

Setting Up github on server

  1. create git repository
  2. create ssh key
  3. add ssh key to github
  4. add remote repo
  5. push local repository to github

Other exploration Install Fail2ban :- this prevent wrong access from hitting the server

DAY 2:

Standard Streams

  • standard output: stdout
  • standard input: stdin
  • standard error: stderr Redirection | - read from stdout > - write stdout to file >> - append stdout to file < - read from stdin 2> - read from stderr

examples $ps | grep bash - grep search output for the given word

Finding things

$find - search file names $grep - seach file content Example $find /bar -name foo.txt -> /bar - the directory to seach for -> -name - the option to search for, in this case name -> foo.txt - file/folder find all the log files in /var/log $find /var/log/nginx -type f -name "*.log" find all directory with the name of 'log' $find / -type d -name log finding things $grep -i 'jem' /var/www -i - options jem - search expression /var/www - directory $zgrep file - search inside gzip file $ ps aux - show all process and details $ps aux | grep node - find node from aux output

Nginx

=>Redirect:

location /help {
	return 301 https://developer.mozilla.org/en-us;
}

=>Adding a subdomian

server {
	listen 80;
	listen [::]80;
	server_name blog.oliver.codes;
	location / {
		proxy_pass http://localhost:3000;
	}
}

Gzip

this package is used for conpression, either lossy or lossless /etc/nginx/nginx.conf

##
# Gzip setttings
##
gzip on;
gzip_disable 'msie6"
....
note
- You don't gzip image files

Security

Read auth.log $sudo cat .var/log/auth

  • SSH
  • Firewalls
  • Update
  • Two Factor authentication
  • VPN

Update install unattended upgrades to make automatic update $sudo apt install unattended-upgrades view conf file $cat /etc/apt/apt.conf.d/50unattended-upgrades

Firewalls

A network security deveice that monitors incoming and outgoing network traffic and decides whether to allow or block specific traffic based on a defined set of security rules.

Running port scan

install nmap $sudo apt install nmap $man nmap

Run nmap $nmap YOUR_SERVER_IP_ADDRESS

Run nmpa with more service/version info $nmap -sV YOUR_SERVER_IP_ADDRESS

=> you see with the use on nmap you can see with ports are open and to what process that runs PORT :- this is a communication endpoint that maps to a specific process or network service

We would want to close port 3000, this is not needed for general internet Lets do this using uncomplicated firewall (ufw) $sudo ufw status - check ufw status $sudo ufw enable - starts ufw $sudo ufw allow ssh - enable ssh $sudo ufw -help - to get more info $sudo ufw enable - enable firewall

Permissions

Controlling read, write, execute and modify files

Upgradding node

Download setup script from nodesource $curl -sL https://deb.nodesource.com/setup_10.x -o nodesource_setup.sh Run script $sudo bash nodesource_setup.sh Install nodejs $sudo apt install nodejs

Alternative to Man page => explainshell.com

Updating global packages => $ sudo npm update -g

Part 3 (HTTP)

http => hypertext transport protocol Http is transported over tcp. http does REQUEST and RESPONSE components of http -HEADERS this is the meta data, it give information about the request such as location, purpose etc -HOST -USER-AGENT -ACCEPT -ACCEPT-ENCODING -ACCEPT-LANGUAGE -X- a way of setting custom header

HTTPS => secure layer over HTTP

Adding HTTPS encryption using certbot website: https://certbot.eff.org Cerbot created a wrapper on letsEncrypt Go to the cerbot website and follow throgh

  1. SSH into the server
  2. Add cerbot PPA
  3. Install certbot
  4. Choose how you would like to run certbot e.g sudo certbot --nginx note: remember to add your server name in the nginx configuration file directory: $ sudo vi /etc/nginx/site-available/default
 set server_name oliver-ke.codes www.oliver-ke.codes

also allow https: $ sudo ufw allow https

listen 443 https ssl; reload nginx: $ sudo service nginx reload

Containers

Microservices :- architecture of loosely connected services, one that does a single thing it different from a monolith which a single application Containers are used to power cloud services, it allows you to include just what your application needs rather than the entire OS, this is done using container manager Containers are independent on the mechine they currently run, and there for can run on any operating system. some features includes

  • lightweight
  • portable
  • Easier for development
  • Easier to manage
  • Faster startup
  • Decouple application from infrastructure

Docker :- A container service

Orchestration : A way to manage several container services, pevent errors and maintain proper load balancing amongs the servers

Kubernetes (k8s) an Orcestration layer. it gives you control over all containers e.g Docker is the sheeps and Kubernetes is a shephard

Elastic Computing => Expanding and contracting resources based on your need. more like paying for what's used. this is done using load balancing technique

| Load Balancers: this is a tool that passes traffic amongs your servers to ensure none is over or under working. they work using a scheduling algorithm such as

  • Roud Roin, - IP hashing, Random choice, Least connections, least load To view running processes on your server do the following $ top - display running processes $ sudo apt install htop - install htop $ htop - display running processes

Using Nginx as a load balancer

upstream backend {
	least_conn;
	server backend1.example.com;
	server backend2.example.com;
	server 192.0.0.1 backup;
}
server {
   location / {
	proxy_pass http://backend;
  }
}

to prevent session auth which is server dependent you can implement Ip hashing provide an authorization layer on top of the load balancer so all traffics passes throgh that server before reaching your load balancer and then routed to any available server.

Deployment

tools :

  • Ansible
  • Vagrant
  • Puppet

Saving Data

-> Files, -> Database

Relational Database: MySQL, PostgreSQL, SQLServer

Non-relational Database: MongoDb, Redis, Elastic search

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment