Code Team Server setup (as router, dns, git-server and pxe boot-loader)

team-server-readme.md

A. Share internet

Your machine/server has more than 1 physical interface (ethernet, wifi or 3G/4G)

check pci-e bus and nic capabilities

sudo lspci
sudo lspci -s <03:00> -vv | grep Lnk

setup serial console output (for headless devices)

Edit /etc/default/grub as follows: (e.g b-rate 9600)

GRUB_CMDLINE_LINUX_DEFAULT=""
GRUB_TERMINAL='serial console'
GRUB_CMDLINE_LINUX="console=tty0 console=ttyS0,9600n8"
GRUB_SERIAL_COMMAND="serial --speed=9600 --unit=0 --word=8 --parity=no --stop=1"

Run update-grub then reboot, connect through gtkterm

static ip (on LAN facing subnet interfaces)

sudo nano /etc/network/interfaces

...
iface ... (WAN interface)
dns-nameservers 8.8.8.8 (WAN interface dns required, or put in /etc/resolv.conf if have resolvconf package)
...
iface enp4s0 inet static (LAN interface)
        address 192.168.0.1/24
        post-up iptables-restore < /etc/iptables.up.rules
...

sudo /etc/init.d/networking restart

/etc/dnsmasq.conf

(dns, dhcp, tftp) set these lines and restart through sudo systemctl restart dnsmasq.service

...
listen-address=127.0.0.1,192.168.0.1
...
dhcp-range=192.168.0.50,192.168.0.200,12h
...

check dhcp-client lease

cat /var/lib/misc/dnsmasq.leases

//or

arp -a | grep 192.168.0
nmap -sn 192.168.0.* 

enable ip forwarding (required as gateway!)

//kernel configure (enable packets pass through)
sudo sysctl net.ipv4.ip_forward=1

//turn port forwarding on permanently
sudo nano /etc/sysctl.conf
    net.ipv4.ip_forward=1
sudo sysctl -p
sudo sysctl --system

Opt A: Install webmin and init its firewall defaults for iptables.

+MASQUERADE (allow LAN hosts to access internet, like SNAT but +conn_state)

after iptables init, go to nat table and add rules on the POSTROUTING chain

...
	If source is 192.168.2.0/24 and output interface is enp1s0
        If source is 192.168.3.0/24 and output interface is enp1s0
        If source is 192.168.4.0/24 and output interface is enp1s0
...

Opt B: Manually

+ip packet forwarding (LAN interface to WAN interface, and vise versa)

//firewall rules (don't drop when passing through)
sudo iptables -A FORWARD -i enp4s0f1 -o enp4s0f0 -j ACCEPT
sudo iptables -A FORWARD -o enp4s0f1 -i enp4s0f0 -j ACCEPT

+MASQUERADE (allow LAN hosts to access internet, like SNAT but +conn_state)

(must have iptable_nat.ko with ip_tables.ko)

The configure is in the nat table on the POSTROUTING chain, can't use input interface, specify source address/network only, as many subnets as needed

//check and apply SNAT (by ip) or Masquerade (interface) action to change src addr
sudo iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -o enp4s0f0 -j MASQUERADE

//[optional, only useful for 1-way request pass through]
sudo iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

+DNAT (allow WAN clients to access LAN host)

(must have iptable_nat.ko with ip_tables.ko)

//check and apply DNAT action to change dest addr
sudo iptables -t nat -A PREROUTING -i enp4s0f0 --dport 80 -j DNAT --to-destination 10.10.10.2

//[optional, only useful for 1-way request pass through]
sudo iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

persist the rules

sudo iptables-save > ~/.fw-rules
sudo iptables-restore < ~/.fw-rules

or

/etc/network/interfaces
iface <external facing interface> inet ...
    ...
    post-up iptables-restore < /etc/iptables.up.rules

debug

sudo tcpdump -i enp4s0f1 host 192.168.0.132
sudo systemctl status dnsmasq
cat /var/lib/misc/dnsmasq.leases
arp -a | grep 192.168.
nmap -sP 192.168.*

re-apply DNAT rules in firewall upon restart;
check /etc/resolv.conf for nameserver lines;
restart docker if dns changes;
change windows network connection ipv4 properties for dns server;

B. Setup a shared Git code server

repos base

touch /mnt/git-server/      #as base point for projects
chgrp -R git /mnt/git-server/
chmod g+rwx /mnt/git-server/
chmod g+s /mnt/git-server   #for auto group permission set to new files

new project

Warning: Do this in your VM instead of Host machine if using vagrant! Else you might get weird permission error and 502 bad gateway error when pushing. (If you have created bare repo directly on Host machine, reload your vagrant vm with same user that owns the folder.)

mkdir|cd /mnt/git-server/ProjectA
git init --bare --shared        #create project remote git base
chgrp -R git /mnt/git-server/   #re-run if --shared didn't work

developer joins the project

Option A: ssh

server side

Require sshd and [iptables rules dport/sport 22]

//Opt A: Add user with limited git-shell and group git without home dir
sudo adduser --shell $(command -v git-shell) --ingroup git --no-create-home

//Opt B: Add user to sudoer (need root and re-login) or git
sudo adduser <username> sudo
sudo adduser <username> git

(you can remove password requirement in sudo by adding `<username> ALL=(ALL) NOPASSWD: ALL` in /etc/sudoer.d/<useranme-nopass>, don't end with ~ or contain . in the file name)

//check user's current groups
groups <username>

client side

git clone <user>@repos-server:/mnt/git-server/ProjectA to access

Option B: Http(s)

server side

1 Install nginx and fcgiwrap (also password util)

sudo apt-get install nginx fcgiwrap apache2-utils

//sample fcgiwrap config in a nginx server block
see /usr/share/doc/fcgiwrap/examples/nginx.conf

2 Add to /etc/nginx/site-available/git-server

server {
    listen 80 default_server;
    #server_name  gitserver.example.com;
    auth_basic "Restricted";
    auth_basic_user_file /etc/nginx/.htpasswd;

    location ~ /git(/.*) {
        fastcgi_pass  unix:/var/run/fcgiwrap.socket;
        fastcgi_param SCRIPT_FILENAME /usr/lib/git-core/git-http-backend;
        fastcgi_param GIT_HTTP_EXPORT_ALL "";
        fastcgi_param GIT_PROJECT_ROOT    /mnt/git-server;
        fastcgi_param PATH_INFO           $1; #use (/.*) in the captured uri in location;
        # Pass authenticated username to CGI app
        fastcgi_param REMOTE_USER $remote_user;
        include       fastcgi_params;
    }
}

3 Create password file (HTTP BasicAuth)

sudo htpasswd -c /etc/nginx/.htpasswd <first team.member>
sudo htpasswd /etc/nginx/.htpasswd <another team.member>

4 Get www-data access to /mnt/git-server

sudo chgrp -R www-data /mnt/git-server

5 Generate cert for ssl/tls (https)

sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
    -keyout ./https.key -out ./https.crt \
    -subj "/C=US/ST=CA/L=Sunnyvale/O=Stagejs/OU=Web Application Team/CN=demo.wat-stagejs.com"

6 Change server conf (https)

    # ==Bind==
    listen       443 ssl;
    server_name localhost; #(domain)
    
    # ==Options(ssl certificate only)==
    ssl_certificate https.crt;
    ssl_certificate_key https.key;
    
    ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
    ssl_session_cache shared:SSL:1m;
    ssl_session_timeout  25m;
    
    ssl_ciphers  HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;
    
    #... location ~ /git(/.*) ...

client side

GIT_SSL_NO_VERIFY=true git clone http(s)://<server ip>/git/ProjectA to access

If using self-signed ssl certificate, you can set sslVerify to false

git config [--global] http.sslVerify false

developer works on project

config

git config user.email "..."
git config user.name "..."
git config core.editor <nano>

push commit

git push origin master      #absolute first commit to create the *master branch during init commit.
git push origin <branch>

change last commit

git reflog
git reset <ref>
git add --all
git commit --amend

*squash last n commits (interactively)

git rebase -i HEAD~5

reset CRLF

git config core.autocrlf <input/true/false>
git rm --cached -r .
git reset --hard
git add .
git commit -m "Normalize all the line endings"

checkout a remote branch

git branch -r   #see the list of remote branches on origin
git fetch origin
git checkout <remote-branch>    #without origin/...

change remote repo url

git remote set-url origin <new url>

create a local branch

git checkout -b <local-branch> 
git checkout -b <local-remote-branch> origin/<remote-branch>

create a patch (with n latest commits)

git format-patch HEAD~<n> --stdout > patchfile.patch

create a patch (with diff so far, compare to other branch)

git format-patch <master branch> --stdout > patchfile.patch

apply a patch

git am *.patch

merge with another branch

git merge <master branch>
git merge --squash <master branch>

resolve pull/merge conflicts (honor current branch)

grep -lr '<<<<<<<' . | xargs git checkout --ours

resolve pull/merge conflicts (honor the other branch)

grep -lr '<<<<<<<' . | xargs git checkout --theirs

resolve pull/merge conflicts (per file base)

git checkout --ours PATH/FILE
git checkout --theirs PATH/FILE

find things back after --hard reset

git fsck --unreachable

re-apply .gitignore

git rm -r --cached .
git add .
git commit -m ".gitignore re-applied"

ignore but keep a sub-set of files

# .gitignore

runtime/doc/*
!runtime/doc/*.txt