Engineering Onboarding Docs

01_resources.md

Resources

Last updated: 2023-08-25

• PEP 8 -- Style Guide for Python Code
• PEP 20 -- The Zen of Python
• Two Scoops of Django
• Python, Django docs
• @kavdev

02_git_policies.md

Git Policies

Last updated: 2023-08-25

Branch Flow

We use the GitFlow Workflow without release branches. When creating a branch, use the {type}/{id}_{short_title}. For example, feature/XXX_12345_fancy_new_feature.

History

A clean git history is important. See this medium article. TL;DR set git config --global pull.rebase merges and GitHub will take care of the rest with it's PR merging system.

For each commit you make, include the issue id. For example, git commit -m "added x to y to improve z. XXX-123456". This helps us relate commits to issues.

We assume you know how to make atomic commits. If you don't, read the article first and then ask questions -- we'll be happy to explain.

Your commit history should tell a story about what's going on in a pull request. During code review, code will be inspected commit-by-commit. If something doesn't make sense in a commit given the commit message, it will get flagged during review.

As explained in the article, don't push garbage commits. If you have one or two test/bugfix commits that would take too much time to put into the right commit, don't worry about it.

If there's a conflict when you attempt to merge your feature branch into development, use git fetch origin;git rebase -p origin/development

You should really learn how to interactive rebase. It's immensely helpful when cleaning up your history. We're happy to help you learn how to do an interactive rebase.

Cloning and submitting commits

• Make sure you use SSH
• Make sure you use sign your commits

03_package_policies.md

Installing Packages

Last updated: 2023-08-25

License Requirements

Any installed packages must have an MIT (or equivalent) license. Including restrictively-licensed packages in a pull request will cause it to be rejected.

04_testing_linting_policies.md

Testing

Last updated: 2023-08-25

Running Tests

python manage.py test

Writing Tests

Learn how to write (meaningful) unit tests using the unittest standard library. There are plenty of examples in our codebase to draw from.

• Your code can't be merged without tests.
• Your code won't pass code review if your tests aren't solid (e.g. bare minimum run-through tests written to pass coverage)
• Any time you're writing a bugfix, write a test that reproduces the bug (preferably before you fix the bug).

Code Coverage

We have a 100% (ish) line and branch coverage policy. This is fairly controversial in that an argument is made that devs will see this as a way to slack on writing good tests. We see it as a good guideline both for you when you're writing your tests (to check that each logic branch is tested) and for the reviewers on your PR (to see that tests were at least written before diving in-depth). We do allow # pragma: no cover statements so long as you mark yourself as the author and provide a reason (e.g. # pragma: no cover ## author: @kavdev reason: simple builtin accessor). Your PR reviewer can reject your reason, so err on the side of testing your code.

• Your code can't be merged unless it passes coverage.
• Your code won't pass code review if your tests are bad (e.g. bare minimum run-through tests written to pass coverage)

Linting

We use Black to auto-format python and Prettier to format javascript (and pretty much everything else). Make sure you have them installed and enabled in your IDE.

• Your code can't be merged unless it passes linting.

05_env_setup.md

Set up your development environment

Last updated: 2023-08-28

This doc is meant to be as universal as possible. Some steps might only be applicable to some operating systems. We'll notate which operating systems each step applies to.

If you notice any errors or find alternatives on a step for a given operating system, reach out.

There might be some issues between intel and m-series mac installs -- please reach out if you encounter them and have solutions.

Step 0: Install WSL2 [Windows]

WSL2 enables windows developers to work directly with linux. This guide supports 22.04.2 LTS. VS Code can talk to WSL using the remote extensions package. Once you install WSL, jump into your Ubuntu terminal for the rest of this document.

https://learn.microsoft.com/en-us/windows/wsl/install

Step 1: Install Tools and Packages

Install ntp [Ubuntu]

NTP (Network Time Protocol) enables your system to always be at the correct time. Sometimes it's not included, which can cause odd issues during testing.

sudo apt-get update
sudo apt-get install ntp ntpdate
sudo ntpdate ntp.ubuntu.com

---

Install Homebrew

Homebrew is a cross-platform package manager.

https://brew.sh/

• [Ubuntu] gcc should already be installed on ubuntu, so don't brew it
• (optional) Run brew analytics off after installing to turn off analytics

---

Install PyGraphviz dependencies

We use graphviz to output uml-style diagrams of our models. This is a dependency for the python package used to do so.

https://pygraphviz.github.io/documentation/stable/install.html#install

• Do not execute the pip install as shown in those instructions.

Known Issue on M1 Mac

There's an issue that pops up when installing the pygraphviz library on M1 macs. It should be resolved in the future, but for now, you'll need to do some special library linking to get the install to work. The installation docs have a solution, but in case that doesn't work, read through the following issue to see if one of the proposed solutions works:

pygraphviz/pygraphviz#11

---

Install latest git

We use git for version control. It's how we track and deploy code changes. See git_policies.md for what we expect.

https://git-scm.com/downloads

• When following the Ubuntu instructions, use the PPA installation method.

---

Install Postgres

We use postgres for the primary database.

[Ubuntu] https://learn.microsoft.com/en-us/windows/wsl/tutorials/wsl-database#install-postgresql

• Also include postgresql-postgis in the install command
• Do not set a password for the postgres user. You can if you want to, but it's not necessary.
• After installing:
  • Create a user and database with your username to make cli life a little easier:
    • sudo -u postgres -i
    • createuser -l -d -P <your_username>
    • createdb <your_username> -O <your_username>
  • Find the path to the hba file: psql -t -P format=unaligned -c 'show hba_file';
  • Edit that file, replacing all of the auth methods in the last column with trust
    • If you plan on storing any kind of sensitive data in your database, don't do this step.
  • Restart postgres: sudo service postgresql restart
  • Test that everything worked: psql

[macOS] https://postgresapp.com/

• The mac package should install with posgis and its required dependencies, but check the guide for more info.

---

Install Redis

We use redis for caching.

[Ubuntu] https://learn.microsoft.com/en-us/windows/wsl/tutorials/wsl-database#install-redis

[macOS] https://redis.io/docs/getting-started/installation/install-redis-on-mac-os/

---

Install python

You need python to run the backend. Install stable latest and latest-dev. See "Finishing Touches" in the article for latest-venv.

[Ubuntu] https://hackersandslackers.com/multiple-python-versions-ubuntu-20-04/ (This guide is for an older Ubuntu version, but it's still a great resource)

• Where it says python3.10, instead use whatever python minor version is listed in the pyproject.toml. At the time of writing, that's python3.11.
• Where it says python3.8, instead use python3.10 (3.10 is the default python in Ubuntu 20.04)
• The config switch will probably have 0 (auto mode) set to python 3.11. If that's the case, keep it that way.

[macOS (via homebrew)] https://docs.brew.sh/Homebrew-and-Python

[macOS (official installer)] https://www.dataquest.io/blog/installing-python-on-mac/

After you install latest pip, install latest wheel and setuptools python packages:

pip3 install -U wheel setuptools

---

Install Poetry

poetry is used to manage python environments. You need it to install python packages.

https://python-poetry.org/docs/

(optional) You can optionally install poems, a cli tool that helps you easily jump into different python environments: https://poetry-poems.readthedocs.io/en/latest/

---

Install nvm

nvm (Node Version Manager) is used to install and manage node versions for javascript. You need node to run the frontend. See package.json to see which version you should install.

https://github.com/nvm-sh/nvm

---

Install Yarn

yarn is a package manager for javascript. You need it to install javascript packages.

https://yarnpkg.com/getting-started/install

• Don't go past corepack enable

Step 2: Install and configure your IDE

We support VS Code. If you're using a different code editor, you're on your own (for the most part). We have a vscode project folder with settings and extensions, and other recommended user settings are in vscode_config.md.

Step 3: Set up Git, SSH Auth, and Commit Signing

Set up authentication and signing keys in GitHub

https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account?tool=webui

https://blog.dbrgn.ch/2021/11/16/git-ssh-signatures/

It's recommended to use your ssh key to both auth to GitHub and sign your commits. To enable that on GitHub, add the same key as both an auth and signing key in the GitHub ui.

If you'd rather use GPG to sign your commits, you're more than welcome to: docs.

(Optional) Hook into 1Password's SSH Agent

Configure SSH Auth and commit signing via 1Password [WSL]

https://developer.1password.com/docs/ssh/integrations/wsl/

Enable Git rerere (you'll probably want this)

https://git-scm.com/blog/2010/03/08/rerere.html

Set vscode as your default git editor

You might also want to make vscode your default git editor: https://www.roboleary.net/vscode/2020/09/15/vscode-git.html.

Part of that might require adding the following to your .bashrc on Ubuntu:

export EDITOR="code --wait --new-window"
export VISUAL="$EDITOR"

Example git config

At the end of all this, your .gitconfig file should look something like this:

[core]
  editor = code --wait

[user]
  name = Bob Jones
  email = bobjones@example.com
  signingkey = ########

[gpg]
  format = ssh

[gpg "ssh"]
  allowedSignersFile = ~/.config/git/allowed_signers

[diff]
  tool = vscode

[difftool "vscode"]
  cmd = code --wait --diff $LOCAL $REMOTE

[merge]
  tool = vscode

[mergetool "vscode"]
  cmd = code --wait $MERGED

[commit]
  gpgsign = true

[push]
  autoSetupRemote = true

[pull]
  rebase = merges

[rerere]
  enabled = true

• Your signing key is your public key without the email at the end.
• The allowed_signers file includes any public keys with the email at the beginning instead of the end.

Step 4: Configure your system

Add some aliases

Add postgres host, user, and password to make connecting easier (Note that you might need to also change your pg_hba.conf file and set the auth options all to trust):

export PGHOST=localhost
export PGUSER=postgres
export PGPASSWORD=#########

Add a copydb command to make copying databases in postgres as easy as the existing createdb and dropdb commands:

alias copydb='createdb -T'

Add django management aliases to save you some keystrokes:

alias pm='python manage.py'
alias pmt='pm test'
alias pmr='pm runserver'
alias pms='pm shell_plus'
alias pmu='unbuffer python manage.py show_urls --format aligned | grep -v /kitchen/ | grep -v /__debug__/ | grep -v /media/ | grep -v /static/'

Add some git aliases to make checking authors and comits quicker:

alias gitauthors="git log --all --format='%aN <%aE>' | sort -u"
alias gitcommits="git shortlog -s -n --all --no-merges"

Add some git aliases to make cherry-picking dependencies quicker (I usually have one terminal open for cherry-picking, one for reset, one for locking/installing, and one for continuing):

alias cherryreset="git add *.lock && git reset *.lock && git restore *.lock"
alias cherrycontinue="git add . && git cherry-pick --continue"

---

Style your prompt with oh-my-posh

This is totally optional, but it makes your terminal experience so much nicer.

https://ohmyposh.dev/docs/themes

---

Make vim less annoying to use with spf13

This is also optional, but life is so much better with spf installed.

http://vim.spf13.com/

.vimrc.before.local

let g:spf13_no_autochdir = 1
let g:airline_powerline_fonts=1

.vimrc.local

set expandtab
set tabstop=4
set softtabstop=4
set shiftwidth=4
set autoindent
set smartindent
set nospell
set nofoldenable
set mouse=a

set background=dark
colorscheme jellybeans

function! s:swap_lines(n1, n2)
    let line1 = getline(a:n1)
    let line2 = getline(a:n2)
    call setline(a:n1, line2)
    call setline(a:n2, line1)
endfunction

function! s:swap_up()
    let n = line('.')
    if n == 1
        return
    endif

    call s:swap_lines(n, n - 1)
    exec n - 1
endfunction

function! s:swap_down()
    let n = line('.')
    if n == line('$')
        return
    endif

    call s:swap_lines(n, n + 1)
    exec n + 1
endfunction

noremap <silent> <a-up> :call <SID>swap_up()<CR>
noremap <silent> <a-down> :call <SID>swap_down()<CR>

06_vscode_config.md

Set up VS Code

Last updated: 2023-09-01

VS Code is the preferred IDE for our project. The project contains a .vscode folder with default settings and recommended extensions. The settings in that .vscode folder are pretty useful; you might consider copying some of them over to your user settings.

Some recommended settings tied to the UI only have an affect if stored in user settings. Those recommendations are listed below.

Recommended Settings

You should pick your own preferred font, but adding the nerd font for the default Cascadia Code font will help the vscode terminal render properly if you're using oh-my-posh

{
    "editor.fontFamily": "'CaskaydiaCove Nerd Font Mono', Cascadia Code Mono, Ubuntu Mono, Consolas, 'Courier New', monospace",
    "editor.fontLigatures": true,
    "editor.cursorBlinking": "phase",
    "editor.cursorSmoothCaretAnimation": "on",
    "workbench.tree.indent": 12,
    "workbench.editor.highlightModifiedTabs": true,
    "workbench.editor.untitled.hint": "hidden",
    "explorer.autoReveal": false,
    "telemetry.telemetryLevel": "off",
    "problems.showCurrentInStatus": true,
    "search.showLineNumbers": true,
    "svg.preview.mode": "svg",
    "terminal.integrated.sendKeybindingsToShell": true,
    "terminal.integrated.tabs.enabled": false,
    "terminal.integrated.cursorBlinking": true,
    "terminal.integrated.shellIntegration.suggestEnabled": true,
    "terminal.integrated.cursorStyle": "line",
    "terminal.integrated.minimumContrastRatio": 1,
    "terminal.integrated.smoothScrolling": true,
    "terminal.integrated.tabStopWidth": 4,
    "terminal.integrated.fontSize": 12,
    "python.languageServer": "Pylance",
    "git-graph.commitDetailsView.fileView.type": "File List",
    "git-graph.date.format": "Relative",
    "git.branchWhitespaceChar": "_",
    "git.showPushSuccessNotification": true,
    "githubPullRequests.pullBranch": "never",
}