Skip to content

Instantly share code, notes, and snippets.

View jelc53's full-sized avatar

Julian Cooper jelc53

  • Stanford University; BCG Gamma
  • Sydney, Australia
View GitHub Profile
@jelc53
jelc53 / pymc_pyro_install.sh
Last active May 14, 2024 09:16
installation guide for pymc and pyro conda env
# ensure conda-forge is set as the priority channel
conda config --add channels conda-forge
conda config --set channel_priority strict
# create conda env
conda create -c conda-forge -c nodefaults -n bayes_env "pymc=5.10" "numpy=1.25.2" "scipy=1.12.0" python=3.10.6
conda activate bayes_env
pip install numba
conda install -c conda-forge mamba
mamba install -c conda-forge nutpie
# Make sure latex installed (example for mac below)
brew install --cask mactex
# Install texlive package to interpret sphinx rst docs
brew install texlive
# Open TeX Live Utility and install for "latexmk"
Tex Live Utility > latexmk > install
# (Option 1) You should now be able to run the following command from the sphinx docs directory
@jelc53
jelc53 / setup_guide.md
Last active August 16, 2023 16:19
void linux distribution

Preparation: remove dual-boot, download theme images and conf repo to USB

  • Removing dual-boot: removing Windows from my lenovo dual-boot machine from both disk and grub menu https://forums.linuxmint.com/viewtopic.php?t=386957. To do this, install gparted (sudo apt install gparted) and unmount, delete and resize partitions from OS partition you want to hold onto (e.g. Ubuntu)
  • In the early stages of installation, access to github is limited and so best to store conf and any theme images on a USB you can mount and access from root terminal straight away

Installation: boot from live iso image of void (flash USB) and run void-installer wizard

  • Download void image: sha256sum.sig, sha256sum.txt and live version of void that matches operating system (base and non-musl) https://repo-default.voidlinux.org/live/current/
  • Prepare USB device: plug in USB, find sdX reference with fdisk -l, unmount with umount /dev/sdX, and write live image with `dd bs=4M if=/path/to/void-live-ARCH-DATE-VARIANT.iso
Finetuning processing
Step 1 (Optional): Train Classification Model
- data_dir = '/home/ubuntu/deepsolar/data/bdappv-france/ft_100/'
- old_ckpt_path = '/home/ubuntu/deepsolar/models/deepsolar_pretrained.pth'
- ckpt_save_dir = 'checkpoint/bdappv_ft100_w0.01'
- model_name = 'deepsolar_classification'
- imbalance_rate = 1
- learning_rate = 0.01
- weight_decay = 0.01
@jelc53
jelc53 / copy_out.sh
Created May 24, 2023 22:14
bash script that takes text file input of paths and copies files into new bar directory
#!/usr/bin/env bash
## The destination folder where your files will
## be copied to.
dest="bar";
## For each file path in your input file
while read path; do
## $target is the name of the file, removing the path.
## For example, given /foo/bar.txt, the $target will be bar.txt.
@jelc53
jelc53 / path_surgery.py
Created May 24, 2023 22:12
helper script to programmatically change file path text input file
import json
import os
from glob import glob
from collections import defaultdict
from argparse import ArgumentParser
import pandas as pd
TIME_LIMIT = 600
@jelc53
jelc53 / img_preproc.sh
Last active June 2, 2023 18:06
pre-processing image data for deepsolar
# download french dataset and unzip in data directory
# sub dir structure: data/bdappv-france/bdappv/[ign,google]/[img,mask]
mkdir bdappv-france
unzip bdappv.zip
# checkout retrain_pytorch github branch
git checkout retrain_pytorch
git pull
# create "dummy" data sub-directory for testing
@jelc53
jelc53 / astactic_runsheet.sh
Last active May 8, 2023 03:06
runsheet for coqgym's astactic evaluation
# checkout main git branch
git pull
git checkout master
# download and scp coqgym's astactic best model into model directory
# https://drive.google.com/drive/folders/1AzLaEpoGS3BPMUz9Bl63MHAFRqlF4CtH
scp model.pth [username]@scq2.stanford.edu:/scr2/[user_dir]/CoqGym-GNN/ASTactic/runs/astactic/checkpoints/model.pth
# evaluation runsheet (5 left to go!)
"weak-up-to", -- done
@jelc53
jelc53 / install_pyenv.sh
Created May 5, 2023 05:37
install pyenv on ubu 20.04
#!/bin/bash
sudo apt-get install git python3-pip python-is-python3 make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev curl libffi-dev
pip install virtualenvwrapper
git clone https://github.com/pyenv/pyenv.git ~/.pyenv
git clone https://github.com/pyenv/pyenv-virtualenv.git ~/.pyenv/plugins/pyenv-virtualenv
git clone https://github.com/pyenv/pyenv-virtualenvwrapper.git ~/.pyenv/plugins/pyenv-virtualenvwrapper
cat << EOF >> ~/.bashrc
@jelc53
jelc53 / max_pool.py
Last active March 22, 2024 10:44
options for implementing conv max pool
def max_pool_forward_naive(x, pool_param):
"""A naive implementation of the forward pass for a max-pooling layer.
Inputs:
- x: Input data, of shape (N, C, H, W)
- pool_param: dictionary with the following keys:
- 'pool_height': The height of each pooling region
- 'pool_width': The width of each pooling region
- 'stride': The distance between adjacent pooling regions