Skip to content

Instantly share code, notes, and snippets.

@bsmith89
Last active April 5, 2021 23:37
Show Gist options
  • Save bsmith89/c6811893c1cbd2a72cc1d144a197bef2 to your computer and use it in GitHub Desktop.
Save bsmith89/c6811893c1cbd2a72cc1d144a197bef2 to your computer and use it in GitHub Desktop.
Example files for http://blog.byronjsmith.com/makefile-shortcuts.html, a default makefile for computational research projects
*.ipynb filter=dropoutput_jupyter
*.[tc]sv diff=daff-csv
*.[tc]sv merge=daff-csv
#!/usr/bin/env bash
# run `chmod +x drop_jupyter_output.sh` to make it executable.
file=$(mktemp)
cat <&0 >$file
jupyter nbconvert --to notebook --ClearOutputPreprocessor.enabled=True \
$file --stdout 2>/dev/null
define PROJECT_HELP_MSG
Usage:
make help show this message
make clean remove intermediate files (see CLEANUP)
make ${VENV} make a virtualenv in the base directory (see VENV)
make python-reqs install python packages in requirements.pip
make git-config set local git configuration
make setup git init; make python-reqs git-config
make start-jupyter launch a jupyter server from the local virtualenv
endef
export PROJECT_HELP_MSG
help:
echo $$PROJECT_HELP_MSG | less
.git:
git init
git-config: | .git
git config --local filter.dropoutput_jupyter.clean \
drop_jupyter_output.sh
git config --local filter.dropoutput_jupyter.smudge cat
git config --local core.page 'less -x4'
git config --local diff.daff-csv.command "daff.py diff --git"
git config --local merge.daff-csv.name "daff.py tabular merge"
git config --local merge.daff-csv.driver "daff.py merge --output %A %O %A %B"
VENV = .venv
export VIRTUAL_ENV := $(abspath ${VENV})
export PATH := ${VIRTUAL_ENV}/bin:${PATH}
${VENV}:
python3 -m venv $@
python-reqs: requirements.pip | ${VENV}
pip install --upgrade -r requirements.pip
setup: python-reqs git-config | .git ${VENV}
start-jupyter:
jupyter notebook --config=jupyter_notebook_config.py
CLEANUP = *.pyc
clean:
rm -rf ${CLEANUP}
.PHONY: help git-config start-jupter python-reqs setup clean
@HariSekhon
Copy link

HariSekhon commented Oct 17, 2019

Thanks for sharing this, there are a few really good ideas in here.

One thing I would note though is that the clean step is dangerous, because if ${CLEANUP} isn't what you expect you could end up destroying your system and losing data.

This is especially true when using larger makefiles and includes and where CLEANUP = definition may be further away from the target, such as if generalizing this to lots of projects.

You may be interested in having a look at my larger Makefile and Makefile.in in my DevOps-Bash-tools repo for tips too. I include that in all my other repos and source the Makefile.in so I don't have to repeat the same targets in lots of projects, and that way improvements refinements are shared to all of them similar to code libraries.

@bsmith89
Copy link
Author

Huh. Hadn't thought about the risks yet of that particular cleanup recipe. I'll take a look at your stuff.

In general, I wouldn't use this makefile outside of a single, well defined, project directory, so I wasn't as concerned about broader fs structure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment