Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Example files for http://blog.byronjsmith.com/makefile-shortcuts.html, a default makefile for computational research projects
*.ipynb filter=dropoutput_jupyter
*.[tc]sv diff=daff-csv
*.[tc]sv merge=daff-csv
#!/usr/bin/env bash
# run `chmod +x drop_jupyter_output.sh` to make it executable.
file=$(mktemp)
cat <&0 >$file
jupyter nbconvert --to notebook --ClearOutputPreprocessor.enabled=True \
$file --stdout 2>/dev/null
define PROJECT_HELP_MSG
Usage:
make help show this message
make clean remove intermediate files (see CLEANUP)
make ${VENV} make a virtualenv in the base directory (see VENV)
make python-reqs install python packages in requirements.pip
make git-config set local git configuration
make setup git init; make python-reqs git-config
make start-jupyter launch a jupyter server from the local virtualenv
endef
export PROJECT_HELP_MSG
help:
echo $$PROJECT_HELP_MSG | less
.git:
git init
git-config: | .git
git config --local filter.dropoutput_jupyter.clean \
drop_jupyter_output.sh
git config --local filter.dropoutput_jupyter.smudge cat
git config --local core.page 'less -x4'
git config --local diff.daff-csv.command "daff.py diff --git"
git config --local merge.daff-csv.name "daff.py tabular merge"
git config --local merge.daff-csv.driver "daff.py merge --output %A %O %A %B"
VENV = .venv
export VIRTUAL_ENV := $(abspath ${VENV})
export PATH := ${VIRTUAL_ENV}/bin:${PATH}
${VENV}:
python3 -m venv $@
python-reqs: requirements.pip | ${VENV}
pip install --upgrade -r requirements.pip
setup: python-reqs git-config | .git ${VENV}
start-jupyter:
jupyter notebook --config=jupyter_notebook_config.py
CLEANUP = *.pyc
clean:
rm -rf ${CLEANUP}
.PHONY: help git-config start-jupter python-reqs setup clean
jupyter
daff
@HariSekhon

This comment has been minimized.

Copy link

commented Oct 17, 2019

Thanks for sharing this, there are a few really good ideas in here.

One thing I would note though is that the clean step is dangerous, because if ${CLEANUP} isn't what you expect you could end up destroying your system and losing data.

This is especially true when using larger makefiles and includes and where CLEANUP = definition may be further away from the target, such as if generalizing this to lots of projects.

You may be interested in having a look at my larger Makefile and Makefile.in in my DevOps-Bash-tools repo for tips too. I include that in all my other repos and source the Makefile.in so I don't have to repeat the same targets in lots of projects, and that way improvements refinements are shared to all of them similar to code libraries.

@bsmith89

This comment has been minimized.

Copy link
Owner Author

commented Oct 17, 2019

Huh. Hadn't thought about the risks yet of that particular cleanup recipe. I'll take a look at your stuff.

In general, I wouldn't use this makefile outside of a single, well defined, project directory, so I wasn't as concerned about broader fs structure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.