Example files for http://blog.byronjsmith.com/makefile-shortcuts.html, a default makefile for computational research projects

.gitattributes

*.ipynb	filter=dropoutput_jupyter
*.[tc]sv	diff=daff-csv
*.[tc]sv	merge=daff-csv

drop_jupyter_output.sh

#!/usr/bin/env bash
# run `chmod +x drop_jupyter_output.sh` to make it executable.

file=$(mktemp)
cat <&0 >$file
jupyter nbconvert --to notebook --ClearOutputPreprocessor.enabled=True \
    $file --stdout 2>/dev/null

Makefile

define PROJECT_HELP_MSG

Usage:
    make help                   show this message
    make clean                  remove intermediate files (see CLEANUP)

    make ${VENV}                make a virtualenv in the base directory (see VENV)
    make python-reqs            install python packages in requirements.pip
    make git-config             set local git configuration
    make setup                  git init; make python-reqs git-config

    make start-jupyter          launch a jupyter server from the local virtualenv

endef
export PROJECT_HELP_MSG

help:
	echo $$PROJECT_HELP_MSG | less

.git:
	git init

git-config: | .git 
	git config --local filter.dropoutput_jupyter.clean \
        drop_jupyter_output.sh
	git config --local filter.dropoutput_jupyter.smudge cat
	git config --local core.page 'less -x4'
	git config --local diff.daff-csv.command "daff.py diff --git"
	git config --local merge.daff-csv.name "daff.py tabular merge"
	git config --local merge.daff-csv.driver "daff.py merge --output %A %O %A %B"

VENV = .venv
export VIRTUAL_ENV := $(abspath ${VENV})
export PATH := ${VIRTUAL_ENV}/bin:${PATH}

${VENV}:
	python3 -m venv $@

python-reqs: requirements.pip | ${VENV}
	pip install --upgrade -r requirements.pip

setup: python-reqs git-config | .git ${VENV}

start-jupyter:
	jupyter notebook --config=jupyter_notebook_config.py

CLEANUP = *.pyc

clean:
	rm -rf ${CLEANUP}

.PHONY: help git-config start-jupter python-reqs setup clean

requirements.pip

jupyter
daff

Thanks for sharing this, there are a few really good ideas in here.

One thing I would note though is that the clean step is dangerous, because if ${CLEANUP} isn't what you expect you could end up destroying your system and losing data.

This is especially true when using larger makefiles and includes and where CLEANUP = definition may be further away from the target, such as if generalizing this to lots of projects.

You may be interested in having a look at my larger Makefile and Makefile.in in my DevOps-Bash-tools repo for tips too. I include that in all my other repos and source the Makefile.in so I don't have to repeat the same targets in lots of projects, and that way improvements refinements are shared to all of them similar to code libraries.

Huh. Hadn't thought about the risks yet of that particular cleanup recipe. I'll take a look at your stuff.

In general, I wouldn't use this makefile outside of a single, well defined, project directory, so I wasn't as concerned about broader fs structure.