Make and Bash completion

.bash_completion

# bash completion for GNU make
# This has been modified from the version in
# `/usr/local/etc/bash_completion.d/make` so that file paths are also
# autocompleted.
# 
# To use, copy or append this to `~/.bash_completion`

have make || have gmake || have gnumake || have pmake &&
_make()
{
    local file makef makef_dir="." makef_inc cur prev i split=false

    COMPREPLY=()
    _get_comp_words_by_ref cur prev

    _split_longopt && split=true

    case $prev in
        -f|-o|-W|--file|--makefile|--old-file|--new-file|--assume-old|--assume-new|--what-if)
            _filedir
            return 0
            ;;
        -I|-C|--directory|--include-dir)
            _filedir -d
            return 0
            ;;
    esac

    $split && return 0

    if [[ "$cur" == -* ]]; then
        COMPREPLY=( $( compgen -W '-b -m -B -C -d -e -f -h -i -I\
            -j -l -k -n -o -p -q -r -R - s -S -t -v -w -W \
            --always-make --directory --debug \
            --environment-overrides --file --makefile --help \
            --ignore-errors --include-dir --jobs --load-average \
            --max-load --keep-going --just-print --dry-run \
            --recon --old-file --assume-old --print-data-base \
            --question --no-builtin-rules --no-builtin-variables \
            --silent --quiet --no-keep-goind --stop --touch \
            --version --print-directory --no-print-directory \
            --what-if --new-file --assume-new \
            --warn-undefined-variables' -- "$cur" ) )
    else
        # before we check for makefiles, see if a path was specified
        # with -C/--directory
        for (( i=0; i < ${#COMP_WORDS[@]}; i++ )); do
            if [[ ${COMP_WORDS[i]} == -@(C|-directory) ]]; then
                # eval for tilde expansion
                eval makef_dir=${COMP_WORDS[i+1]}
                break
            fi
        done

        # before we scan for targets, see if a Makefile name was
        # specified with -f/--file/--makefile
        for (( i=0; i < ${#COMP_WORDS[@]}; i++ )); do
            if [[ ${COMP_WORDS[i]} == -@(f|-?(make)file) ]]; then
                # eval for tilde expansion
                eval makef=${COMP_WORDS[i+1]}
                break
            fi
        done

        [ -n "$makef" ] && makef="-f ${makef}"
        [ -n "$makef_dir" ] && makef_dir="-C ${makef_dir}"

        # The original regex used with awk was:
        # `/^[a-zA-Z0-9][^$#\/\t=]*:([^=]|$)/`
        # That is, it included a '/' in the characters to NOT match which
        # caused tab completion to not work for paths. Without overriding
        # this, `make a<tab>` would complete to `make all` but
        # `make data/src/f<tab>` would not complete to complete to
        # `make data/src/file.csv`.
        COMPREPLY=( $( compgen -W "$( make -qp $makef $makef_dir 2>/dev/null | \
            awk -F':' '/^[a-zA-Z0-9][^$#\t=]*:([^=]|$)/ \
            {split($1,A,/ /);for(i in A)print A[i]}' )" \
            -- "$cur" ) )

    fi
} &&
complete -F _make make gmake gnumake pmake

# Local variables:
# mode: shell-script
# sh-basic-offset: 4
# sh-indent-comment: t
# indent-tabs-mode: nil
# End:
# ex: ts=4 sw=4 et filetype=sh

README.md

Make and Bash Completion

I find using GNU Make for data completion really helpful for data pipelines.

This process is well-documented in this guide from DataMade.

However, my Makefiles can get somewhat large and my data paths pretty long. I was frustrated that bash completion wasn't working the way I wanted it to. I wanted to type make and then start typing the target and then be able to tab-complete the rest of the target.

First, install bash-completion

Mac OS doesn't come with a robust set of bash completion rules. For example, while git comes when you install the XCode tools, git's bash completion, that doesn't come with git's bash completion which is super-useful for things like tab-completing branch names.

Luckily, you can install better bash completion with Homebrew.

This is as simple as:

brew install bash-completion

Override the make completion rules

However, this still wasn't working well with autocompleting make targets.

I took a look at the script that does the completion for make, which is installed at /usr/local/etc/bash_completion.d/make.

I noticed that the list of targets seemed to be generated by searching through the output of.

This is the relevant part of the script:

        COMPREPLY=( $( compgen -W "$( make -qp $makef $makef_dir 2>/dev/null | \
            awk -F':' '/^[a-zA-Z0-9][^$#\/\t=]*:([^=]|$)/ \
            {split($1,A,/ /);for(i in A)print A[i]}' )" \
            -- "$cur" ) )

It uses awk to parse through the output of make -qp. The -q runs make in "question" mode, which the manpage says causes make to:

not run any commands, or print anything; just return an exit status that is zero if the specified targets are already up to date, nonzero otherwise.

The -p prints makes "database", which is a list of targets and variables.

I noticed that the regex provided to awk, /^[a-zA-Z0-9][^$#\/\t=]*:([^=]|$)/ includes a '/' in the non-matching characters causing targets with paths in them to be excluded.

That is, make a<tab> would complete to make all but make data/out/da<tab> would not complete make data/out/data.csv.

To fix this, I copied /usr/local/etc/bash_completion.d/make to ~/.bash_completion, which I saw from looking in /usr/local/etc/bash_completion is sourced by that script. I then modified my version to remove / from the excluded list of characters for targets.

If I used make to build more software, this might not be a great idea because it might take a long time for make to run and print the targets each time I hit tab. However, I think it's worth it for my case where I'm using make to build data files from other data.

A similar approach

While tab-completion is super-helpful to me, I like this idea of defining a list target in your Makefile to list other targets. See this StackOverflow answer for a recipe on how to do that. You'll notice that the use of make -qp and awk together is very similar to the bash completion script.