Dockerfile alternatives for heredoc

heredoc-dockerfile.snip

#printf
RUN printf '#!/bin/bash\n\
echo hello world from line 1\n\
echo hello world from line 2'\
>> /tmp/hello

#echo
RUN echo -e '#!/bin/bash\n\
echo hello world from line 1\n\
echo hello world from line 2'\
>> /tmp/hello

Thanks. Seems so simple in retrospect but found this via google while wondering how to do multiline strings in a Dockerfile I was making.

thanks!

I like wrapping it in braces like they do in the official MySQL image:

RUN { \
		echo mysql-community-server mysql-community-server/data-dir select ''; \
		echo mysql-community-server mysql-community-server/root-pass password ''; \
		echo mysql-community-server mysql-community-server/re-root-pass password ''; \
		echo mysql-community-server mysql-community-server/remove-test-db select false; \
	} | debconf-set-selections

@adamelliotfields those braces are shell syntax that groups a set of commands; they don't affect Docker's parsing at all. See https://www.gnu.org/software/bash/manual/html_node/Command-Grouping.html#Command-Grouping . What that's doing is aggregating the stdout of all the commands in the brace-group and passing it all to debconf-set-selections. Nothing to do with a here-doc.

If Docker would let you specify the shell, you could use bash's <<<$'blah' construct like

RUN cat >>somewhere <<<$'\
whatever\n\
I\n\
want\n'

but it'll choke with "redirection unexpected" because it'll try to run with /bin/sh which is dash on Debian/Ubuntu.

@ringerc: https://docs.docker.com/engine/reference/builder/#shell

RUN printf '%s\n' \
  '#!/bin/bash' \
  '' \
  'set -xe' \
  '' \
  'INTERFACES="$(ip -o link | sed -ne "s/[[:digit:]]*: \\(eth[[:digit:]]*\\)[^$]*/\\1/p")"' \
  '' \
  'echo "${INTERFACES}"' \
  \
> /entrypoint.sh && chmod a+x /entrypoint.sh

A more readable version imo.

👍

You can base64 encode the heredoc contents and then have the docker build process decode them.

A python example:

from textwrap import dedent
import base64
heredoc = dedent(
    """
    echo 'your
    script
    goes
    here'
    """
)
heredoc_b64 = base64.b64encode(bytes(heredoc, "utf-8")).decode()
dockerfile = dedent(
    f"""
    FROM whatever
    RUN echo '{heredoc_b64}' | base64 -d  > myscript.sh
    RUN bash myscript.sh
    """
)

@MatrixManAtYrService while this is nice, it completely obfuscates what is going on. I think the other approaches also had the intention to show the content of the script (e. g. maybe because some paths or variables are being used or even substituted).
Otherwise yes, this would good for longer scripts and to just use a single Dockerfile without anything else. But then maybe also add compression (if gzip is installed) for base64 encoding to somewhat decrease the size of the string. Not sure if base64 is a default package/tool for most images?

Finally buildkit added support for heredocs : https://www.docker.com/blog/introduction-to-heredocs-in-dockerfiles/

@Querela agreed, it's not a good solution for persistent Dockerfiles. Later on in the script I pipe the string into docker build - so there's never a Dockerfile for anyone to read and be confused about. heredoc support in buildkit seems even better though.

Yes, heredoc support is great.
I did some decrypting of data etc in a dockerfile entrypoint and embedded the script so no external files were required. But developing this took longer than expected because of different levels of quote escaping, subshells and parameter substitution or escaping ... It was fun but nothing you would want to revisit later. And a nightmare to debug.
An excerpt below: (DECRYPT_* are docker run environment variables)

RUN printf '%s\n' \
    '#!/bin/bash' \
    '' \
    'DATA_DIR=/mnt/data' \
    'echo "Data dir: ${DATA_DIR} ... $(du -sh ${DATA_DIR} | cut -f1)"' \
    '' \
    'echo "* Decrypt data ..."' \
    ' ( \' \
    '   KEY="${DECRYPT_KEY}" IV="${DECRYPT_IV}" \' \
    '   bash -c " \' \
    '       find ${DATA_DIR} -iname '"'"'*.jsonl.bz2.enc'"'"' -type f -exec \' \
    '       bash -c '"'"' \' \
    '           FILE="\${0%.enc}"; \' \
    '           openssl aes-256-cbc -K \${KEY} -iv \${IV} -d -in \"\${FILE}.enc\" -out \"\${FILE}\"' \
    "       '"' {} \; \' \
    '   " \' \
    ' )' \
    '' \
    'echo "* Decompress data ..."' \
    'find ${DATA_DIR} -iname '"'*.jsonl.bz2'"' -type f -exec bzip2 -d {} \;' \
    '' \
    'echo "* Cleanup data ..."' \
    '# remove encrypted and compressed data' \
    'find ${DATA_DIR} -iname '"'*.jsonl.bz2.enc'"' -type f -exec rm {} \;' \
    'find ${DATA_DIR} -iname '"'*.jsonl.bz2'"' -type f -exec rm {} \;' \
    '' \
    'echo "Data dir (ready): $(du -sh ${DATA_DIR} | cut -f1)"' \
    '' \
    'echo -e "\nDataset list:"' \
    'ls -1shR ${DATA_DIR}/*' \
    '' \
    > prepare_data.sh && \
    chmod +x prepare_data.sh

Also, there is official way to do it:

# syntax=docker/dockerfile:1.3-labs

RUN <<EOF
echo "Hello" >> /hello
echo "World!" >> /hello
EOF

I use it as the following:

# syntax=docker/dockerfile:1.4
RUN cat <<EOF >> /usr/local/bin/startup.sh
if [ \$# -gt 0 ]; then
    exec "\$@"
else
    /usr/bin/tini -- /usr/local/bin/jenkins.sh
fi
EOF