Skip to content

Instantly share code, notes, and snippets.

@rnag
Last active August 17, 2022 22:14
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rnag/4e845ea85c79493978944998bdb1aa4f to your computer and use it in GitHub Desktop.
Save rnag/4e845ea85c79493978944998bdb1aa4f to your computer and use it in GitHub Desktop.
Python 3.9 Dockerfile using a Private PyPI Repository
# Ref:
# - https://pythonspeed.com/articles/base-image-python-docker-images/
# - https://github.com/FNNDSC/ubuntu-python3/blob/master/Dockerfile
#
# This Docker image uses the `ubuntu` base image for a slight performance
# improvement as mentioned in the article. At build time, you'll need to pass
# in the PyPI authentication info, as shown below:
#
# docker build -t ${image_tag_name} \
# --build-arg PYPI_URL=${PYPI_URL} \
# --build-arg PYPI_USER=${pypi_user} \
# --build-arg PYPI_PASS=${pypi_password} .
#
# Please note: if you are building the image on an M1 Mac, you'll need to
# pass `--platform=linux/amd64` to `docker build`, as mentioned here:
# https://stackoverflow.com/a/70614305/10237506
#
# Static Assets
FROM ubuntu:20.04
ARG PYPI_URL
ARG PYPI_USER
ARG PYPI_PASS
# Ref: https://dev.to/christo22694524/installing-private-python-packages-in-docker-images-1hgm
RUN echo "machine $(echo "$PYPI_URL" | awk -F[/:] '{print $4}') \
" login ${PYPI_USER} \
" password ${PYPI_PASS}" > /root/.netrc && \
chown root ~/.netrc && \
chmod 0600 ~/.netrc
WORKDIR /app
COPY requirements.txt /app/
# Install Python dependencies
RUN apt-get update && \
# Installing `python3.9-distutils` appears to install Python 3.8 as well,
# so we'll need to remove Python 3.8 in the same RUN command to reduce the
# total image size.
apt-get install -y python3.9 python3.9-distutils curl && \
ln -s /usr/bin/python3.9 /usr/local/bin/python && \
# Now we'll need to install `pip` (note that this requires `distutils`)
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python get-pip.py && \
rm get-pip.py && \
# The problem here is that the `pip install`s seem to still require
# Python 3.8, so we'll need to have them in the same RUN command.
pip install -U --no-cache-dir \
pip && \
pip install --no-cache-dir \
-r requirements.txt \
# The AWS SDK for Python already comes installed on Lambda Functions,
# however we need to install it separately for AWS Fargate and ECS.
boto3 \
--extra-index-url=${PYPI_URL}/simple && \
# Note that `pip` won't work once we uninstall `distutils`, so we can
# go ahead and remove it now.
pip uninstall -y pip && \
# Once all necessary Python packages are installed, we can go ahead and
# remove the `distutils` package and the bundled Python 3.8 version.
apt-get purge -y --auto-remove curl python3.8 python3.9-distutils && \
# These lines should help to further reduce the overall image size
apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false && \
rm -rf /var/lib/apt/lists/*
# Copy Python files
# TODO: Replace MY_APP with the path to your Python package
COPY MY_APP /app/MY_APP
# This is our entrypoint to the Docker image. So if we need to run a script on
# the container, we can call it like `docker run <tag> MY_APP.<name of module>`
ENTRYPOINT ["python", "-m"]
# Slight modification of the above Dockerfile when using a custom certificate file
#
# Please note: if you are building the image on an M1 Mac, you'll need to
# pass `--platform=linux/amd64` to `docker build`, as mentioned here:
# https://stackoverflow.com/a/70614305/10237506
#
# Static Assets
FROM ubuntu:20.04
ARG PYPI_URL
ARG PYPI_USER
ARG PYPI_PASS
# Ref: https://dev.to/christo22694524/installing-private-python-packages-in-docker-images-1hgm
RUN echo "machine $(echo "$PYPI_URL" | awk -F[/:] '{print $4}') \
" login ${PYPI_USER} \
" password ${PYPI_PASS}" > /root/.netrc && \
chown root ~/.netrc && \
chmod 0600 ~/.netrc
WORKDIR /app
COPY requirements.txt /app/
# TODO replace name of the cert
COPY MyCert.crt /usr/local/share/ca-certificates/
# Install Python dependencies
RUN apt-get update && \
# Installing `python3.9-distutils` appears to install Python 3.8 as well,
# so we'll need to remove Python 3.8 in the same RUN command to reduce the
# total image size.
apt-get install -y python3.9 python3.9-distutils curl ca-certificates && \
update-ca-certificates && \
ln -s /usr/bin/python3.9 /usr/local/bin/python && \
# Now we'll need to install `pip` (note that this requires `distutils`)
curl -k https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python get-pip.py \
--trusted-host pypi.python.org \
--trusted-host pypi.org \
--trusted-host files.pythonhosted.org && \
rm get-pip.py && \
pip config set global.trusted-host \
"pypi.org files.pythonhosted.org pypi.python.org" && \
# The problem here is that the `pip install`s seem to still require
# Python 3.8, so we'll need to have them in the same RUN command.
pip install -U --no-cache-dir \
pip && \
pip install --no-cache-dir \
-r requirements.txt \
# The AWS SDK for Python already comes installed on Lambda Functions,
# however we need to install it separately for AWS Fargate.
boto3 \
--extra-index-url=${PYPI_URL}/simple && \
# Note that `pip` won't work once we uninstall `distutils`, so we can
# go ahead and remove it now.
pip uninstall -y pip && \
# Once all necessary Python packages are installed, we can go ahead and
# remove the `distutils` package and the bundled Python 3.8 version.
apt-get purge -y --auto-remove curl python3.8 python3.9-distutils && \
# These lines should help to further reduce the overall image size
apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false && \
rm -rf /var/lib/apt/lists/*
# Copy Python files
# TODO: Replace MY_APP with the path to your Python package
COPY MY_APP /app/MY_APP
# This is our entrypoint to the Docker image. So if we need to run a script on
# the container, we can call it like `docker run <tag> MY_APP.<name of module>`
ENTRYPOINT ["python", "-m"]
# Dockerfile for an AWS Lambda Layer
FROM lambci/lambda:build-python3.8
# Keep this as the first line, so that we can quit early if the contents
# of `requirements.txt` hasn't changed since our last run.
COPY requirements.txt .
ARG PYPI_URL
ARG PYPI_USER
ARG PYPI_PASS
ARG ZIP_NAME
RUN echo "machine $(echo "$PYPI_URL" | awk -F[/:] '{print $4}') \
" login ${PYPI_USER} \
" password ${PYPI_PASS}" > /root/.netrc && \
chown root ~/.netrc && \
chmod 0600 ~/.netrc
RUN pip config set global.trusted-host \
"pypi.org files.pythonhosted.org pypi.python.org" && \
pip install -U --no-cache-dir pip && \
pip install --no-cache-dir \
-r requirements.txt \
--target python \
--extra-index-url=${PYPI_URL}/simple && \
find python \( -type d -name "*.dist-info" -or -name "__pycache__" \) -exec rm -rf {} + && \
zip -r ${ZIP_NAME} .
# Need a default entry point, I think anything will do.
CMD ["/bin/bash"]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment