Skip to content

Instantly share code, notes, and snippets.

@bitsofinfo
Last active June 3, 2021 16:27
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save bitsofinfo/dca033c0552a7aa0a957abd3cbc2b29c to your computer and use it in GitHub Desktop.
Save bitsofinfo/dca033c0552a7aa0a957abd3cbc2b29c to your computer and use it in GitHub Desktop.
example Dockerfile for installing pip python modules from private Azure Artifacts feed; more info: https://bitsofinfo.wordpress.com/2021/06/03/private-python-azure-artifacts-feeds-alpine-docker/
#-----------------------------------------
# Build layer: here we setup the base
# python environment, install core modules
# including the pre-requisites for talking
# to a private Azure Artifacts repo
#
# https://docs.microsoft.com/en-us/azure/devops/artifacts/quickstarts/python-packages
#
# https://bitsofinfo.wordpress.com/2021/06/03/private-python-azure-artifacts-feeds-alpine-docker/
#
#-----------------------------------------
#
FROM python:3.9.5-alpine3.13 as build
# the system dependencies required for any of our
# app specific python modules we will be installing etc
#
RUN apk add <WHATEVER (IF ANY) SYSTEM LEVEL DEPENDENCIES YOUR APP NEEDS>
#
# Azure Artifacts artifacts-keyring/dotnet based requirements
# https://docs.microsoft.com/en-us/azure/devops/artifacts/quickstarts/python-packages?view=azure-devops#use-artifacts-keyring-to-set-up-authentication
#
#
# without this the dotnet-installer doesn't work
#
RUN apk update && \
apk add ca-certificates wget && \
update-ca-certificates
# ok lets get all our pre-reqs for dotnet....
# https://docs.microsoft.com/en-us/dotnet/core/install/linux-alpine
#
RUN apk add bash icu-libs krb5-libs libgcc libintl libssl1.1 libstdc++ zlib
RUN apk add libgdiplus --repository https://dl-3.alpinelinux.org/alpine/edge/testing/
# next lets install dotnet https://github.com/Microsoft/artifacts-keyring
# https://docs.microsoft.com/en-us/dotnet/core/install/linux-alpine
# https://docs.microsoft.com/en-us/dotnet/core/install/linux-scripted-manual#scripted-install
#
RUN wget -O /dotnet-install.sh https://dot.net/v1/dotnet-install.sh
RUN chmod +x /dotnet-install.sh && /dotnet-install.sh -c Current
# https://stackoverflow.com/questions/66118337/how-to-get-rid-of-cryptography-build-error
#
# When the artifacts-keyring module gets installed, one of its dependencies is
# the cryptography module which... won't build without these additional items.
# supposedly alpine 3.12+ handles this, but the particular python:3x-alpine base
# image may not already have it
#
RUN apk add libffi-dev rust cargo libressl-dev
#
# Create a virtual env for our python environment
# where all modules will go that will be copied
# out of this base layer in the build below
#
RUN python3 -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
RUN mkdir /opt/build
# requirements-dev.txt delcares artifacts-keyring deps
# and includes any runtime modules.
#
# It's contents looks like this:
#
# --------
# wheel
# twine
# artifacts-keyring --pre
# -r requirements.txt
#
# where requirements.txt contains whatever actual app
# runtime modules you'd normally declare (i.e. including
# the module(s) that are in your private azure artifacts feed)
#
COPY requirements-dev.txt /opt/build
COPY requirements.txt /opt/build
# here we run the pip install, who's requirements has one or more
# modules that reside in a private Azure Artifacts repo, the
# docker build --secret "pip.url.secret" contains a simple URI
# in the format of:
#
# https://<feedname>:<pattoken>@pkgs.dev.azure.com/<org>/<project>/_packaging/<feed>/pypi/simple/
#
# The PAT token is sensitive! When you create your PAT token in azure
# make sure it has Packaging > READ access:
# https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops#create-a-pat
#
# All the .NET dependencies above are being utilized here for the
# artifacts-keyring based auth flow that occurs when the private
# Azure Artifacts feed is accessed.
#
# https://docs.docker.com/develop/develop-images/build_enhancements/#new-docker-build-secret-information
#
# i.e. (where pip.url.secret is a file next your Dockerfile that is secured and not in git!)
# docker build . -t <yourapp>:<yourtag> --secret id=pip.url.secret,src=pip.url.secret
#
# Note we also cleanup the build related modules and their deps post install, then
# just do a 2nd pip install on the requirements.txt just to ensure none were auto
# smoked by the auto-remover uninitentionally
#
RUN --mount=type=secret,id=pip.url.secret,dst=/opt/build/pip.url.secret \
PRIVATE_PIP_AZURE_ARTIFACTS_URL=$(cat /opt/build/pip.url.secret) && \
cd /opt/build && \
source /opt/venv/bin/activate && \
pip install -r requirements-dev.txt --extra-index-url=${PRIVATE_PIP_AZURE_ARTIFACTS_URL} && \
pip install pip-autoremove && \
pip-autoremove wheel twine artifacts-keyring -y && \
pip uninstall pip-autoremove -y && \
pip install -r requirements.txt
# ----------------
# END build layer
# ----------------
# --------------------------------
# RELEASE LAYER
#
# Here is where you actually pull
# the virtual python environment
# out of the build layer + any
# system level libraries your modules
# depend on, install your python app
# and anything else you need
#
# --------------------------------
FROM python:3.9.5-alpine3.13 as release
COPY --from=build /opt/venv /opt/venv
# copy any system libs your modules depend on...
# this is just an EXAMPLE below!!!! CUSTOMIZE
# FOR YOUR USE CASE!
COPY --from=build /usr/lib/lib*xslt* /usr/lib/
COPY --from=build /usr/lib/libxml* /usr/lib/
COPY --from=build /usr/lib/libgcrypt* /usr/lib/
COPY --from=build /usr/lib/libgpg* /usr/lib/
# install your app etc
COPY myapp.py /opt/scripts/myapp.py
ENV PATH="/opt/scripts:/opt/venv/bin:$PATH"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment