MichaelSimons/Keeping_runtime-deps_up-to-date.MD

## Keeping_runtime-deps_up-to-date.MD

      
    Raw
  

              Keeping_runtime-deps_up-to-date.MD
            
          
    Problem Statement

The runtime-deps dockerfile currently has a static list of what the runtime dependencies are.  There is no link (e.g. tooling that automatically updates the dependencies) to the actual dependencies the runtime has.  Because of this, there is a potential for the Docker image to get out of sync with the runtime.  The Docker repo CI includes tests which build a basic self-contained hello world application and runs it in the runtime-deps image but this is not extensive enough to fully validate the right set of dependencies are included in the image.
FROM debian:stretch

RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        ca-certificates \
        \
# .NET Core dependencies
        libc6 \
        libcurl3 \
        libgcc1 \
        libgssapi-krb5-2 \
        libicu57 \
        liblttng-ust0 \
        libssl1.0.2 \
        libstdc++6 \
        libunwind8 \
        libuuid1 \
        zlib1g \
    && rm -rf /var/lib/apt/lists/*

Possible Solutions

Utilize core-setup native installers

Update the runtime-deps Dockerfiles to install the dependencies via the .NET Core native installers.  Core-Setup already produces a set of runtime-deps native installers (e.g. Debian 9 Runtime-Deps.deb) that could be used.
Note: runtime-deps.deb is a new installer that was added in 2.1.
Pros


Using the mechanism we recommend customers use to acquire the dependencies.
Docker repo isn't in the business of maintaining the dependency list rather this is done in Core-Setup which is closer to the actual product.

Cons


Core-Setup isn't producing native installers for all platforms supported in microsoft/dotnet.  Specifically Alpine and Debian 9 arm32.
Mitigation: Get product team to produce native installers for all supported distros.


Ideally Dockerfiles would obtain native installers from the public package feeds as shown below.
FROM debian:stretch

# Install .NET Core dependencies
ENV DOTNET_RUNTIME_DEPS_VERSION 2.1.0
RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        apt-transport-https  \
        ca-certificates \
        curl \
        gettext \
        gnupg2 \
        libunwind8 \
    && curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg \
    && mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg \
    && sh -c 'echo "deb [arch=amd64] https://apt-mo.trafficmanager.net/repos/microsoft-debian-stretch-prod stretch main" > /etc/apt/sources.list.d/dotnetdev.list' \
    && apt-get update \
    && apt-get install -y --no-install-recommends dotnet-runtime-deps-$DOTNET_RUNTIME_DEPS_VERSION \
    && rm -rf /var/lib/apt/lists/*

This presents the following challenges.


Releases can't be tested until public feed availability.  Testing would require hand crafted Dockerfiles which prevents testing the actual Dockerfiles that will be released.


Package feed delays - From the CLI readme - 'Newest SDK binaries for 2.0.0 in debian feed may be delayed due to external issues by up to 24h.'  On the other hand the release team hasn't noticed issues like this when publishing releases.  This would delay the release of the Docker assets on release day.  Additionaly updating the dependencies in the nightly repo would be difficult because there isn't a signal mechanism for when the bits are available in the feed.


Daily builds are no longer pushed to feeds.  Native installers are only available via the build drop location.  This means the nightly repo Dockerfiles wouldn't align with the official repo which defeats the value in having the nightly repo be a staging area for official.


Mitigation: Download native installers from blob storage instead of public feeds.  This would mitigate the technical problem, but wouldn't be showcasing how customers should be installing the runtime dependencies.
FROM debian:stretch

RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        ca-certificates \
        curl \
    && rm -rf /var/lib/apt/lists/*

# .NET Core dependencies
ENV DOTNET_RUNTIME_DEPS_VERSION 2.1.0-preview1-26112-04

RUN curl -SL --output runtime-deps.deb https://dotnetcli.blob.core.windows.net/dotnet/Runtime/$DOTNET_RUNTIME_DEPS_VERSION/dotnet-runtime-deps-$DOTNET_RUNTIME_DEPS_VERSION-debian.9-x64.deb \
    && runtime_deps_sha512='08ED052D8CFABE9B3E755139A19BC1B1C02C4B1C3B4AAF573DB7BE73D7FEC2E7FF4EEFD09684F8BADFA2407DA536868E93AFB2C02088629BE9B21C865632FEAC' \
    && echo "$runtime_deps_sha512 runtime-deps.deb" | sha512sum -c - \
    && dpkg -i --force-depends runtime-deps.deb \
    && rm runtime-deps.deb \
    && apt-get update \
    && apt-get install -y --no-install-recommends -f \
    && rm -rf /var/lib/apt/lists/*


Add automation to update runtime-deps as product changes

Get the runtime to declare its dependencies.  Once this is in place, automation (via Maestro) could be implemented to update the current runtime-deps Dockerfile whenever a new dependency is introduced.
Pros


Core-Setup (e.g native installers) would also benefit from having this metadata.  They are currently facing the same issue described here.

Cons


The Runtime-Deps Dockerfiles still wouldn't be showcasing the recommended pattern for installing the dependencies.


The dependency metadata doesn't exist today.
Mitigation: Get product team to define metadata.


Enhanced tests

Leave the runtime-deps as is and add better test coverage to help catch when there is a dependency mismatch.  For example a test could be written that utilizes the Core-Setup metadata that is used to produce the native installers.  This metadata can be retrieved to verify the same components are installed within the Runtime-Deps Docker image.
Pros


Actionable today.

Cons


The Runtime-Deps Dockerfiles still wouldn't be showcasing the recommended pattern for installing the dependencies.
An exhaustive set of tests that ensures all scenarios are covered isn't feasible.