Skip to content

Instantly share code, notes, and snippets.

@rcanavan rcanavan/Dockerfile
Created Feb 9, 2018

Embed
What would you like to do?
Dockerfile to demonstrate mod_brotli heap fragmentation issues on CentOS
FROM centos:7
# ensure www-data user exists
RUN set -x \
&& groupadd www-data \
&& useradd -u 82 -g www-data www-data
# 82 is the standard uid/gid for "www-data" in Alpine
# http://git.alpinelinux.org/cgit/aports/tree/main/apache2/apache2.pre-install?h=v3.3.2
# http://git.alpinelinux.org/cgit/aports/tree/main/lighttpd/lighttpd.pre-install?h=v3.3.2
# http://git.alpinelinux.org/cgit/aports/tree/main/nginx-initscripts/nginx-initscripts.pre-install?h=v3.3.2
ENV HTTPD_PREFIX /usr/local/apache2
ENV BROTLI_PREFIX /usr/local
ENV PATH $HTTPD_PREFIX/bin:$PATH
RUN mkdir -p "$HTTPD_PREFIX" \
&& chown www-data:www-data "$HTTPD_PREFIX"
WORKDIR $HTTPD_PREFIX
ENV HTTPD_VERSION 2.4.29
ENV HTTPD_SHA256 777753a5a25568a2a27428b2214980564bc1c38c1abf9ccc7630b639991f7f00
# https://httpd.apache.org/security/vulnerabilities_24.html
ENV HTTPD_PATCHES=""
ENV APACHE_DIST_URLS \
# https://issues.apache.org/jira/browse/INFRA-8753?focusedCommentId=14735394#comment-14735394
https://www.apache.org/dyn/closer.cgi?action=download&filename= \
# if the version is outdated (or we're grabbing the .asc file), we might have to pull from the dist/archive :/
https://www-us.apache.org/dist/ \
https://www.apache.org/dist/ \
https://archive.apache.org/dist/
ENV foo bar
RUN set -eux; \
\
# build deps
yum -y install \
epel-release \
net-tools \
bash-completion \
libtool-ltdl \
bzip2 \
zip \
unzip \
nasm \
cmake \
gcc \
gcc-c++ \
httpd-devel \
patch \
make \
autoconf \
automake \
libtool \
libtool-ltdl-devel \
pcre-devel \
curl \
libcurl-devel \
binutils-devel \
zlib-devel \
git \
wget \
which
RUN ddist() { \
local f="$1"; shift; \
local distFile="$1"; shift; \
local success=; \
local distUrl=; \
for distUrl in $APACHE_DIST_URLS; do \
if wget -O "$f" "$distUrl$distFile"; then \
success=1; \
break; \
fi; \
done; \
[ -n "$success" ]; \
}; \
\
ddist 'httpd.tar.bz2' "httpd/httpd-$HTTPD_VERSION.tar.bz2"; \
echo "$HTTPD_SHA256 *httpd.tar.bz2" | sha256sum -c -; \
\
# see https://httpd.apache.org/download.cgi#verify
ddist 'httpd.tar.bz2.asc' "httpd/httpd-$HTTPD_VERSION.tar.bz2.asc"; \
export GNUPGHOME="$(mktemp -d)"; \
gpg --keyserver ha.pool.sks-keyservers.net --recv-keys A93D62ECC3C8EA12DB220EC934EA76E6791485A8; \
gpg --batch --verify httpd.tar.bz2.asc httpd.tar.bz2; \
rm -rf "$GNUPGHOME" httpd.tar.bz2.asc; \
true
RUN set -eux; \
mkdir -p src; \
wget -O brotli.tar.gz https://github.com/google/brotli/archive/v1.0.2.tar.gz ; \
tar -xf brotli.tar.gz -C src ; \
rm brotli.tar.gz ; \
tar -xf httpd.tar.bz2 -C src --strip-components=1; \
rm httpd.tar.bz2;
RUN set -eux; \
cd src/brotli-1.0.2; \
sed -i 's;libbrotlidec_la_LDFLAGS = .*;& -lbrotlicommon;' Makefile.am; \
sed -i 's;libbrotlienc_la_LDFLAGS = .*;& -lbrotlicommon;' Makefile.am; \
sed -i 's;bin/sh -e;bin/bash -xe;' bootstrap; \
/bin/bash -xe ./bootstrap; \
./configure \
--prefix="$BROTLI_PREFIX" \
; \
make; \
make install; \
cd ../..; \
cd src; \
patches() { \
while [ "$#" -gt 0 ]; do \
local patchFile="$1"; shift; \
local patchSha256="$1"; shift; \
ddist "$patchFile" "httpd/patches/apply_to_$HTTPD_VERSION/$patchFile"; \
echo "$patchSha256 *$patchFile" | sha256sum -c -; \
patch -p0 < "$patchFile"; \
rm -f "$patchFile"; \
done; \
}; \
patches $HTTPD_PATCHES; \
\
for f in ./modules/filters/config.m4 ./configure; do \
sed -i 's/-lbrotlienc -lbrotlicommon/-lbrotlienc -lbrotlicommon -lm/' $f; \
done; \
./configure \
--prefix="$HTTPD_PREFIX" \
--enable-mods-shared=reallyall \
--enable-mpms-shared=all \
--with-brotli=/usr/local \
; \
make -j "$(nproc)"; \
make install; \
\
cd ..; \
sed -ri \
-e 's!^(\s*CustomLog)\s+\S+!\1 /proc/self/fd/1!g' \
-e 's!^(\s*ErrorLog)\s+\S+!\1 /proc/self/fd/2!g' \
"$HTTPD_PREFIX/conf/httpd.conf"; \
rm -r src man manual;
# configure mod_brotli
RUN \
echo LoadModule brotli_module modules/mod_brotli.so >> /usr/local/apache2/conf/httpd.conf && \
echo BrotliCompressionQuality 7 >> /usr/local/apache2/conf/httpd.conf && \
echo AddOutputFilterByType BROTLI_COMPRESS text/html >> /usr/local/apache2/conf/httpd.conf
# create a compressible test file
RUN find / >> /usr/local/apache2/htdocs/index.html
EXPOSE 80
CMD ["httpd", "-DFOREGROUND"]
#
# test it with:
# $ docker build -t brotlikeaksdemo .
# $ docker run --name bloattest --rm -d brotlikeaksdemo
# $ docker exec -ti bloattest ab -n 1000 -c 2 -H 'Accept-Encoding: br' http://localhost/
#
# and see the resulting httpd PIDs and their size with:
# $ docker exec -ti bloattest ps -C httpd -o rss,pid
#
# fix the leak with:
# $ docker run --name bloattest -e MALLOC_TRIM_THRESHOLD_=16384 -e MALLOC_ARENA_MAX=8 --rm -d brotlikeaksdemo
#
@filex

This comment has been minimized.

Copy link

commented Feb 9, 2018

The story behind this file: https://blog.wao.io/the-brotlileaks-affair/

We explain how libbrotli causes heap fragmentation in Apaches mod_brotli – and how to fix it with glibc tunables.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.