Skip to content

Instantly share code, notes, and snippets.

@concatime
Last active April 19, 2018 21:41
Show Gist options
  • Save concatime/36d58b967c7ef7f66e7acee91aea973c to your computer and use it in GitHub Desktop.
Save concatime/36d58b967c7ef7f66e7acee91aea973c to your computer and use it in GitHub Desktop.
ngx_wizard

NGiИX WIZARD

Features of ngx_wizard:

  • Bleading Edge ;
  • Self Contained Package ;
  • Compiled from scratch ;
  • Based on shared libraries ;
  • LibreSSL as SSL library ;
  • PCRE Jit as RegEx library ;
  • Brotli compression dynamic module ;
  • Atomic & Jemalloc for internal purpose ;

Simple wizard for compiling nginx from scratch, independently from distros.

This script is provided AS IS & comes with ABSOLUTELY NO WARRANTY.

I may need help to tweak with-cc-opt & with-ld-opt.

Instructions

You need to install git, curl, make and gcc (or clang). Then, run this command:

bash <(curl https://gist.githubusercontent.com/concatime/36d58b967c7ef7f66e7acee91aea973c/raw/.sh)

And…, that's it!

Note: You could specify which c compiler will be use by typing CC=gcc-X or CC=clang, followed by the command.

About

Why I used shared version of nginx?

  • Inside builded Makefile are really crap. See this as a concrete example.
  • Following the same vibe, nginx makefile requires a c++ compiler for PCRE. I tried --with-pcre-opt=--disable-cpp, in vain.
  • Upgrading parts seamlessly, without recompiling everything.

Enhancements

Bugs

  • Actually, there is a bug between systemd and nginx which requires this hack. Source.

Warnings

Those are warnings I get usnig gcc-7.1.1:

atomic_ops/sysdeps/standard_ao_double_t.h:35:37: warning: ‘-pedantic’ is not an option that controls warnings [-Wpragmas]
 #     pragma GCC diagnostic ignored "-pedantic"
                                     ^~~~~~~~~~~
  CC       asn1/libcrypto_la-a_bitstr.lo
asn1/a_bitstr.c: In function 'i2c_ASN1_BIT_STRING':
asn1/a_bitstr.c:121:2: warning: 'memcpy': specified size between 18446744071562067968 and 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Wstringop-overflow=]
  memcpy(p, d, len);
  ^~~~~~~~~~~~~~~~~
[…]
Making install in ssl
[…]
  CC       d1_both.lo
d1_both.c: In function 'dtls1_retransmit_message':
d1_both.c:1147:3: warning: 'save_write_sequence' may be used uninitialized in this function [-Wmaybe-uninitialized]
   memcpy(S3I(s)->write_sequence, save_write_sequence,
   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
       sizeof(S3I(s)->write_sequence));
       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[…]
  CC       s3_lib.lo
s3_lib.c: In function 'ssl3_handshake_msg_finish':
s3_lib.c:1516:17: warning: variable 'd' set but not used [-Wunused-but-set-variable]
  unsigned char *d, *p;
../ngx_brotli/src/ngx_http_brotli_filter_module.c:272:9: warning: ‘BrotliEncoderInputBlockSize’ is deprecated [-Wdeprecated-declarations]
         ctx->brotli_ring = BrotliEncoderInputBlockSize(ctx->encoder);
         ^~~
In file included from ../ngx_brotli/src/ngx_http_brotli_filter_module.c:16:0:
../ngx_brotli/deps/brotli/include/brotli/encode.h:220:41: note: declared here
 BROTLI_DEPRECATED BROTLI_ENC_API size_t BrotliEncoderInputBlockSize(
                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~
../ngx_brotli/src/ngx_http_brotli_filter_module.c: In function ‘ngx_http_brotli_filter_create_encoder’:
../ngx_brotli/src/ngx_http_brotli_filter_module.c:427:16: warning: ‘kBrotliMinWindowBits’ is deprecated [-Wdeprecated-declarations]
                && wbits > kBrotliMinWindowBits)
                ^~
In file included from ../ngx_brotli/src/ngx_http_brotli_filter_module.c:16:0:
../ngx_brotli/deps/brotli/include/brotli/encode.h:39:36: note: declared here
 BROTLI_DEPRECATED static const int kBrotliMinWindowBits =
                                    ^~~~~~~~~~~~~~~~~~~~
../ngx_brotli/src/ngx_http_brotli_filter_module.c: In function ‘ngx_http_brotli_filter_add_data’:
../ngx_brotli/src/ngx_http_brotli_filter_module.c:498:5: warning: ‘BrotliEncoderCopyInputToRingBuffer’ is deprecated [-Wdeprecated-declarations]
     BrotliEncoderCopyInputToRingBuffer(ctx->encoder, size, b->pos);
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../ngx_brotli/src/ngx_http_brotli_filter_module.c:16:0:
../ngx_brotli/deps/brotli/include/brotli/encode.h:224:39: note: declared here
 BROTLI_DEPRECATED BROTLI_ENC_API void BrotliEncoderCopyInputToRingBuffer(
                                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../ngx_brotli/src/ngx_http_brotli_filter_module.c: In function ‘ngx_http_brotli_filter_process’:
../ngx_brotli/src/ngx_http_brotli_filter_module.c:534:5: warning: ‘BrotliEncoderWriteData’ is deprecated [-Wdeprecated-declarations]
     if (!BrotliEncoderWriteData(ctx->encoder, ctx->last, ctx->flush, &size,
     ^~
In file included from ../ngx_brotli/src/ngx_http_brotli_filter_module.c:16:0:
../ngx_brotli/deps/brotli/include/brotli/encode.h:229:46: note: declared here
 BROTLI_DEPRECATED BROTLI_ENC_API BROTLI_BOOL BrotliEncoderWriteData(
                                              ^~~~~~~~~~~~~~~~~~~~~~
../ngx_brotli/src/ngx_http_brotli_filter_module.c: In function ‘ngx_http_brotli_window’:
../ngx_brotli/src/ngx_http_brotli_filter_module.c:903:5: warning: ‘kBrotliMaxWindowBits’ is deprecated [-Wdeprecated-declarations]
     wbits = kBrotliMaxWindowBits;
     ^~~~~
In file included from ../ngx_brotli/src/ngx_http_brotli_filter_module.c:16:0:
../ngx_brotli/deps/brotli/include/brotli/encode.h:41:36: note: declared here
 BROTLI_DEPRECATED static const int kBrotliMaxWindowBits =
                                    ^~~~~~~~~~~~~~~~~~~~
../ngx_brotli/src/ngx_http_brotli_filter_module.c:904:5: warning: ‘kBrotliMinWindowBits’ is deprecated [-Wdeprecated-declarations]
     mbits = kBrotliMinWindowBits;
     ^~~~~
In file included from ../ngx_brotli/src/ngx_http_brotli_filter_module.c:16:0:
../ngx_brotli/deps/brotli/include/brotli/encode.h:39:36: note: declared here
 BROTLI_DEPRECATED static const int kBrotliMinWindowBits =
                                    ^~~~~~~~~~~~~~~~~~~~

Credits

NGINX=1.13.3
LIBRESSL=2.6.0
LIBATOMIC=7.6.0
JEMALLOC=5.0.1
PCRE=8.41
# Too lazy ;)
sudo echo 'You should have git, curl, make and a c compiler installed BEFORE running this script!'
read -p 'Press ENTER to continue, otherwise hit CNTRL’C to abort'
set -e
pushd /opt
sudo mkdir local
cd $_
sudo mkdir include lib
cd $_
sudo mkdir pkgconfig
for k in pcre atomic_ops atomic_ops_gpl
do sudo ln -sf lib$k.so.1 lib$k.so
done
sudo ln -sf libpcreposix.so.0 libpcreposix.so
echo `pwd` | sudo dd status=none of=/etc/ld.so.conf.d/opt.local.lib.conf
popd
pushd $(mktemp --directory)
for k in nginx.org/download/nginx-$NGINX ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-$LIBRESSL github.com/ivmai/libatomic_ops/releases/download/v$LIBATOMIC/libatomic_ops-$LIBATOMIC
do curl --location --silent https://$k.tar.gz | tar xz &
done
for k in github.com/jemalloc/jemalloc/releases/download/$JEMALLOC/jemalloc-$JEMALLOC ftp.pcre.org/pub/pcre/pcre-$PCRE
do curl --location --silent https://$k.tar.bz2 | tar xj &
done
git clone https://github.com/google/ngx_brotli.git --recursive
# Background processes
wait
pushd jemalloc-$JEMALLOC
./configure --disable-cxx --disable-stats --libdir=/opt/local/lib --includedir=/opt/local/include
sudo make install_lib_shared install_include
sudo install -c jemalloc.pc /opt/local/lib/pkgconfig
popd
pushd pcre-$PCRE
./configure --disable-cpp --disable-static --enable-jit --enable-unicode-properties
make
sudo install -c pcre.h pcreposix.h /opt/local/include
sudo install -c libpcre.pc libpcreposix.pc /opt/local/lib/pkgconfig
cd .libs
sudo install -c libpcreposix.so.0.0.5 /opt/local/lib
sudo install -c libpcre.so.1.2.9 /opt/local/lib
sudo cp --no-dereference --target-directory /opt/local/lib libpcre.so.1 libpcreposix.so.0
popd
pushd libatomic_ops-$LIBATOMIC
./configure --disable-docs --disable-static --enable-shared --libdir=/opt/local/lib --includedir=/opt/local/include
sudo make install
popd
pushd libressl-$LIBRESSL
./configure --disable-static --prefix=`pwd`/die --libdir=/opt/local/lib --includedir=/opt/local/include
sudo make install
popd
sudo ldconfig
pushd nginx-$NGINX
./configure \
--prefix=/opt/nginx \
--pid-path=/run/nginx.pid \
--group=nginx \
--with-compat \
--with-stream \
--with-threads \
--with-file-aio \
--with-libatomic \
--with-http_v2_module \
--with-http_ssl_module \
--with-stream_ssl_module \
--without-http_gzip_module \
--without-select_module \
--without-poll_module \
--without-mail_pop3_module \
--without-mail_imap_module \
--without-mail_smtp_module \
--add-dynamic-module=../ngx_brotli \
--with-ld-opt='-L/opt/local/lib -ljemalloc -lrt -Wl,-z,relro,-z,now' \
--with-cc-opt='-I/opt/local/include -pipe -pthread -g -O2 -march=native -fuse-ld=gold -fstack-protector-strong -Wno-error -Wp,-D_FORTIFY_SOURCE=2'
sudo make install -j`nproc`
popd
rm `pwd` --recursive --force
popd
pushd /etc/systemd/system
sudo curl https://gist.githubusercontent.com/concatime/36d58b967c7ef7f66e7acee91aea973c/raw/nginx.service --remote-name
popd
sudo groupadd --force --system nginx
sudo systemctl enable nginx
echo '# systemctl (re)start/stop/reload nginx'
echo …done
[Unit]
Description=NGiИX ~ High performance web server
Documentation=https://nginx.org/en/docs
After=network-online.target remote-fs.target nss-lookup.target
Wants=network-online.target
[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/opt/nginx/sbin/nginx -t
ExecStartPost=/bin/sleep .01
ExecStart=/opt/nginx/sbin/nginx
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
[Install]
WantedBy=multi-user.target
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment