Skip to content

Instantly share code, notes, and snippets.

@AsgerPetersen

AsgerPetersen/Dockerfile Secret

Last active May 22, 2019 09:17
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save AsgerPetersen/3b98b02461d745813dda92a10185b22a to your computer and use it in GitHub Desktop.
Save AsgerPetersen/3b98b02461d745813dda92a10185b22a to your computer and use it in GitHub Desktop.
Download ZIP
QGIS server on AWS Lambda
Raw
readme.md

Notes on QGIS on AWS Lambda

Inspired by https://github.com/bitner/mapserverless and https://github.com/bitner/mapserverless-layer/blob/master/Dockerfile

build

docker build -t qgiserverless .

QT

https://superuser.com/questions/812195/installing-qt-on-an-aws-instance https://doc.qt.io/qt-5/linux-building.html https://doc.qt.io/qt-5/configure-options.html

Later maybe skip some unneeded modules: https://forum.qt.io/topic/76649/qt-build-options-to-skip-few-modules-through-configuration-file https://doc.qt.io/qt-5/configure-options.html#modules-and-features

QGIS

Problem with gcc 4.8 http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-Problems-compiling-QGIS-3-0-no-matching-function-for-call-to-QgsLayerTreeLayer-connect-td5356871.html

Possible solution https://www.vultr.com/docs/how-to-install-gcc-on-centos-6

/build/qgis/src/core/layertree/qgslayertreelayer.cpp:72:94: error: no matching function for call to 'QgsLayerTreeLayer::connect( maybe caused by https://stackoverflow.com/questions/34040791/no-matching-function-for-call-to-connect-qt-5-5 Possible solution http://osgeo-org.1560.x6.nabble.com/QGIS-Developer-Problems-compiling-QGIS-3-0-no-matching-function-for-call-to-QgsLayerTreeLayer-connect-td5356871.html

https://jdhao.github.io/2017/09/04/install-gcc-newer-version-on-centos/

Building qca with the custom build gcc yields an error like this https://stackoverflow.com/questions/29230777/program-linking-fails-when-using-custom-built-gcc

In file included from /build/qgis/src/server/qgsserverplugins.cpp:22:0: /build/qgis/src/python/qgspythonutils.h:22:10: fatal error: qgis_python.h: No such file or directory

Enabling bindings raises this error

Scanning dependencies of target zzz-db_manager-15-depend
make[2]: *** No rule to make target `python/plugins/db_manager/pygui', needed by `python/plugins/db_manager/ui_DlgAddGeometryColumn.py'.  Stop.
make[1]: *** [python/plugins/db_manager/CMakeFiles/zzz-db_manager-15-depend.dir/all] Error 2

Trying -DWITH_STAGED_PLUGINS=OFF

Now getting

CMake Error at python/plugins/db_manager/cmake_install.cmake:41 (file):
  file INSTALL cannot find
  "/build/qgis/python/plugins/db_manager/resources_rc.py".
Call Stack (most recent call first):
  python/plugins/cmake_install.cmake:42 (include)
  python/cmake_install.cmake:144 (include)
  cmake_install.cmake:63 (include)
Raw
Dockerfile
FROM lambci/lambda:build-provided
# install system libraries
RUN \
yum makecache fast; \
yum install -y wget libpng-devel nasm libzip-devel libxml2-devel; \
yum install -y bash-completion --enablerepo=epel; \
yum clean all; \
yum autoremove
# versions of packages
ENV \
CURL_VERSION=7.59.0 \
GEOS_VERSION=3.7.1 \
GEOTIFF_VERSION=1.4.3 \
GDAL_VERSION=2.4.1 \
HDF4_VERSION=4.2.14 \
HDF5_VERSION=1.10.5 \
NETCDF_VERSION=4.6.2 \
NGHTTP2_VERSION=1.35.1 \
OPENJPEG_VERSION=2.3.0 \
LIBJPEG_TURBO_VERSION=2.0.1 \
PKGCONFIG_VERSION=0.29.2 \
PROJ_VERSION=5.2.0 \
SZIP_VERSION=2.1.1 \
WEBP_VERSION=1.0.1 \
ZSTD_VERSION=1.3.8 \
QGIS_VERSION=3_6_2;
# Paths to things
ENV \
BUILD=/build \
NPROC=4 \
PREFIX=/usr/local \
GDAL_CONFIG=/usr/local/bin/gdal-config \
LD_LIBRARY_PATH=/usr/local/lib:/usr/local/lib64:/usr/local/lib/postgresql \
PATH=/usr/local/bin:$PATH \
GDAL_DATA=/usr/local/share/gdal
# switch to a build directory
WORKDIR /build
# gcc
# https://www.vultr.com/docs/how-to-install-gcc-on-centos-6
# https://gcc.gnu.org/wiki/InstallingGCC
RUN \
mkdir gcc && mkdir gcc_build && \
wget -qO- ftp://ftp.fu-berlin.de/unix/languages/gcc/releases/gcc-7.4.0/gcc-7.4.0.tar.gz \
| tar xvz -C gcc --strip-components=1 && cd gcc
RUN \
cd ${BUILD}/gcc && ./contrib/download_prerequisites && \
cd ${BUILD}/gcc_build && \
../gcc/configure --disable-multilib --enable-languages=c,c++ && \
make -j ${NPROC} && sudo make install
RUN cd ${BUILD} && rm -rf gcc && rm -rf gcc_build;
# cmake seems to not always find the right compiler.
ENV \
CC=/usr/local/bin/gcc \
CXX=/usr/local/bin/c++
# Cmake
# https://stackoverflow.com/questions/47508591/running-cmake-on-amazon-linux/47616012#47616012
RUN \
yum erase cmake; \
mkdir cmake && \
wget -qO- https://github.com/Kitware/CMake/releases/download/v3.14.3/cmake-3.14.3.tar.gz \
| tar xvz -C cmake --strip-components=1 && cd cmake && \
./bootstrap && \
make -j ${NPROC} && \
make install && \
cd ${BUILD} && rm -rf cmake;
ENV PATH="/usr/local/bin/cmake:${PATH}"
# pkg-config - version > 2.5 required for GDAL 2.3+
RUN \
mkdir pkg-config && \
wget -qO- https://pkg-config.freedesktop.org/releases/pkg-config-$PKGCONFIG_VERSION.tar.gz \
| tar xvz -C pkg-config --strip-components=1 && cd pkg-config && \
./configure --prefix=$PREFIX CFLAGS="-O2 -Os" && \
make -j ${NPROC} install && \
cd ../ && rm -rf pkg-config
# proj
RUN \
mkdir proj && \
wget -qO- http://download.osgeo.org/proj/proj-$PROJ_VERSION.tar.gz \
| tar xvz -C proj --strip-components=1 && cd proj && \
./configure --prefix=$PREFIX && \
make -j ${NPROC} install && \
cd .. && rm -rf proj
# nghttp2
RUN \
mkdir nghttp2 && \
wget -qO- https://github.com/nghttp2/nghttp2/releases/download/v${NGHTTP2_VERSION}/nghttp2-${NGHTTP2_VERSION}.tar.gz \
| tar xvz -C nghttp2 --strip-components=1 && cd nghttp2 && \
./configure --enable-lib-only --prefix=${PREFIX} && \
make -j ${NPROC} install && \
cd .. && rm -rf nghttp2
# curl
RUN \
mkdir curl && \
wget -qO- https://curl.haxx.se/download/curl-${CURL_VERSION}.tar.gz \
| tar xvz -C curl --strip-components=1 && cd curl && \
./configure --prefix=${PREFIX} --disable-manual --disable-cookies --with-nghttp2=${PREFIX} && \
make -j ${NPROC} install && \
cd .. && rm -rf curl
# GEOS
RUN \
mkdir geos && \
wget -qO- http://download.osgeo.org/geos/geos-$GEOS_VERSION.tar.bz2 \
| tar xvj -C geos --strip-components=1 && cd geos && \
./configure --enable-python --prefix=$PREFIX CFLAGS="-O2 -Os" && \
make -j ${NPROC} install && \
cd .. && rm -rf geos
# WEBP
RUN \
mkdir webp && \
wget -qO- https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-${WEBP_VERSION}.tar.gz \
| tar xvz -C webp --strip-components=1 && cd webp && \
CFLAGS="-O2 -Wl,-S" PKG_CONFIG_PATH="/usr/lib64/pkgconfig" ./configure --prefix=$PREFIX && \
make -j ${NPROC} install && \
cd .. && rm -rf webp
# ZSTD
RUN \
mkdir zstd && \
wget -qO- https://github.com/facebook/zstd/archive/v${ZSTD_VERSION}.tar.gz \
| tar -xvz -C zstd --strip-components=1 && cd zstd && \
make -j ${NPROC} install PREFIX=$PREFIX ZSTD_LEGACY_SUPPORT=0 CFLAGS=-O1 --silent && \
cd .. && rm -rf zstd
# openjpeg
RUN \
mkdir openjpeg && \
wget -qO- https://github.com/uclouvain/openjpeg/archive/v$OPENJPEG_VERSION.tar.gz \
| tar xvz -C openjpeg --strip-components=1 && cd openjpeg && mkdir build && cd build && \
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$PREFIX && \
make -j ${NPROC} install && \
cd ../.. && rm -rf openjpeg
# jpeg_turbo
RUN \
mkdir jpeg && \
wget -qO- https://github.com/libjpeg-turbo/libjpeg-turbo/archive/${LIBJPEG_TURBO_VERSION}.tar.gz \
| tar xvz -C jpeg --strip-components=1 && cd jpeg && \
cmake -G"Unix Makefiles" -DCMAKE_INSTALL_PREFIX=$PREFIX . && \
make -j $(nproc) install && \
cd .. && rm -rf jpeg
# geotiff
RUN \
mkdir geotiff && \
wget -qO- https://download.osgeo.org/geotiff/libgeotiff/libgeotiff-$GEOTIFF_VERSION.tar.gz \
| tar xvz -C geotiff --strip-components=1 && cd geotiff && \
./configure --prefix=${PREFIX} \
--with-proj=${PREFIX} --with-jpeg=${PREFIX} --with-zip=yes && \
make -j ${NPROC} install && \
cd ${BUILD} && rm -rf geotiff
# exiv2 # Needed to build qgis/src/analysis
# RUN \
# mkdir exiv2; \
# wget -qO- https://github.com/Exiv2/exiv2/archive/0.27.1.tar.gz \
# | tar xvz -C exiv2 --strip-components=1; \
# cd exiv2; \
# mkdir build && cd build; \
# cmake .. -DCMAKE_BUILD_TYPE=Release; \
# cmake --build . \
# make install; \
# cd ${BUILD}; rm -rf exiv2
# Protobufc
RUN \
mkdir protobuf && \
wget -qO- https://github.com/google/protobuf/archive/v3.0.2.tar.gz \
| tar xvz -C protobuf --strip-components=1 && cd protobuf && \
./autogen.sh && \
./configure && \
make -j ${NPROC} install && \
cd ${BUILD} && rm -rf protobuf
# Protobufc
RUN \
mkdir protobufc && \
wget -qO- https://github.com/protobuf-c/protobuf-c/releases/download/v1.2.1/protobuf-c-1.2.1.tar.gz \
| tar xvz -C protobufc --strip-components=1 && cd protobufc && \
./configure && \
make -j ${NPROC} install && \
cd ${BUILD} && rm -rf protobufc
# #postgresql
RUN \
mkdir postgresql && \
wget -qO- https://ftp.postgresql.org/pub/source/v11.2/postgresql-11.2.tar.gz \
| tar xvz -C postgresql --strip-components=1 && cd postgresql && \
./configure --with-openssl --prefix=$PREFIX && \
make -j ${NPROC} install && \
cd ${BUILD} && rm -rf postgresql
# libspatialindex
# https://libspatialindex.org/install.html
# v 1.9 fails with cmake
RUN \
mkdir libspatialindex && \
wget -qO- https://github.com/libspatialindex/libspatialindex/archive/1.8.5.tar.gz \
| tar xvz -C libspatialindex --strip-components=1 && cd libspatialindex && \
cmake . && \
make -j ${NPROC} && \
make install && \
cd ${BUILD} && rm -rf libspatialindex
# Spatialite
# https://www.gaia-gis.it/gaia-sins/index.html
RUN \
mkdir spatialite && \
wget -qO- http://www.gaia-gis.it/gaia-sins/libspatialite-sources/libspatialite-4.3.0a.tar.gz \
| tar xvz -C spatialite --strip-components=1 && cd spatialite && \
./configure --enable-libxml2=no --enable-freexl=no && \
make -j ${NPROC} install-strip && \
cd ${BUILD} && rm -rf spatialite
# GDAL
RUN \
mkdir gdal && \
wget -qO- http://download.osgeo.org/gdal/$GDAL_VERSION/gdal-$GDAL_VERSION.tar.gz \
| tar xvz -C gdal --strip-components=1 && cd gdal && \
./configure \
--disable-debug \
--disable-static \
--prefix=${PREFIX} \
--with-openjpeg \
--with-geotiff=${PREFIX} \
--with-webp=${PREFIX} \
--with-zstd=${PREFIX} \
--with-jpeg=${PREFIX} \
--with-threads=yes \
--with-curl=${PREFIX}/bin/curl-config \
--without-python \
--without-libtool \
--with-geos=$PREFIX/bin/geos-config \
--with-hide-internal-symbols=yes \
CFLAGS="-O2 -Os" CXXFLAGS="-O2 -Os" \
LDFLAGS="-Wl,-rpath,'\$\$ORIGIN'" && \
make -j ${NPROC} install && \
cd ${BUILD} && rm -rf gdal
#postgis
RUN \
mkdir postgis && \
wget -qO- https://download.osgeo.org/postgis/source/postgis-2.5.2.tar.gz \
| tar xvz -C postgis --strip-components=1 && cd postgis && \
./configure --prefix=$PREFIX && \
make -j ${NPROC} install && \
cd ${BUILD} && rm -rf postgis
# Qt
RUN \
mkdir qt && \
wget -qO- http://download.qt.io/official_releases/qt/5.12/5.12.3/single/qt-everywhere-src-5.12.3.tar.xz \
| tar xvJ -C qt --strip-components=1
# -make libs -nomake tools -skip qttranslations -skip qttools -skip qtserialport -skip qtlocation
#./configure -opensource -confirm-license -nomake examples -nomake tests -skip qt3d -skip qtandroidextras -skip qtcanvas3d -skip qtcharts -skip qtconnectivity -skip qtdatavis3d -skip qtdeclarative -skip qtdoc -skip qtgamepad -skip qtgraphicaleffects -skip qtimageformats -skip qtmacextras -skip qtmultimedia -skip qtnetworkauth -skip qtpurchasing -skip qtquickcontrols -skip qtquickcontrols2 -skip qtscript -skip qtscxml -skip qtsensors -skip qtserialbus -skip qtspeech -skip qtvirtualkeyboard -skip qtwayland -skip qtwebchannel -skip qtwebengine -skip qtwebsockets -skip qtwebview -skip qtwinextras -skip qtx11extras -skip qtxmlpatterns && \
RUN cd qt && \
./configure -opensource -confirm-license -nomake examples -nomake tests -skip qt3d -skip qtandroidextras -skip qtcanvas3d -skip qtcharts -skip qtconnectivity -skip qtdatavis3d -skip qtdeclarative -skip qtdoc -skip qtgamepad -skip qtgraphicaleffects -skip qtimageformats -skip qtmacextras -skip qtmultimedia -skip qtnetworkauth -skip qtpurchasing -skip qtquickcontrols -skip qtquickcontrols2 -skip qtscript -skip qtscxml -skip qtsensors -skip qtserialbus -skip qtspeech -skip qtvirtualkeyboard -skip qtwayland -skip qtwebchannel -skip qtwebengine -skip qtwebsockets -skip qtwebview -skip qtwinextras -skip qtx11extras -skip qtxmlpatterns && \
gmake -j ${NPROC}
RUN \
cd qt && \
gmake install && \
cd ${BUILD} && rm -rf qt
# Make qt based libs able to detect and use qmake
ENV PATH="/usr/local/Qt-5.12.3/bin:${PATH}"
# SIP
# https://www.riverbankcomputing.com/static/Docs/sip/installation.html
# https://www.riverbankcomputing.com/static/Docs/PyQt5/installation.html#building-and-installing-from-source
RUN \
mkdir sip && \
wget -qO- https://www.riverbankcomputing.com/static/Downloads/sip/4.19.17/sip-4.19.17.tar.gz \
| tar xvz -C sip --strip-components=1 && \
cd sip && \
python3 configure.py --sip-module PyQt5.sip && \
make install && \
cd ${BUILD} && rm -rf sip
# PyQt
# https://www.riverbankcomputing.com/static/Docs/PyQt5/installation.html#building-and-installing-from-source
RUN \
mkdir pyqt && \
wget -qO- https://www.riverbankcomputing.com/static/Downloads/PyQt5/5.12.2/PyQt5_gpl-5.12.2.tar.gz \
| tar xvz -C pyqt --strip-components=1 && \
cd pyqt && \
python3 configure.py --confirm-license && \
make install && \
cd ${BUILD} && rm -rf pyqt
# qwt
# https://qwt.sourceforge.io/qwtinstall.html
RUN \
mkdir qwt && \
wget -qO- https://sourceforge.net/projects/qwt/files/qwt/6.1.4/qwt-6.1.4.tar.bz2/download \
| tar xvj -C qwt --strip-components=1 && cd qwt && \
qmake qwt.pro && \
make -j ${NPROC} install && \
cd ${BUILD} && rm -rf qwt
# qscintilla2
# https://www.riverbankcomputing.com/static/Docs/QScintilla/
RUN \
mkdir qscintilla2 && \
wget -qO- https://www.riverbankcomputing.com/static/Downloads/QScintilla/2.11.1/QScintilla_gpl-2.11.1.tar.gz \
| tar xvz -C qscintilla2 --strip-components=1 && cd qscintilla2/Qt4Qt5 && \
qmake && \
make -j ${NPROC} install
# Python bindings
RUN \
cd ${BUILD}/qscintilla2/Python && \
python3 configure.py --pyqt=PyQt5 && \
make -j ${NPROC} && make install && \
cd ${BUILD} && rm -rf qscintilla2
# qca
# https://github.com/KDE/qca
# https://www.mail-archive.com/kde-bugs-dist@kde.org/msg197506.html
# -DCMAKE_C_COMPILER=/usr/local/bin/gcc -DCMAKE_CXX_COMPILER=/usr/local/bin/c++
RUN \
mkdir qca && \
wget -qO- https://github.com/KDE/qca/archive/v2.1.3.tar.gz \
| tar xvz -C qca --strip-components=1 && cd qca && \
cmake -DCMAKE_INSTALL_PREFIX=${PREFIX} . && \
make -j ${NPROC} install && \
cd ${BUILD} && rm -rf qca
# qtkeychain
# /usr/local/Qt-5.12.3/lib/cmake/Qt5LinguistTools
RUN \
mkdir qtkeychain && \
wget -qO- https://github.com/frankosterfeld/qtkeychain/archive/v0.9.1.tar.gz \
| tar xvz -C qtkeychain --strip-components=1 && cd qtkeychain && mkdir build && cd build && \
cmake -DCMAKE_INSTALL_PREFIX=${PREFIX} .. && \
make -j ${NPROC} install && \
cd ${BUILD} && rm -rf qtkeychain
# exiv2 # Needed to build qgis/src/analysis otherwise linking of qgis/src/server fails...
RUN \
mkdir exiv2 && \
wget -qO- https://github.com/Exiv2/exiv2/archive/0.27.1.tar.gz \
| tar xvz -C exiv2 --strip-components=1 && \
cd exiv2 && \
mkdir build && cd build && \
cmake .. -DCMAKE_BUILD_TYPE=Release && \
cmake --build . && \
make install && \
cd ${BUILD} && rm -rf exiv2
# QGIS
ENV QT_SELECT=5
ENV LANG=C.UTF-8
RUN \
mkdir qgis && \
wget -qO- https://github.com/qgis/QGIS/archive/final-3_6_2.tar.gz \
| tar xvz -C qgis --strip-components=1
RUN yum install -y fcgi-devel
RUN \
cd qgis && \
gcc --version && \
cmake \
-G"Unix Makefiles" \
-DUSE_CCACHE=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr \
-DWITH_SERVER=ON \
# Building server fails without having server_plugins=on see https://issues.qgis.org/issues/21382
# Server_plugins fails without python
-DWITH_SERVER_PLUGINS=ON \
-DWITH_DESKTOP=OFF \
-DWITH_GUI=OFF \
-DWITH_QUICK=OFF \
-DWITH_3D=OFF \
# Maybe this turns of plugins like dbmanager
-DWITH_STAGED_PLUGINS=OFF \
-DSUPPRESS_QT_WARNINGS=ON \
-DDISABLE_DEPRECATED=ON \
-DENABLE_TESTS=OFF \
# Python must be enabled to build server_plugins
-DWITH_BINDINGS=ON \
-DWITH_QTWEBKIT=OFF \
-DWITH_QSPATIALITE=ON \
-DWITH_APIDOC=OFF \
-DWITH_ASTYLE=OFF \
# Building server fails without analysis
-DWITH_ANALYSIS=ON \
-DWITH_GEOREFERENCER=OFF \
-DWITH_GLOBE=OFF \
-DQWT_INCLUDE_DIR="/usr/local/qwt-6.1.4/include/" \
. \
&& make -j ${NPROC} install;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment