opencv-python piwheels build scripts

README.rst

opencv-python build scripts

This is all extremely hacky so you'll probably need to play around with it to get it working. On a fresh Raspbian Jessie or Raspbian Stretch install, add all the build-deps we usually have on a piwheels build slave (see https://github.com/bennuttall/piwheels/blob/master/deploy_slave.sh and install all the -dev stuff and python libs, basically). Next, add a 1Gb swap file.

In the default "pi" user, make a "packaging" directory and stuff opencv-python-build and opencv-setup.py under there. Make opencv-python-build executable and edit the for-loops to build what you want (the "v" loop at the top determines the versions to build; 18 is the latest at the time of writing - the other loops like "contrib", "headless", etc. determine the variants to build).

From within the "packaing" directory, run the opencv-python-build script. Depending on the number of versions and variants you're build: sit back and wait several days :)

At the end you should have several .whl files and bunch of *.log files sat around in your packaging directory (which is usually what I then use to import into piwheels).

opencv-python-build

#!/bin/bash
# vim: set et sw=4 sts=4:

set -eu

PYTHON_VERSION_STRING=$(python3 -c "from platform import python_version; print(python_version())")
PYTHON_INCLUDE_PATH=$(python3 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())")
PYTHON_PACKAGES_PATH=$(python3 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")
PYTHON_NUMPY_INCLUDE_DIRS=$(python3 -c "import numpy.distutils; print(numpy.distutils.misc_util.get_numpy_include_dirs()[0])")
PYTHON_NUMPY_VERSION=$(python3 -c "import numpy; print(numpy.version.version)")

if [[ ! -d opencv-python ]]; then
    git clone https://github.com/skvark/opencv-python.git
else
    pushd opencv-python
    git fetch --all
    popd
fi

for v in 18; do
    pushd opencv-python
    git reset --hard
    git checkout $v
    git submodule sync
    git submodule update --init
    for contrib in 0 1; do
        for headless in 0 1; do
            MAKE_ERR=0
            if [[ $headless -eq 1 ]] && [[ $v -lt 13 ]]; then
                echo '!!' "Skipping headless build on old version"
                break
            fi
            echo $contrib > contrib.enabled
            echo $headless > headless.enabled
            for neon in 0 1; do
                date
                LOG_BASE=${HOME}/packaging/opencv-v${v}-contrib${contrib}-headless${headless}-neon${neon}
                export ENABLE_CONTRIB=$(<contrib.enabled)
                export ENABLE_HEADLESS=$(<headless.enabled)
                echo "Preparing ${LOG_BASE}"
                if [[ $contrib -eq 1 ]]; then
                    CONTRIB_PATH=${HOME}/packaging/opencv-python/opencv_contrib/modules
                else
                    CONTRIB_PATH=
                fi
                if [[ $headless -eq 1 ]]; then
                    QT_VERSION=OFF
                else
                    QT_VERSION=4
                fi
                if [[ $neon -eq 1 ]]; then
                    NEON_SWITCH=ON
                else
                    NEON_SWITCH=OFF
                fi
                rm -fr opencv/build
                mkdir opencv/build
                MAKE_ERR=0
                cmake opencv -B"opencv/build" \
                        -G "Unix Makefiles" \
                        -DOPENCV_EXTRA_MODULES_PATH=$CONTRIB_PATH \
                        -DCMAKE_BUILD_TYPE=Release \
                        -DBUILD_opencv_apps=OFF \
                        -DBUILD_opencv_python2=OFF \
                        -DBUILD_opencv_python3=ON \
                        -DBUILD_opencv_java=OFF \
                        -DBUILD_SHARED_LIBS=OFF \
                        -DENABLE_NEON=$NEON_SWITCH \
                        -DENABLE_VFPV3=$NEON_SWITCH \
                        -DENABLE_PRECOMPILED_HEADERS=OFF \
                        -DWITH_IPP=OFF \
                        -DBUILD_DOCS=OFF \
                        -DWITH_QT=$QT_VERSION \
                        -DINSTALL_C_EXAMPLES=OFF \
                        -DINSTALL_PYTHON_EXAMPLES=OFF \
                        -DBUILD_TESTS=OFF \
                        -DBUILD_PERF_TESTS=OFF \
                        -DBUILD_EXAMPLES=OFF \
                        -DPYTHON3INTERP_FOUND=ON \
                        -DPYTHON3LIBS_FOUND=ON \
                        -DPYTHON3_EXECUTABLE=python3 \
                        -DPYTHON3_VERSION_STRING="$PYTHON_VERSION_STRING" \
                        -DPYTHON3_INCLUDE_PATH="$PYTHON_INCLUDE_PATH" \
                        -DPYTHON3_PACKAGES_PATH="$PYTHON_PACKAGES_PATH" \
                        -DPYTHON3_NUMPY_INCLUDE_DIRS="$PYTHON_NUMPY_INCLUDE_DIRS" \
                        -DPYTHON3_NUMPY_VERSION="$PYTHON_NUMPY_VERSION" \
                        -DWITH_JPEG=ON \
                        -DBUILD_JPEG=OFF \
                        -DWITH_V4L=ON >${LOG_BASE}-cmake.log 2>&1 || MAKE_ERR=$?
                if [[ $MAKE_ERR -eq 0 ]]; then
                    make -C opencv/build -j3 >${LOG_BASE}-make.log 2>&1 || MAKE_ERR=$?
                    if [[ $MAKE_ERR -eq 0 ]]; then
                        cp opencv/build/lib/python3/*.so cv2/
                        cp ../opencv-setup.py ./setup.py
                        pip3 wheel --log ${LOG_BASE}-pip.log -w . --no-deps --no-cache-dir . >/dev/null
                        WHEEL=$(echo *.whl) # dirty hack
                        if [ $neon -eq 0 ]; then
                            mv $WHEEL ../${WHEEL/armv7l/armv6l}
                        else
                            mv $WHEEL ..
                        fi
                    fi
                fi
                if [[ $neon -eq 0 ]] && [[ $MAKE_ERR -ne 0 ]]; then
                    echo '!!' "Skipping ARMv7 build because ARMv6 build failed"
                    break
                fi
            done
        done
        if [[ $contrib -eq 0 ]] && [[ $MAKE_ERR -ne 0 ]]; then
            echo '!!' "Skipping contrib build because non-contrib build failed"
            break
        fi
    done
    popd
done
date

opencv-setup.py

import os
import io
from subprocess import check_output
from pkg_resources import require
from setuptools import setup, find_packages

with io.open('opencv/modules/core/include/opencv2/core/version.hpp', encoding="utf-8") as f:
    opencv_version = ""
    for line in f:
        words = line.split()
        if "CV_VERSION_MAJOR" in words:
            opencv_version += words[2]
            opencv_version += "."
        if "CV_VERSION_MINOR" in words:
            opencv_version += words[2]
            opencv_version += "."
        if "CV_VERSION_REVISION" in words:
            opencv_version += words[2]
            break
git_tag = int(check_output([
    'git', 'name-rev', '--tags', '--name-only', 'HEAD'
    ]).splitlines()[0].decode())
opencv_version += ".{}".format(git_tag)
if git_tag >= 13:
    headless_build = bool(int(os.getenv('ENABLE_HEADLESS', '0')))
else:
    headless_build = False
contrib_build = bool(int(os.getenv('ENABLE_CONTRIB', '0')))
numpy_version = require('numpy')[0].version
package_name = "opencv{contrib}-python{headless}".format(
    contrib="-contrib" if contrib_build else "",
    headless="-headless" if headless_build else "")
package_data = {
    '': ['*.xml'],
    'cv2': ['*.so', 'LICENSE.txt', 'LICENSE-3RD-PARTY.txt'],
}
if git_tag < 13:
    if contrib_build:
        with io.open('README_CONTRIB.rst', encoding="utf-8") as f:
            long_description = f.read()
    else:
        with io.open('README.rst', encoding="utf-8") as f:
            long_description = f.read()
else:
    with io.open('README.md', encoding="utf-8") as f:
        long_description = f.read()

print(opencv_version)
print(numpy_version)
print(package_name)
print(repr(package_data))

# This creates a list which is empty but returns a length of 1.
# Should make the wheel a binary distribution and platlib compliant.
class EmptyListWithLength(list):
    def __len__(self):
        return 1


setup(name=package_name,
      version=opencv_version,
      url='https://github.com/skvark/opencv-python',
      license='MIT',
      description='Wrapper package for OpenCV python bindings.',
      long_description=long_description,
      packages=find_packages(),
      package_data=package_data,
      maintainer="Olli-Pekka Heinisuo",
      include_package_data=True,
      ext_modules=EmptyListWithLength(),
      install_requires="numpy>=%s" % numpy_version,
      classifiers=[
        'Development Status :: 5 - Production/Stable',
        'Environment :: Console',
        'Intended Audience :: Developers',
        'Intended Audience :: Education',
        'Intended Audience :: Information Technology',
        'Intended Audience :: Science/Research',
        'License :: OSI Approved :: MIT License',
        'Operating System :: MacOS',
        'Operating System :: Microsoft :: Windows',
        'Operating System :: POSIX',
        'Operating System :: Unix',
        'Programming Language :: Python',
        'Programming Language :: Python :: 2',
        'Programming Language :: Python :: 2.7',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.4',
        'Programming Language :: Python :: 3.5',
        'Programming Language :: Python :: 3.6',
        'Programming Language :: C++',
        'Programming Language :: Python :: Implementation :: CPython',
        'Topic :: Scientific/Engineering',
        'Topic :: Scientific/Engineering :: Image Recognition',
        'Topic :: Software Development',
        ]
      )