phfaist/environment-pyinstaller-osx-compatibility.md

## environment-pyinstaller-osx-compatibility.md

      
    Raw
  

              environment-pyinstaller-osx-compatibility.md
            
          
    Make mac binaries with pyinstaller that are backwards-compatible on Mac OS X

Here are some instructions to freeze Python applications that are compatible
with versions of OS X earlier than the one that PyInstaller is used on.
These steps involve compiling and installing Python, PyQt5 etc. manually.
Surprisingly, on my 2016 MacBook, it didn't take too long to set up.  The basic
idea is that Apple's LLVM/clang compiler can produce binaries that are
compatible with earlier versions of Mac OS X, but special compiler flags need to
be set so that the compiler uses the appropriate SDK.
These notes document, for reference, roughly what steps I did to make this work.
I'm sure there might be other solutions, or there might be problems I haven't
found out about yet.  In any case, no guarantees.
Here I'll use the Mac OS X 10.8 Mountain Lion SDK to freeze python applications
that are compatible with that version of the os.  A wonderful tool to manage OS
X SDK's is the Xcodelegacy project (Xcode just has this
distasteful habit of removing SDKs when it deems them to have sat there long
enough).
A clean environment

First, set up a clean environment. You want to make sure you won't interfere
with other Python environments and libraries.  I created a directory
/opt/env-osx108, and to set up the environment I use the following
instructions, which I've saved in a file /opt/env-osx108/SETUP_ENV.txt:
export PATH="/opt/env-osx108/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin"

export MACOSXSDK=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk

export MACOSX_DEPLOYMENT_TARGET="10.8"

export CFLAGS="-isysroot $MACOSXSDK  -I/opt/env-osx108/include "
export LDFLAGS="-isysroot $MACOSXSDK -L/opt/env-osx108/lib "
export CXXFLAGS="-isysroot $MACOSXSDK -I/opt/env-osx108/include "
export CPPFLAGS="-I$MACOSXSDK/usr/include -I/opt/env-osx108/include "

alias ls='ls -F'

This way, we can set up the environment simply by sourcing this file:
. /opt/env-osx108/SETUP_ENV.txt

UPDATE: I added the -I... and -L... in CFLAGS etc later, after having
compiled python, pyqt etc., but before openssl. This shouldn't make a difference,
though, right?
Python

I compiled and installed Python manually, using the Mac 10.8 SDK.  I downloaded
the python source from their download page,
and ran the configure script as follows:
./configure --prefix=/opt/env-osx108/ \
            --enable-ipv6 \
            --enable-framework=/opt/env-osx108/Frameworks/ \
            --without-gcc \
            CFLAGS="-isysroot $MACOSXSDK -I$MACOSXSDK/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers" \
            LDFLAGS="-isysroot $MACOSXSDK" \
            CPPFLAGS="-I$MACOSXSDK/usr/include" \
            MACOSX_DEPLOYMENT_TARGET="10.8"

(It is inspired by homebrew's python formula.)
Make with make and make install PYTHONAPPSDIR=/opt/env-osx108/Applications
pip packages

Pure-source python packages can be installed with pip3 install <pkg> w/o any
problem.
Make sure you don't install wheels of binaries.  You can use
pip3 install --no-binary :all: ...

You can also use --verbose to verify compilation commands.
SIP & PyQt5

First, install Qt5.  The last version of Qt5 that is compatible with Mountain
Lion is Qt 5.7.1, which is available
from Qt's archives.
My Qt5 installation is in /opt/Qt5.7.1, with qmake at the location
/opt/Qt5.7.1/5.7/clang_64/bin/qmake.
It is tempting to install python wheels with pip3 install PyQt5, but it turns
out that the binaries provided are not compatible with Mountain Lion (even
though the wheel's filename suggests that it's compatible with mac 10.6).
First we compile & install sip.  I downloaded this file
from PyQt5's download files and extracted it.  I used the
following configure command:
python3 configure.py \
        --deployment-target=10.8 \
        --sdk=$MACOSXSDK \
        --sysroot=/opt/env-osx108

(Remember that $MACOSXSDK is the full path to the *.sdk folder.)
For some reason, however, running make didn't seem to use the correct
SDK. (You can inspect which version of OSX a binary is compiled for by using
otool -l ... and looking for a LC_VERSION_MIN_MACOSX section.)  So I hacked
around the make command to achieve what I wanted:
cd sip-4.19.8/
(cd sipgen && make CFLAGS="-pipe -Os -Wall -W -isysroot $MACOSXSDK" CXXFLAGS="-pipe -Os -Wall -W -isysroot $MACOSXSDK" LFLAGS="-headerpad_max_install_names -isysroot $MACOSXSDK")
(cd siplib && make CFLAGS="-pipe -fPIC -Os -Wall -W -isysroot $MACOSXSDK" CXXFLAGS="-pipe -fPIC -Os -Wall -W -isysroot $MACOSXSDK" LFLAGS="-headerpad_max_install_names -bundle -undefined dynamic_lookup -isysroot $MACOSXSDK")

and then
make
make install

I followed a similar procedure for PyQt5, using this download.
Here is the configure command I used:
python3 configure.py \
        --sysroot=/opt/env-osx108 \
        --confirm-license \
        --qmake=/opt/Qt5.7.1/5.7/clang_64/bin/qmake \
        --no-tools \
        --designer-plugindir=/opt/env-osx108/pyqt5-plugins/designer \
        --qml-plugindir=/opt/env-osx108/pyqt5-plugins/PyQt5 \
        --no-python-dbus

For the compilation step, the wrong SDK was being used and I had no idea how to
fix it properly.  So I ended up hacking the Makefile, to replace all instances
of the wrong SDK with the correct SDK.  On my system, the makefiles in
QtCore/Makefile etc. all had explicit flags -isysroot /Applications/Xcode.app/..../SDKs/MacOSX10.13.sdk, referring to my actual
current system and not the SDK I wanted PyQt5 to use, so I ran a perl
one-liner to replace all these flag instances with the correct SDK reference:
for i in */Makefile; do
    perl -pi -we 's|/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk|/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk|g' "$i"
done

and then, as usual,
make
make install

Finally, I also had trouble getting QLibraryInfo to report the right paths.
This caused a crash with an error message about Qt not being able to find the
cocoa platform plugin.  Curiously, that turned out to be a problem with the
base Qt library apparently, not PyQt's.  I fixed that by creating the file
/opt/env-osx108/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/Resources/qt.conf
with the following contents:
[Paths]
Prefix = /opt/Qt5.7.1/5.7/clang_64

OpenSSL

Compile a more recent version of OpenSSL for requests and other crypto/https-related packages.
I downloaded openssl here, version 1.0.2r; untarred in local directory.  With the
environment set up as above, ran the configure command:
> ./Configure --prefix=/opt/env-osx108/ no-hw no-hw-xxx no-ssl2 no-ssl3 no-zlib zlib-dynamic shared enable-cms darwin64-x86_64-cc enable-ec_nistp_64_gcc_128 -isysroot$MACOSXSDK -mmacosx-version-min=10.8

And then:
> make depend
> make
> make install

I could then install SSL stuff required by requests:
pip3 install --no-binary :all: requests[security] --verbose

PyInstaller

Can be installed using pip:
pip3 install --no-binary :all: pyinstaller --verbose

(Actually it looks like these don't compile any binaries, so you probably don't
need fancy options like I used.)
I also symlinked the pyinstaller executable to the environment's binary dir:
cd /opt/env-osx108/bin && ln -s /opt/env-osx108/Frameworks/Python.framework/Versions/3.6/bin/pyinstaller .

That's it! Now I can freeze PyQt5 applications with PyInstaller that can run on
Mountain Lion or later.