Skip to content

Instantly share code, notes, and snippets.

@j9ac9k

j9ac9k/macOS_app_creation.md

Last active March 11, 2025 21:36
Download ZIP

Revisions

  1. j9ac9k revised this gist Sep 4, 2019. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion macOS_app_creation.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    # Building PyQt5/PySide2 macOS Application
    # Building a PyQt5/PySide2 macOS Application

    ## Preface

  2. j9ac9k revised this gist Sep 3, 2019. 1 changed file with 13 additions and 1 deletion.
    14 changes: 13 additions & 1 deletion macOS_app_creation.md
    View file
    Original file line number Diff line number Diff line change
    @@ -416,7 +416,7 @@ The build time on my MacBook Air was approximately 30 minutes. 3 wheels will be
    pip install --no-index --find-links=path_to_wheel_directory pyside2
    ```

    This will install the pyside2 and shiboken2 wheels you just created instead of the ones on PyPI.
    This will install the `pyside2` and `shiboken2` wheels you just created instead of the ones on PyPI.

    ### Querying Git Commits for Versioning

    @@ -439,3 +439,15 @@ My working theory here is that `logging.FileHandler()` takes only relative paths
    After some googling, the main suggestion seems to be to [create your own logging handler](https://stackoverflow.com/questions/28655198/best-way-to-display-logs-in-pyqt). I have not implemented this myself yet, so I cannot speak to it, so for now the only advice I'll give you, is to not write logs via `FileHandler`.
    If any readers have a good solution to the logging issue with python app bundles, please let me know, I am all ears.
    ### Homebrew `coreutils` Package
    If you use `homebrew` and have the `coreutils` package, make sure you do not have your `PATH` modified so that the g-prefixed utilities under their normal names. Modifying `PATH` so that the `coreutils` utilities are available under their normal names [will break the `fbs installer` command](https://github.com/mherrmann/fbs/issues/105). If you run into this issue, you likely just need to _remove_ this line in your `.bashrc` or `.zshrc`.
    ```text
    PATH="/usr/local/opt/coreutils/libexec/gnubin:$PATH"
    ```
    ### PyInstaller Virtualenv and Distutils
    Talk about `distutils` bundling issue with `pyinstaller`
  3. j9ac9k revised this gist Sep 2, 2019. 1 changed file with 27 additions and 27 deletions.
    54 changes: 27 additions & 27 deletions macOS_app_creation.md
    View file
    Original file line number Diff line number Diff line change
    @@ -6,7 +6,7 @@ As part of my professional duties, I have been tasked with developing a desktop

    In short, here were the objectives I aimed to have

    * Cross-Platform compatable GUI application
    * Cross-Platform compatible GUI application
    * Deployment to users via native installers (no assumption of python being on the target machine)

    This post details the steps I took to create the installer and some issues I came across the way. I will be focusing on the macOS component as I did some extra steps such as code-signing my application, as well as notarizing it.
    @@ -15,17 +15,17 @@ What this blog post will not contain is instructions on building and running a Q

    ### Code Signing - What Is It

    Codesigning is the process of telling your end users that, you, or your organization is the one that developed the application. To codesign an application, you will need an [Apple Developer ID Application Certificate](https://developer.apple.com/support/certificates/) which costs $100/year (you get more than the cert with the subscription, but the cert is what is needed here). While codesigning is not a requirement for an end user to install an application, it is a requirement to get apple to let you by [GateKeeper](https://support.apple.com/en-us/HT202491).
    Code signing is the process of telling your end users that, you, or your organization is the one that developed the application. To code-sign an application, you will need an [Apple Developer ID Application Certificate](https://developer.apple.com/support/certificates/) which costs $100/year (you get more than the cert with the subscription, but the cert is what is needed here). While code-signing is not a requirement for an end user to install an application, it is a requirement to get apple to let you by [GateKeeper](https://support.apple.com/en-us/HT202491).

    > Note: `fbs` does have some codesign support but at the time of this writing it is only for `.deb` packages on linux. For macOS (and Windows) we will have to codesign ourselves.
    > Note: `fbs` does have some code-sign support but at the time of this writing it is only for `.deb` packages on linux. For macOS (and Windows) we will have to code-sign ourselves.
    ## Shoulders of Giants

    There are a number of libraries that are used along the way. I wanted my application to be installable as a python binary wheel, and I wanted to deploy native installers for end users. The python library that has an easy to use API and fantastic documentation generate the installers is [fbs](https://build-system.fman.io/). `fbs` makes heavy use of `pyinstaller` under the hood but provides a nice, cross-platform compatable API, but it requires some specific directory strcture that is not intuitive (which I will get to later).
    There are a number of libraries that are used along the way. I wanted my application to be installable as a python binary wheel, and I wanted to deploy native installers for end users. The python library that has an easy to use API and fantastic documentation generate the installers is [fbs](https://build-system.fman.io/). `fbs` makes heavy use of `pyinstaller` under the hood but provides a nice, cross-platform compatible API, but it requires some specific directory structure that is not intuitive (which I will get to later).

    The work here could not be done without the hard work of maintainers and contributors of `setuptools`, `pyinstaller`, and `fbs` libraries. Furthermore, the `pyinstaller` [issue tracker](https://github.com/pyinstaller/pyinstaller/issues) has been a gold mine of information when I am getting unexplained behavior.

    I also want to point out the [Qt for Python developer gitter channel](https://gitter.im/PySide/pyside2) is a great place to ask for help if needed (these are the PySide2 devs, keep in mind they're pretty busy so don't expect a prompt reply). The folks I have interacted with there have been very helpful, and have verified bugs I have come across.
    I also want to point out the [Qt for Python developer gitter channel](https://gitter.im/PySide/pyside2) is a great place to ask for help if needed (these are the PySide2 developers, keep in mind they're pretty busy so don't expect a prompt reply). The folks I have interacted with there have been very helpful, and have verified bugs I have come across.

    ## Considerations As We Start

    @@ -35,7 +35,7 @@ Pin your dependencies folks. `pyinstaller` is fragile (or I should say, the pro

    With pinned dependencies, you can go back in git history and recreate the environment at the time of the commit to a very precise level. You can pin dependencies via `pipenv` or making use of the output of `pip freeze` and `requirements.txt` files. I currently use `pipenv` but using the output of `pip freeze` is sufficient. With `pipenv` I ran into my first gotcha, you have to be careful with _which_ python binary it grabs.

    > **GOTCHA:** PySide 5.12 (and maybe more recent versions) has an issue where if you are using a conda distributed version of python, your application will not respect macOS's dark mode setting (tl;dr no dark mode for your app). This is problematic with `pipenv` as when `pipenv` creates a virtual environment, it searches through `$PATH` for the first python binary that matches the version requirements, if that version of python comes from a conda distribution, you have _silenty_ broken dark mode on your application. In short, if you use `pipenv` and have a `conda` distribution of python on your system, be very careful.
    > **GOTCHA:** PySide 5.12 (and maybe more recent versions) has an issue where if you are using a conda distributed version of python, your application will not respect macOS's dark mode setting (tl;dr no dark mode for your app). This is problematic with `pipenv` as when `pipenv` creates a virtual environment, it searches through `$PATH` for the first python binary that matches the version requirements, if that version of python comes from a conda distribution, you have _silently_ broken dark mode on your application. In short, if you use `pipenv` and have a `conda` distribution of python on your system, be very careful.
    <!-- -->
    > **GOTCHA:** `pipenv` [does not handle OS specific dependencies well](https://github.com/pypa/pipenv/issues/1954) and you will have OS specific build dependencies. If you're going to use `pipenv` I urge you to be careful with dependencies you will need come build time (like `pypiwin32` for Windows), and store those dependencies outside of your `Pipfile`.
    @@ -108,7 +108,7 @@ Before building the app bundle, there are a few checks we want to do to ensure t

    > What you usually don't want to do is to refer to fbs from your application's implementation (src/main/). If at all, you should only refer to fbs from build scripts (build.py and/or src/build/python/).

    What this lets you do is you can have `fbs run` launch your application, but we are not required to have `fbs` as part of runtime dependencies. The alternative being lauching your application via `setuptools` entry point, which can be of the form of of running `python src/main/python/my_app/__main__.py`, and having a corresponding file with something along the lines of
    What this lets you do is you can have `fbs run` launch your application, but we are not required to have `fbs` as part of runtime dependencies. The alternative being launching your application via `setuptools` entry point, which can be of the form of of running `python src/main/python/my_app/__main__.py`, and having a corresponding file with something along the lines of

    ```python
    import os
    @@ -147,7 +147,7 @@ Before building the app bundle, there are a few checks we want to do to ensure t

    3. Create an `Info.plist` file

    ... and place it in `./src/freeze/mac/Contents/Info.plist`. The following is what I creaated, mostly getting tips from google. The `${}` fields are grabbed from the `build.json` file above under the `public_settings` key.
    ... and place it in `./src/freeze/mac/Contents/Info.plist`. The following is what I created, mostly getting tips from google. The `${}` fields are grabbed from the `build.json` file above under the `public_settings` key.

    ```xml
    <?xml version="1.0" encoding="UTF-8"?>
    @@ -218,21 +218,21 @@ fbs installer
    That command will produce a `MyApp.dmg` file that you can distribute to others.
    If you do not have an Apple Developer account/subscription, or if you just have no interest in codesigning your application, you can stop here.
    If you do not have an Apple Developer account/subscription, or if you just have no interest in code-signing your application, you can stop here.
    ## Codesigning and Notarizing
    ## Code-signing and Notarizing
    If you want to codesign your application, it can be done, but fair warning, you are proceeding into mostly uncharted territory. Apple's documentation written with Xcode projects in mind, so a lot of it will not be relevant. That said, the issue tracker on pyinstaller has been very helpful for debugging issues.
    If you want to code-sign your application, it can be done, but fair warning, you are proceeding into mostly uncharted territory. Apple's documentation written with Xcode projects in mind, so a lot of it will not be relevant. That said, the issue tracker on pyinstaller has been very helpful for debugging issues.

    ### Fix Gendrated .app Bundle
    ### Fix Generated .app Bundle

    Before codesigning your application, we actually need to "fix" the app bundle that pyinstaller generated. Here is a [link to the issue on the pyinstaller repo](https://github.com/pyinstaller/pyinstaller/issues/3680), and the associated [pyinstaller wiki entry](https://github.com/pyinstaller/pyinstaller/wiki/Recipe-OSX-Code-Signing-Qt). The issue tracker goes into what the issue is, but it fixes a couple of problems, one of which is macOS treating directories with periods in them as separate bundles. The tl;dr is that you should run the script against your `fbs` generates `.app bundle`.
    Before code-signing your application, we actually need to "fix" the app bundle that pyinstaller generated. Here is a [link to the issue on the pyinstaller repo](https://github.com/pyinstaller/pyinstaller/issues/3680), and the associated [pyinstaller wiki entry](https://github.com/pyinstaller/pyinstaller/wiki/Recipe-OSX-Code-Signing-Qt). The issue tracker goes into what the issue is, but it fixes a couple of problems, one of which is macOS treating directories with periods in them as separate bundles. The tl;dr is that you should run the script against your `fbs` generates `.app bundle`.

    The modifications I made removed unneeded files that would not preserve their extended attributes during a move, and would then invalidate the codesigning.
    The modifications I made removed unneeded files that would not preserve their extended attributes during a move, and would then invalidate the code-signing.

    ### Create Application Entitlements (optional)

    One of the requirements for application notarization (which will be required starting macOS 10.15) is codesigning with a [hardened runtime](https://developer.apple.com/documentation/security/hardened_runtime_entitlements). There may be "entitlements" you want to enable your application to have (check the link above for what's available), but if you are bundling `numba` or anything else that wants to execute JIT-compiled code, you should consider adding the [Allow Execution of JIT-compiled Code Entitlement](https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_cs_allow-jit?language=objc).
    One of the requirements for application notarization (which will be required starting macOS 10.15) is code-signing with a [hardened runtime](https://developer.apple.com/documentation/security/hardened_runtime_entitlements). There may be "entitlements" you want to enable your application to have (check the link above for what's available), but if you are bundling `numba` or anything else that wants to execute JIT-compiled code, you should consider adding the [Allow Execution of JIT-compiled Code Entitlement](https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_cs_allow-jit?language=objc).
    To add entitlements is fairly straight forward, in your project root folder create a file `.entitlements`, and put the following contents in it (of course add entitlements you think are needed).
    @@ -247,9 +247,9 @@ To add entitlements is fairly straight forward, in your project root folder crea
    </plist>
    ```
    ### Finally To the Codesigning Part
    ### Finally To the Code-signing Part
    After running this step, you can now codesign your `.app` bundle!
    After running this step, you can now code-sign your `.app` bundle!
    ```bash
    codesign -fs "$CODESIGN_ID" --deep --force --verbose -o runtime --entitlements .entitlements --preserve-metadata=identifier,entitlements,requirements,runtime --timestamp ./target/MyApp.app
    @@ -260,12 +260,12 @@ Some explanation of the options
    * `-o runtime` enables hardened runtime (required for app notarization)
    * `--entitlements .entitlements` passes the entitlements you created to your application
    * `--timestamp` enables secure timestamp option (also required for notarization)
    * `--deep` does a recursive codesign through the bundle. The Apple documentation recommends using this _only_ for troubleshooting, however the docuemntation is aimed toward Xcode projects, and not particularly relevant in our use case.
    * `--deep` does a recursive code-sign through the bundle. The Apple documentation recommends using this _only_ for troubleshooting, however the documentation is aimed toward Xcode projects, and not particularly relevant in our use case.
    * `"$CODESIGN_ID` is an env variable for my "Developer ID Application" certificate name that I have stored in my system keychain
    ### Verifying the codesign operation
    > **GOTCHA:** Doing a verification of the codesign after you do a codesign sounds reasonable, right? *WRONG* queue evil laugh. Part of the codesign operation involves verifying the "extended attributes" of the files inside the .app bundle. When you copy/move the .app bundle, the extended attributes change, but if you never touch the file, the extended attributes will remain fixed, and a codesign verficiation step may falsely "pass" (don't ask me how long it took me to troubleshoot this).
    > **GOTCHA:** Doing a verification of the `codesign` command ran as expected after you do run `codesign` sounds reasonable, right? *WRONG* queue evil laugh. Part of the codesign operation involves verifying the "extended attributes" of the files inside the .app bundle. When you copy/move the .app bundle, the extended attributes change, but if you never touch the file, the extended attributes will remain fixed, and a codesign verification step may falsely "pass" (don't ask me how long it took me to troubleshoot this).

    To address this, before verifying the `codesign` operation, we need to modify the extended attributes, which can be done by running the following command.

    @@ -286,7 +286,7 @@ That should return
    ./target/MyApp.app: satisfies its Designated Requirement
    ```

    Feeling paranoid? I know I have been during this process, there is a second way we can verify the codesigning operation
    Feeling paranoid? I know I have been during this process, there is a second way we can verify the code-signing operation

    ```bash
    spctl --assess --verbose ./target/MyApp.app
    @@ -299,7 +299,7 @@ This should return
    source=Developer ID
    ```

    At this point, the application is codesigned, and we should generate our `.dmg` file.
    At this point, the application is code-signed, and we should generate our `.dmg` file.

    ```bash
    fbs installer
    @@ -313,7 +313,7 @@ hdiutil: internet-enable: enable succeeded
    Created target/MyApp.dmg.
    ```

    So now we have a `.app` bundle that is codesigned and a `.dmg` file that is made from the codesigned bundle; next step, we need to [notarize the application](https://developer.apple.com/documentation/security/notarizing_your_app_before_distribution).
    So now we have a `.app` bundle that is code-signed and a `.dmg` file that is made from the code-signed bundle; next step, we need to [notarize the application](https://developer.apple.com/documentation/security/notarizing_your_app_before_distribution).

    ### Notarizing your Application

    @@ -362,7 +362,7 @@ xcrun altool --notarization-history 0 -u $APPLE_DEVELOPER_EMAIL -p "@keychain:AC

    If there is an error, it will return a URL you can look at and see more specific issues.

    > **GOTCHA:** PySide 5.12.0, 5.12.1, 5.12.2, 5.12.3, 5.12.4 and 5.13.0 on PyPi had some of their binaries build linking to an incompatable SDK. They are fully functional, but if you use the wheels them, notarization will fail. I reported the issue to their [bug-tracker here](https://bugreports.qt.io/browse/PYSIDE-1066) and they have resolved it in time for the next release (5.12.5 and 5.13.1). If you are using any of these libraries you will need to build the pyside2 wheels yourself, and when installing dependencies of your app you will need to install those custom built wheels, not the ones from PyPI. See the Appendix for instructions
    > **GOTCHA:** PySide 5.12.0, 5.12.1, 5.12.2, 5.12.3, 5.12.4 and 5.13.0 on PyPi had some of their binaries build linking to an incompatible SDK. They are fully functional, but if you use the wheels them, notarization will fail. I reported the issue to their [bug-tracker here](https://bugreports.qt.io/browse/PYSIDE-1066) and they have resolved it in time for the next release (5.12.5 and 5.13.1). If you are using any of these libraries you will need to build the pyside2 wheels yourself, and when installing dependencies of your app you will need to install those custom built wheels, not the ones from PyPI. See the Appendix for instructions

    If it the query status says the package is accepted, you can "staple" the notarization with the following command

    @@ -385,13 +385,13 @@ Similarly, when I noticed my .app bundle wouldn't run after I upgraded from nump
    * `numpy.random.bounded_integers`
    * `numpy.random.entropy`
    For the record, I do not think these needed to be explicitely defined prior as of numpy 1.17.1. Once those imports/modules were added, and I rebuilt my `.app` bundle, it ran successfully.
    For the record, I do not think these needed to be explicitly defined prior as of numpy 1.17.1. Once those imports/modules were added, and I rebuilt my `.app` bundle, it ran successfully.
    That `hidden_imports` feature will often be the first place to look to patch things up, especially when you know it is an issue with part of a dependency that is missing.
    ### Building Custom PySide2 Wheels
    If you are using PySide 5.12.0, 5.12.1, 5.12.2, 5.12.3, 5.12.4, or 5.13.0, and plan to noterize your application, you will need to build your own wheels, as there is an [issue with the SDK linked](https://bugreports.qt.io/browse/PYSIDE-1066). Luckily for us, the Qt for Python folks have provided [instructions on building PySide from source](https://wiki.qt.io/Qt_for_Python/GettingStarted/MacOS).
    If you are using PySide 5.12.0, 5.12.1, 5.12.2, 5.12.3, 5.12.4, or 5.13.0, and plan to notarize your application, you will need to build your own wheels, as there is an [issue with the SDK linked](https://bugreports.qt.io/browse/PYSIDE-1066). Luckily for us, the Qt for Python folks have provided [instructions on building PySide from source](https://wiki.qt.io/Qt_for_Python/GettingStarted/MacOS).
    I won't repeat the instructions on the wiki here, but the dependencies needed to build the wheels involve

    @@ -424,9 +424,9 @@ This was a "gotcha" that was an issue for almost a month before I figured out wh

    Perfectly sensible.

    What the issue was, that it did this at runtime, and when the applicationo is in bundle form, it's no longer in a git repository, thus querying the git status must generate an error.
    What the issue was, that it did this at runtime, and when the application is in bundle form, it's no longer in a git repository, thus querying the git status must generate an error.
    The frustarting part here is that the failure mode was when opening `MyApp.app`, the icon would bounce for a few seconds, and then just close out, with no meaningful error of any kind.
    The frustrating part here is that the failure mode was when opening `MyApp.app`, the icon would bounce for a few seconds, and then just close out, with no meaningful error of any kind.
    The way I isolated this issue was via git bisect, and I tracked down the commit with the diff, which incorporated the feature I just described.
  4. j9ac9k revised this gist Sep 2, 2019. No changes.
  5. j9ac9k revised this gist Sep 2, 2019. 1 changed file with 33 additions and 28 deletions.
    61 changes: 33 additions & 28 deletions macOS_app_creation.md
    View file
    Original file line number Diff line number Diff line change
    @@ -2,48 +2,47 @@

    ## Preface

    As part of my professional duties, I have been tasked with developing a desktop application to analyze audio files, and make use of data generated by other internal tools. This desktop application would need to be cross platform compatable (Windows, macOS and Ubuntu), and the end users may or may not have much in the way of dependencies installed. I naturally turned to Qt (and more specifically, Qt for Python) due to my past experience with it, and the ease that you can create desktop applications with it. One of the requirements was to deploy this application to users machines that may or may not have developer dependencies (such as `python` itself) already on there. This made things more complicated; so I decided it may be worth trying to make native installers for each operating system (`deb` packages for linux, `.app` bundles and `.dmg` files for macOS, and `setup.exe` files for Windows). What I had no idea on, how do I build a PyQt5/PySide2 application, and bundle it inside a native installers.
    As part of my professional duties, I have been tasked with developing a desktop application to analyze audio files. This desktop application would need to be work across platforms (Windows, macOS and Ubuntu), and I cannot rely on dependencies having already been installed (such as Python itself). Naturally I turned to Qt (and more specifically, Qt for Python) due to its capabilities for creating GUI applications that work across OSs. As I could not be dependent on the end user having Python installed (so no deploying a wheel and calling it good), I decided it may be worth trying to make native installers (`deb` packages for linux, `.app` bundles and `.dmg` files for macOS, and `setup.exe` files for Windows).

    In short, here were the objectives I aimed to have

    * Cross platform compatable codebase
    * Cross-Platform compatable GUI application
    * Deployment to users via native installers (no assumption of python being on the target machine)
    * Installable with binary wheel

    This post details the steps I took to create the installer and some issues I came across the way. I will be focusing on the macOS component as I did some extra steps such as code-signing my application, as well as notarizing it, which I will detail how to do so as well.
    This post details the steps I took to create the installer and some issues I came across the way. I will be focusing on the macOS component as I did some extra steps such as code-signing my application, as well as notarizing it.

    What this blog post will not contain is instructions on building and running a Qt application in Python in general. To get a sample minimum working example after you install `fbs` you can run `fbs startproject`.
    What this blog post will not contain is instructions on building and running a Qt application in Python in general. To get a sample minimum working example after you install `fbs` you can run `fbs startproject`, and one will be generated for you.

    ### Code Signing - What Is It

    Codesigning is the process of telling your end users that, you, or your organization is the one that developed the application. To codesign an application, you will need an [Apple Developer ID Application Certificate](https://developer.apple.com/support/certificates/) which costs $100/year (you get more than the cert, but the cert is what is needed here). While codesigning is not a requirement for an end user to install an application, it is a requirement to get apple to let you by [GateKeeper](https://support.apple.com/en-us/HT202491).
    Codesigning is the process of telling your end users that, you, or your organization is the one that developed the application. To codesign an application, you will need an [Apple Developer ID Application Certificate](https://developer.apple.com/support/certificates/) which costs $100/year (you get more than the cert with the subscription, but the cert is what is needed here). While codesigning is not a requirement for an end user to install an application, it is a requirement to get apple to let you by [GateKeeper](https://support.apple.com/en-us/HT202491).

    > Note: `fbs` does have some codesign support but at the time of this writing it is only for `.deb` packages on linux. For macOS we will have to codesign ourselves.
    > Note: `fbs` does have some codesign support but at the time of this writing it is only for `.deb` packages on linux. For macOS (and Windows) we will have to codesign ourselves.
    ## Shoulders of Giants

    There are a number of libraries that are used along the way. I wanted my application to be installable as a python binary wheel, and I wanted to deploy native installers for end users. The python library that has an easy to use API and fantastic documentation generate the installers is [fbs](https://build-system.fman.io/). `fbs` makes heavy use of `pyinstaller` under the hood but provides a nice, cross-platform compatable API, but it requires some specific directory strcture that is not intuitive, and takes a little work to make `setuptools` happy with it for creating the wheel.
    There are a number of libraries that are used along the way. I wanted my application to be installable as a python binary wheel, and I wanted to deploy native installers for end users. The python library that has an easy to use API and fantastic documentation generate the installers is [fbs](https://build-system.fman.io/). `fbs` makes heavy use of `pyinstaller` under the hood but provides a nice, cross-platform compatable API, but it requires some specific directory strcture that is not intuitive (which I will get to later).

    The work here could not be done without the hard work of maintainers and contributors of `setuptools`, `pyinstaller`, and `fbs` libraries. Furthermore, the `pyinstaller` [issue tracker](https://github.com/pyinstaller/pyinstaller/issues) has been a gold mine of information when I am getting unexplained behavior.

    I also want to point out the [Qt for Python developer gitter channel](https://gitter.im/PySide/pyside2) is a great place to ask for help if needed (these are the PySide2 devs, keep in mind they're pretty busy, I try to go here as a last resort). The folks I've interacted with there have been very helpful, and have verified bugs I have come across.
    I also want to point out the [Qt for Python developer gitter channel](https://gitter.im/PySide/pyside2) is a great place to ask for help if needed (these are the PySide2 devs, keep in mind they're pretty busy so don't expect a prompt reply). The folks I have interacted with there have been very helpful, and have verified bugs I have come across.

    ## Considerations As We Start

    > **TIP:** Pin your dependencies
    Pin your dependencies folks. `pyinstaller` is fragile, having pinned dependency versions lets you backtrack in your git history (bonus points if you use `git bisect`) to isolate what change in dependency suddenly broke the app bundle. I have had version changes in `numpy` (1.16.4 > 1.17.0) and `fbs` (0.8.2 > 0.8.3), that completely break the `pyinstaller` bundles.
    Pin your dependencies folks. `pyinstaller` is fragile (or I should say, the product `pyinstaller` creates is fragile), having pinned dependency versions lets you backtrack in your git history (bonus points if you use `git bisect`) to isolate what change in dependency suddenly broke your app bundle. I have had version changes in `numpy` (1.16.4 > 1.17.0) and `fbs` (0.8.2 > 0.8.3), that completely break the `pyinstaller` generated bundles.

    With pinned dependencies, you can go back in git history and recreate the environment at the time of the commit to a very precise level. You can pin dependencies via `pipenv` or making use of the output of `pip freeze` and `requirements.txt` files. Initially I gravitated toward `pipenv` but using the output of `pip freeze` is sufficient, however with `pipenv` I ran into my first gotcha, you have to be careful with _which_ python binary it grabs.
    With pinned dependencies, you can go back in git history and recreate the environment at the time of the commit to a very precise level. You can pin dependencies via `pipenv` or making use of the output of `pip freeze` and `requirements.txt` files. I currently use `pipenv` but using the output of `pip freeze` is sufficient. With `pipenv` I ran into my first gotcha, you have to be careful with _which_ python binary it grabs.

    > **GOTCHA:** PySide 5.12 (and maybe more recent versions) has an issue where if you are using a conda distributed version of python, your application will not respect macOS's dark mode setting (dark mode will not work for your app). This is problematic with `pipenv` as when `pipenv` creates a virtual environment, it searches through `$PATH` for the first python binary that matches the version requirements, if that version of python is a conda version, you have _silenty_ broken dark mode on your application. In short, if you use `pipenv` and have a `conda` distribution of python on your system, be very careful.
    > **GOTCHA:** PySide 5.12 (and maybe more recent versions) has an issue where if you are using a conda distributed version of python, your application will not respect macOS's dark mode setting (tl;dr no dark mode for your app). This is problematic with `pipenv` as when `pipenv` creates a virtual environment, it searches through `$PATH` for the first python binary that matches the version requirements, if that version of python comes from a conda distribution, you have _silenty_ broken dark mode on your application. In short, if you use `pipenv` and have a `conda` distribution of python on your system, be very careful.
    <!-- -->
    > **GOTCHA:** `pipenv` [does not handle OS specific dependencies well](https://github.com/pypa/pipenv/issues/1954) and you will have OS specific build dependencies. If you're going to use `pipenv` I urge you to be careful with dependencies you will need come build time (like `pypiwin32` for Windows, or `pytest-xvfb` for Linux), and store those dependencies outside of your `Pipfile`
    > **GOTCHA:** `pipenv` [does not handle OS specific dependencies well](https://github.com/pypa/pipenv/issues/1954) and you will have OS specific build dependencies. If you're going to use `pipenv` I urge you to be careful with dependencies you will need come build time (like `pypiwin32` for Windows), and store those dependencies outside of your `Pipfile`.
    ## Project Directory Structure

    `fbs` is very particular about the [directory structure](https://build-system.fman.io/manual/#directory-structure) of your project. Key here is to follow this structure, and we will then conform our `setup.py` to make it work. I would suggest having your project code inside the `src/main/python/my_app/` directory, as this makes the name-space a bit easier to work with. I will not repeat `fbs`'s documentation here, but I will suggest to users
    `fbs` is very particular about the [directory structure](https://build-system.fman.io/manual/#directory-structure) of your project. Key here is to follow this structure _precisely_, and we will make our `setup.py` work with it. I would suggest having your project code inside the `src/main/python/my_app/` directory, as this makes the name-space a bit easier to work with. I will not repeat the contents of `fbs`'s documentation here, just follow their recommended structure.

    > **TIP** Do not just pack all your requirements into `setup.py`'s `install_requires`, use a `requirements.txt` file. The `requirements.txt` file is the same as `./requirements/base.txt`, so you can just copy it over as part of your build process. The trick will be to keep _only_ runtime dependencies in your `requirements.txt` file, so you will need to put developer dependencies elsewhere (like `requirements-dev.txt` or `setup.py`'s `extra_requires`).
    @@ -66,7 +65,7 @@ The main reason I wanted to make my application into an installable package is t

    ## Steps Before Building the .app Bundle

    Before building a bundle, there are a few checks we want to do to ensure things run correctly.
    Before building the app bundle, there are a few checks we want to do to ensure things will run correctly.

    1. Ensure `fbs run` actually launches the application.

    @@ -105,9 +104,11 @@ Before building a bundle, there are a few checks we want to do to ensure things
    main()
    ```

    Also from the `fbs` documentation:

    > What you usually don't want to do is to refer to fbs from your application's implementation (src/main/). If at all, you should only refer to fbs from build scripts (build.py and/or src/build/python/).

    What this lets you do is you can have `fbs run` but we are not required to have `fbs` as part of runtime dependencies. That means, in addition to `fbs run` we want another way to launch the application, which can be of the form of of running `python src/main/python/my_app/__main__.py`, and having a corresponding file with something along the lines of
    What this lets you do is you can have `fbs run` launch your application, but we are not required to have `fbs` as part of runtime dependencies. The alternative being lauching your application via `setuptools` entry point, which can be of the form of of running `python src/main/python/my_app/__main__.py`, and having a corresponding file with something along the lines of

    ```python
    import os
    @@ -129,7 +130,7 @@ Before building a bundle, there are a few checks we want to do to ensure things

    2. Update `./src/build/settings/base.json`

    Here is the version I have:
    Here are the basic contents of `base.json` for my app:

    ```json
    {
    @@ -142,7 +143,7 @@ Before building a bundle, there are a few checks we want to do to ensure things
    }
    ```

    > **TIP:** The hidden imports field is the place you will add modules to that pyinstaller did not grab on its own as part of the bundling process. For my applications, I had to include `PySide2.QtMultimediaWidgets` when I added audio playback capability to my application.
    > **TIP:** The `hidden_imports` field is the place you will add modules to that pyinstaller did not grab on its own during the bundling process. For my applications, I had to include `PySide2.QtMultimediaWidgets` when I added audio playback capability to my application.

    3. Create an `Info.plist` file

    @@ -209,29 +210,31 @@ but if the `.app` bundle built, best thing to do is just attempt to open that an
    open ./target/MyApp.app`
    ```

    If this executes your Python app, congratulations you've done it, next step is to run
    If this executes your application, congratulations you've done it. The next step is to generate the `.dmg` installer, which can be done by running:
    ```bash
    fbs installer
    ```
    And this will produce a `MyApp.dmg` file that you can distribute to others.
    That command will produce a `MyApp.dmg` file that you can distribute to others.
    If you do not want to codesign your application, or you do not have an Apple Developer account (aka no access to a Developer ID Application certificate) then you can stop here!
    If you do not have an Apple Developer account/subscription, or if you just have no interest in codesigning your application, you can stop here.
    ## Codesigning and Notarizing
    If you want to codesign your application, that is definitely doable, but fair warning, you are proceeding into mostly uncharted territory. Apple's documentation is mostly aimed at Xcode projects, so little of it will be of help. That said, the issue tracker on pyinstaller has been very helpful.
    If you want to codesign your application, it can be done, but fair warning, you are proceeding into mostly uncharted territory. Apple's documentation written with Xcode projects in mind, so a lot of it will not be relevant. That said, the issue tracker on pyinstaller has been very helpful for debugging issues.

    ### Fix Gendrated .app Bundle

    Before codesigning your application, we actually need to "fix" the app bundle that pyinstaller generated. Here is a [link to the issue on the pyinstaller repo](https://github.com/pyinstaller/pyinstaller/issues/3680), and the associated [pyinstaller wiki entry](https://github.com/pyinstaller/pyinstaller/wiki/Recipe-OSX-Code-Signing-Qt). The issue tracker goes into what the issue is, but it fixes a couple of problems, one of which is macOS treating directories with periods in them as separate bundles. The tl;dr is that you should run the script against your `fbs` generates `.app bundle`.

    The modifications I made removed unneeded files that would not preserve their extended attributes during a move, and would then invalidate the codesigning.

    ### Create Application Entitlements (optional)

    One of the requirements for application notarization (which will be required starting macOS 10.15) is codesigning with a [hardened runtime](https://developer.apple.com/documentation/security/hardened_runtime_entitlements). There may be "entitlements" you want to enable your application to have (check the link above for what's available), but if you are bundling `numba` or anything else that wants to execute JIT-compiled code, you should consider adding the [Allow Execution of JIT-compiled Code Entitlement](https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_cs_allow-jit?language=objc).
    To add entitlements is fairly straight forward, in your project root folder create a file `.entitlements` and put the following contents in it (of course add entitlements you think are needed).
    To add entitlements is fairly straight forward, in your project root folder create a file `.entitlements`, and put the following contents in it (of course add entitlements you think are needed).
    ```xml
    <?xml version="1.0" encoding="UTF-8"?>
    @@ -262,9 +265,9 @@ Some explanation of the options
    ### Verifying the codesign operation
    > **GOTCHA:** Doing a verification of the codesign after you do a codesign sounds reasonable, right? *WRONG* queue evil laugh. Part of the codesign operation involves verifying the "extended attributes" of the files inside the .app bundle. When you copy/move the .app bundle, the extended attributes change, but if you never touch the file, the extended attributes will not be subject to changing, and a codesign verficiation step may falsely "pass" (don't ask me how long it took me to troubleshoot this).
    > **GOTCHA:** Doing a verification of the codesign after you do a codesign sounds reasonable, right? *WRONG* queue evil laugh. Part of the codesign operation involves verifying the "extended attributes" of the files inside the .app bundle. When you copy/move the .app bundle, the extended attributes change, but if you never touch the file, the extended attributes will remain fixed, and a codesign verficiation step may falsely "pass" (don't ask me how long it took me to troubleshoot this).

    Before verifying the `codesign` operation, we need to modify the extended attributes, this can easily be done by running
    To address this, before verifying the `codesign` operation, we need to modify the extended attributes, which can be done by running the following command.

    ```bash
    xattr -cr ./target/MyApp.app
    @@ -296,7 +299,7 @@ This should return
    source=Developer ID
    ```

    At this point, the application is codesigned, and we should generate our .dmg file.
    At this point, the application is codesigned, and we should generate our `.dmg` file.

    ```bash
    fbs installer
    @@ -310,9 +313,9 @@ hdiutil: internet-enable: enable succeeded
    Created target/MyApp.dmg.
    ```

    So now we have a `.app` bundle that is codesigned and a `.dmg` file thata is made from the codesigned bundle; next step, we need to [notarize the application](https://developer.apple.com/documentation/security/notarizing_your_app_before_distribution).
    So now we have a `.app` bundle that is codesigned and a `.dmg` file that is made from the codesigned bundle; next step, we need to [notarize the application](https://developer.apple.com/documentation/security/notarizing_your_app_before_distribution).

    ### Notarizing the App
    ### Notarizing your Application

    This is a case where the [Apple Documentation](https://developer.apple.com/documentation/security/notarizing_your_app_before_distribution/customizing_the_notarization_workflow) is actually quite helpful.

    @@ -332,6 +335,8 @@ Next, we need to ensure our Apple developer credentials, with a application-spec
    security add-generic-password -a $APPLE_DEVELOPER_EMAIL -w $PASSWORD -s "AC_PASSWORD"
    ```

    Once that is done, we can move on with actually performing the notarization.

    #### Do the Notarization

    To perform the notarization run the following command
  6. j9ac9k revised this gist Sep 1, 2019. 1 changed file with 18 additions and 2 deletions.
    20 changes: 18 additions & 2 deletions macOS_app_creation.md
    View file
    Original file line number Diff line number Diff line change
    @@ -38,6 +38,7 @@ With pinned dependencies, you can go back in git history and recreate the enviro

    > **GOTCHA:** PySide 5.12 (and maybe more recent versions) has an issue where if you are using a conda distributed version of python, your application will not respect macOS's dark mode setting (dark mode will not work for your app). This is problematic with `pipenv` as when `pipenv` creates a virtual environment, it searches through `$PATH` for the first python binary that matches the version requirements, if that version of python is a conda version, you have _silenty_ broken dark mode on your application. In short, if you use `pipenv` and have a `conda` distribution of python on your system, be very careful.
    <!-- -->
    > **GOTCHA:** `pipenv` [does not handle OS specific dependencies well](https://github.com/pypa/pipenv/issues/1954) and you will have OS specific build dependencies. If you're going to use `pipenv` I urge you to be careful with dependencies you will need come build time (like `pypiwin32` for Windows, or `pytest-xvfb` for Linux), and store those dependencies outside of your `Pipfile`
    ## Project Directory Structure
    @@ -284,7 +285,6 @@ That should return

    Feeling paranoid? I know I have been during this process, there is a second way we can verify the codesigning operation


    ```bash
    spctl --assess --verbose ./target/MyApp.app
    ```
    @@ -343,7 +343,7 @@ xcrun altool --notarize-app --primary-bundle-id "com.my_company.my_group.my_app"
    * for the argument to `--primary-bundle-id` place what you have in your `Info.plist` file earlier
    * replace `$APPLE_DEVELOPER_EMAIL` with your apple developer email address

    Once the upload succeeds, you should get an identifier back along the lines of
    Once the upload succeeds, you should get an identifier back along the lines of

    ```text
    RequestUUID = 2EFE2717-52EF-43A5-96DC-0797E4CA1041
    @@ -368,6 +368,22 @@ xcrun stapler staple "MyApp.dmg"

    ## Appendix

    ### Import Error During Runtime

    Periodically you get an import error during runtime. I got this when I tried to play audio through `QAudioOutput` configured device. Error was along the lines of an import error seeking `PySide2.QtMultimediaWidgets`.

    The solution to this issue, is to add that import to the list of `hidden_imports` in your `./src/build/settings/base.json` file.

    Similarly, when I noticed my .app bundle wouldn't run after I upgraded from numpy 1.16.4 to 1.17.0, I had to add some modules to the `hidden_imports` list to solve [the issue](https://github.com/numpy/numpy/issues/14163), specifically:
    * `numpy.random.common`
    * `numpy.random.bounded_integers`
    * `numpy.random.entropy`
    For the record, I do not think these needed to be explicitely defined prior as of numpy 1.17.1. Once those imports/modules were added, and I rebuilt my `.app` bundle, it ran successfully.
    That `hidden_imports` feature will often be the first place to look to patch things up, especially when you know it is an issue with part of a dependency that is missing.
    ### Building Custom PySide2 Wheels
    If you are using PySide 5.12.0, 5.12.1, 5.12.2, 5.12.3, 5.12.4, or 5.13.0, and plan to noterize your application, you will need to build your own wheels, as there is an [issue with the SDK linked](https://bugreports.qt.io/browse/PYSIDE-1066). Luckily for us, the Qt for Python folks have provided [instructions on building PySide from source](https://wiki.qt.io/Qt_for_Python/GettingStarted/MacOS).
  7. j9ac9k revised this gist Sep 1, 2019. 1 changed file with 42 additions and 3 deletions.
    45 changes: 42 additions & 3 deletions macOS_app_creation.md
    View file
    Original file line number Diff line number Diff line change
    @@ -370,12 +370,51 @@ xcrun stapler staple "MyApp.dmg"

    ### Building Custom PySide2 Wheels

    Still need to write this up
    If you are using PySide 5.12.0, 5.12.1, 5.12.2, 5.12.3, 5.12.4, or 5.13.0, and plan to noterize your application, you will need to build your own wheels, as there is an [issue with the SDK linked](https://bugreports.qt.io/browse/PYSIDE-1066). Luckily for us, the Qt for Python folks have provided [instructions on building PySide from source](https://wiki.qt.io/Qt_for_Python/GettingStarted/MacOS).

    I won't repeat the instructions on the wiki here, but the dependencies needed to build the wheels involve
    * `Xcode`
    * `Qt` (matching version) installed
    * Should contain `qmake` binary
    * CMake
    * `setuptools` and `wheel` python libraries
    The command to build the wheel is
    ```bash
    python setup.py bdist_wheel --qmake=/path/to/qmake --ignore-git --parallel=8 --standalone -macos-sysroot=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
    ```
    * `--qmake=` should contain the path to your qmake binary that should will be in your associated version of Qt you installed earlier as part of the dependencies
    * `--macos-sysroot=` should be the path to the `MacOSX10.X.sdk`, I put the path for it on my machine, but it may be different on yours, verify this is accurate.
    The build time on my MacBook Air was approximately 30 minutes. 3 wheels will be generated, `pyside2`, `shiboken2` and `shiboken2-generator`. When building your application, you can install the wheels by running
    ```bash
    pip install --no-index --find-links=path_to_wheel_directory pyside2
    ```
    This will install the pyside2 and shiboken2 wheels you just created instead of the ones on PyPI.
    ### Querying Git Commits for Versioning
    Don't do this at runtime...
    This was a "gotcha" that was an issue for almost a month before I figured out what happened. A coworker submitted a merge request that modifies the version of the application by appending the first 6 characters of the current commit hash to the version number.
    Perfectly sensible.
    What the issue was, that it did this at runtime, and when the applicationo is in bundle form, it's no longer in a git repository, thus querying the git status must generate an error.

    The frustarting part here is that the failure mode was when opening `MyApp.app`, the icon would bounce for a few seconds, and then just close out, with no meaningful error of any kind.

    The way I isolated this issue was via git bisect, and I tracked down the commit with the diff, which incorporated the feature I just described.

    ### Logging

    logging filehandler takes relative paths?
    While I am still working on debugging this, what I feel comfortable saying right now is that `logging.FileHandler('any_file.log')` will cause `MyApp.app` bundle to fail on launch silently (just like the previous git issue).

    My working theory here is that `logging.FileHandler()` takes only relative paths, not absolute paths, and it's attempting to create a log file in a write protected area of the `.app` bundle.
    After some googling, the main suggestion seems to be to [create your own logging handler](https://stackoverflow.com/questions/28655198/best-way-to-display-logs-in-pyqt). I have not implemented this myself yet, so I cannot speak to it, so for now the only advice I'll give you, is to not write logs via `FileHandler`.
    If any readers have a good solution to the logging issue with python app bundles, please let me know, I am all ears.
  8. j9ac9k revised this gist Aug 31, 2019. 1 changed file with 290 additions and 93 deletions.
    383 changes: 290 additions & 93 deletions macOS_app_creation.md
    View file
    Original file line number Diff line number Diff line change
    @@ -38,11 +38,13 @@ With pinned dependencies, you can go back in git history and recreate the enviro

    > **GOTCHA:** PySide 5.12 (and maybe more recent versions) has an issue where if you are using a conda distributed version of python, your application will not respect macOS's dark mode setting (dark mode will not work for your app). This is problematic with `pipenv` as when `pipenv` creates a virtual environment, it searches through `$PATH` for the first python binary that matches the version requirements, if that version of python is a conda version, you have _silenty_ broken dark mode on your application. In short, if you use `pipenv` and have a `conda` distribution of python on your system, be very careful.
    > **GOTCHA:** `pipenv` [does not handle OS specific dependencies well](https://github.com/pypa/pipenv/issues/1954) and you will have OS specific build dependencies. If you're going to use `pipenv` I urge you to be careful with dependencies you will need come build time (like `pypiwin32` for Windows, or `pytest-xvfb` for Linux), and store those dependencies outside of your `Pipfile`
    ## Project Directory Structure

    `fbs` is very particular about the [directory structure](https://build-system.fman.io/manual/#directory-structure) of your project. Key here is to follow this structure, and we will then conform our `setup.py` to make it work. I would suggest having your project code inside the `src/main/python/my_project/` directory, as this makes the name-space a bit easier to work with. I will not repeat `fbs`'s documentation here, but I will suggest to users
    `fbs` is very particular about the [directory structure](https://build-system.fman.io/manual/#directory-structure) of your project. Key here is to follow this structure, and we will then conform our `setup.py` to make it work. I would suggest having your project code inside the `src/main/python/my_app/` directory, as this makes the name-space a bit easier to work with. I will not repeat `fbs`'s documentation here, but I will suggest to users

    > **TIP** Do not just pack all your requirements into `setup.py`'s `install_requires`, use a `requirements.txt` file as that is the same as `./requirements/base.txt`, so you can just copy it over as part of your build process. The trick will be to keep _only_ runtime dependencies in your `requirements.txt` file, so you will need to put developer dependencies elsewhere (like `requirements-dev.txt` or `setup.py`'s `extra_requires`).
    > **TIP** Do not just pack all your requirements into `setup.py`'s `install_requires`, use a `requirements.txt` file. The `requirements.txt` file is the same as `./requirements/base.txt`, so you can just copy it over as part of your build process. The trick will be to keep _only_ runtime dependencies in your `requirements.txt` file, so you will need to put developer dependencies elsewhere (like `requirements-dev.txt` or `setup.py`'s `extra_requires`).
    To make `setup.py` conform to this directory structure we just need to add the following lines inside our `setup` call.

    @@ -61,124 +63,319 @@ setup(

    The main reason I wanted to make my application into an installable package is that makes things much easier come testing, I can import my modules via `import MyApp` instead of `from src.main.python import MyApp`.

    ## Pre-Bundle Build Steps
    ## Steps Before Building the .app Bundle

    Before building a bundle, there are a few checks we want to do to ensure things run correctly.

    1. Ensure `fbs run` actually launches the application.

    This has been somewhat of an issue for me as I want to keep `fbs` as a non-runtime dependency [due to it's GPLv3 license](https://github.com/mherrmann/fbs/blob/master/LICENSE), and from the [`fbs` manual](https://build-system.fman.io/manual/#api)
    This has been somewhat of an issue for me as I want to keep `fbs` as a non-runtime dependency [due to it's GPLv3 license](https://github.com/mherrmann/fbs/blob/master/LICENSE), and from the [`fbs` manual](https://build-system.fman.io/manual/#api)

    > When you use fbs, you typically add references to fbs_runtime to your source code. For example, the default main.py does this with ApplicationContext from this package.
    > When you use fbs, you typically add references to fbs_runtime to your source code. For example, the default main.py does this with ApplicationContext from this package.
    What I do here is I have a `src/main/python/main.py` file that includes the following command:
    What I do here is I have a `src/main/python/main.py` file that includes the following command:

    ```python
    import sys
    from fbs_runtime.application_context.PySide2 import (
    ApplicationContext,
    cached_property,
    )
    ```python
    import sys
    from fbs_runtime.application_context.PySide2 import (
    ApplicationContext,
    cached_property,
    )

    from my_app.App import MyApp
    from my_app.App import MyApp

    class AppContext(ApplicationContext):
    @cached_property
    def app(self) -> MyApp:
    app = MyApp()
    return app.qtapp
    class AppContext(ApplicationContext):
    @cached_property
    def app(self) -> MyApp:
    app = MyApp()
    return app.qtapp

    def run(self) -> int:
    self.app.showMainWindow()
    return self.app.exec_()
    def run(self) -> int:
    self.app.showMainWindow()
    return self.app.exec_()

    def main() -> None:
    appctxt = AppContext()
    exit_code = appctxt.run()
    sys.exit(exit_code)
    def main() -> None:
    appctxt = AppContext()
    exit_code = appctxt.run()
    sys.exit(exit_code)


    if __name__ == "__main__":
    main()
    ```
    if __name__ == "__main__":
    main()
    ```

    > What you usually don't want to do is to refer to fbs from your application's implementation (src/main/). If at all, you should only refer to fbs from build scripts (build.py and/or src/build/python/).
    > What you usually don't want to do is to refer to fbs from your application's implementation (src/main/). If at all, you should only refer to fbs from build scripts (build.py and/or src/build/python/).

    What this lets you do is you can have `fbs run` but we are not required to have `fbs` as part of runtime dependencies. That means, in addition to `fbs run` we want another way to launch the application, which can be of the form of of running `python src/main/python/my_app/__main__.py`, and having a corresponding file with something along the lines of
    What this lets you do is you can have `fbs run` but we are not required to have `fbs` as part of runtime dependencies. That means, in addition to `fbs run` we want another way to launch the application, which can be of the form of of running `python src/main/python/my_app/__main__.py`, and having a corresponding file with something along the lines of

    ```python
    import os
    import sys
    ```python
    import os
    import sys


    def main() -> None:
    os.environ["QT_API"] = "pyside2"
    from App import MyApp
    def main() -> None:
    os.environ["QT_API"] = "pyside2"
    from App import MyApp

    app = MyApp()
    app.display()
    sys.exit(app.qtapp.exec_())
    app = MyApp()
    app.display()
    sys.exit(app.qtapp.exec_())


    if __name__ == "__main__":
    main()
    ```
    if __name__ == "__main__":
    main()
    ```

    2. Update `./src/build/settings/base.json`

    Here is the version I have, I do want to highlight one import field, that is not placed there by default, the `hidden_imports` field.
    Here is the version I have:

    > **TIP:** The hidden imports field is the place you will add modules to that pyinstaller did not grab on its own as part of the bundling process. For my applications, I had to include `PySide2.QtMultimediaWidgets` when I added audio playback capability to my application.
    ```json
    {
    "app_name": "my_app_name",
    "author": "my_company_name",
    "version": "0.1.0",
    "main_module": "src/main/python/main.py",
    "public_settings": ["app_name", "author", "version"],
    "hidden_imports": []
    }
    ```

    ```json
    {
    "app_name": "my_app_name",
    "author": "my_company_name",
    "version": "0.1.0",
    "main_module": "src/main/python/main.py",
    "public_settings": ["app_name", "author", "version"],
    "hidden_imports": []
    }
    ```
    > **TIP:** The hidden imports field is the place you will add modules to that pyinstaller did not grab on its own as part of the bundling process. For my applications, I had to include `PySide2.QtMultimediaWidgets` when I added audio playback capability to my application.

    3. Create an `Info.plist` file

    ... and place it in `./src/freeze/mac/Contents/Info.plist`. The following is what I creaated, mostly getting tips from google. The `${}` fields are grabbed from the `build.json` file above under the `public_settings` key.

    ```xml
    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
    <dict>
    <key>CFBundleExecutable</key>
    <string>${app_name}</string>
    <key>CFBundleDisplayName</key>
    <string>${app_name}</string>
    <key>CFBundlePackageType</key>
    <string>APPL</string>
    <key>CFBundleInfoDictionaryVersion</key>
    <string>6.0</string>
    <key>CFBundleIconFile</key>
    <string>Icon.icns</string>
    <key>CFBundleIdentifier</key>
    <string>com.mycompany.mygroup.app_name</string>
    <key>CFBundleDevelopmentRegion</key>
    <string>English</string>
    <key>LSBackgroundOnly</key>
    <string>0</string>
    <key>CFBundleShortVersionString</key>
    <string>${version}</string>
    <key>CFBundleVersion</key>
    <string>${version}</string>
    <key>CFBundleName</key>
    <string>${app_name}</string>
    <!-- Enable Retina support on OS X: -->
    <key>NSPrincipalClass</key>
    <string>NSApplication</string>
    <key>NSRequiresAquaSystemAppearance</key>
    <string>NO</string>
    <key>NSHighResolutionCapable</key>
    <string>YES</string>
    </dict>
    </plist>
    ```
    ... and place it in `./src/freeze/mac/Contents/Info.plist`. The following is what I creaated, mostly getting tips from google. The `${}` fields are grabbed from the `build.json` file above under the `public_settings` key.

    ```xml
    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
    <dict>
    <key>CFBundleExecutable</key>
    <string>${app_name}</string>
    <key>CFBundleDisplayName</key>
    <string>${app_name}</string>
    <key>CFBundlePackageType</key>
    <string>APPL</string>
    <key>CFBundleInfoDictionaryVersion</key>
    <string>6.0</string>
    <key>CFBundleIconFile</key>
    <string>Icon.icns</string>
    <key>CFBundleIdentifier</key>
    <string>com.mycompany.mygroup.app_name</string>
    <key>CFBundleDevelopmentRegion</key>
    <string>English</string>
    <key>LSBackgroundOnly</key>
    <string>0</string>
    <key>CFBundleShortVersionString</key>
    <string>${version}</string>
    <key>CFBundleVersion</key>
    <string>${version}</string>
    <key>CFBundleName</key>
    <string>${app_name}</string>
    <!-- Enable Retina support on OS X: -->
    <key>NSPrincipalClass</key>
    <string>NSApplication</string>
    <key>NSRequiresAquaSystemAppearance</key>
    <string>NO</string>
    <key>NSHighResolutionCapable</key>
    <string>YES</string>
    </dict>
    </plist>
    ```

    With that done, you should be ready to move onto actually building the `.app` bundle!

    ## Making the Bundle and Installer

    The commands to run here are straight forward

    ```bash
    cp -f requirements.txt ./requirements/base.txt
    fbs freeze
    ```

    The first copy command is to ensure fbs will grab the relevant dependencies, the second command will generate the `./target/MyApp.app` bundle. This process may take a minute or so, and the terminal may output a bunch of errors/warnings along the lines of

    ```text
    ERROR: Can not find path ./libshiboken2.abi3.5.12.dylib (needed by ....)
    ```

    but if the `.app` bundle built, best thing to do is just attempt to open that and see what happens

    ```bash
    open ./target/MyApp.app`
    ```

    If this executes your Python app, congratulations you've done it, next step is to run
    ```bash
    fbs installer
    ```
    And this will produce a `MyApp.dmg` file that you can distribute to others.
    If you do not want to codesign your application, or you do not have an Apple Developer account (aka no access to a Developer ID Application certificate) then you can stop here!
    ## Codesigning and Notarizing
    If you want to codesign your application, that is definitely doable, but fair warning, you are proceeding into mostly uncharted territory. Apple's documentation is mostly aimed at Xcode projects, so little of it will be of help. That said, the issue tracker on pyinstaller has been very helpful.

    ### Fix Gendrated .app Bundle

    Before codesigning your application, we actually need to "fix" the app bundle that pyinstaller generated. Here is a [link to the issue on the pyinstaller repo](https://github.com/pyinstaller/pyinstaller/issues/3680), and the associated [pyinstaller wiki entry](https://github.com/pyinstaller/pyinstaller/wiki/Recipe-OSX-Code-Signing-Qt). The issue tracker goes into what the issue is, but it fixes a couple of problems, one of which is macOS treating directories with periods in them as separate bundles. The tl;dr is that you should run the script against your `fbs` generates `.app bundle`.

    ### Create Application Entitlements (optional)

    One of the requirements for application notarization (which will be required starting macOS 10.15) is codesigning with a [hardened runtime](https://developer.apple.com/documentation/security/hardened_runtime_entitlements). There may be "entitlements" you want to enable your application to have (check the link above for what's available), but if you are bundling `numba` or anything else that wants to execute JIT-compiled code, you should consider adding the [Allow Execution of JIT-compiled Code Entitlement](https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_cs_allow-jit?language=objc).
    To add entitlements is fairly straight forward, in your project root folder create a file `.entitlements` and put the following contents in it (of course add entitlements you think are needed).
    ```xml
    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
    <dict>
    <key>com.apple.security.cs.allow-jit</key>
    <true/>
    </dict>
    </plist>
    ```
    ### Finally To the Codesigning Part
    After running this step, you can now codesign your `.app` bundle!
    ```bash
    codesign -fs "$CODESIGN_ID" --deep --force --verbose -o runtime --entitlements .entitlements --preserve-metadata=identifier,entitlements,requirements,runtime --timestamp ./target/MyApp.app
    ```
    Some explanation of the options
    * `-o runtime` enables hardened runtime (required for app notarization)
    * `--entitlements .entitlements` passes the entitlements you created to your application
    * `--timestamp` enables secure timestamp option (also required for notarization)
    * `--deep` does a recursive codesign through the bundle. The Apple documentation recommends using this _only_ for troubleshooting, however the docuemntation is aimed toward Xcode projects, and not particularly relevant in our use case.
    * `"$CODESIGN_ID` is an env variable for my "Developer ID Application" certificate name that I have stored in my system keychain
    ### Verifying the codesign operation
    > **GOTCHA:** Doing a verification of the codesign after you do a codesign sounds reasonable, right? *WRONG* queue evil laugh. Part of the codesign operation involves verifying the "extended attributes" of the files inside the .app bundle. When you copy/move the .app bundle, the extended attributes change, but if you never touch the file, the extended attributes will not be subject to changing, and a codesign verficiation step may falsely "pass" (don't ask me how long it took me to troubleshoot this).

    Before verifying the `codesign` operation, we need to modify the extended attributes, this can easily be done by running

    ```bash
    xattr -cr ./target/MyApp.app
    ```

    Once that is run, then we can execute a codesign verification step.

    ```bash
    codesign --verify --verbose ./target/MyApp.app
    ```

    That should return

    ```text
    ./target/MyApp.app: valid on disk
    ./target/MyApp.app: satisfies its Designated Requirement
    ```

    Feeling paranoid? I know I have been during this process, there is a second way we can verify the codesigning operation


    ```bash
    spctl --assess --verbose ./target/MyApp.app
    ```

    This should return

    ```text
    ./target/MyApp.app: accepted
    source=Developer ID
    ```

    At this point, the application is codesigned, and we should generate our .dmg file.

    ```bash
    fbs installer
    ```

    That should return something along the lines of:

    ```text
    waited 1 seconds for .DS_STORE to be created.
    hdiutil: internet-enable: enable succeeded
    Created target/MyApp.dmg.
    ```

    So now we have a `.app` bundle that is codesigned and a `.dmg` file thata is made from the codesigned bundle; next step, we need to [notarize the application](https://developer.apple.com/documentation/security/notarizing_your_app_before_distribution).

    ### Notarizing the App

    This is a case where the [Apple Documentation](https://developer.apple.com/documentation/security/notarizing_your_app_before_distribution/customizing_the_notarization_workflow) is actually quite helpful.

    #### First Time Only Commands

    The following commands only need to be run once on a system.

    First, we need to select the appropriate Xcode.app

    ```bash
    sudo xcode-select -s /path/to/Xcode10.app
    ```

    Next, we need to ensure our Apple developer credentials, with a application-specific password are set in our keychain.

    ```bash
    security add-generic-password -a $APPLE_DEVELOPER_EMAIL -w $PASSWORD -s "AC_PASSWORD"
    ```

    #### Do the Notarization

    To perform the notarization run the following command

    ```bash
    xcrun altool --notarize-app --primary-bundle-id "com.my_company.my_group.my_app" --username $APPLE_DEVELOPER_EMAIL --password "@keychain:AC_PASSWORD" --file ./target/MyApp.dmg
    ```

    * for the argument to `--primary-bundle-id` place what you have in your `Info.plist` file earlier
    * replace `$APPLE_DEVELOPER_EMAIL` with your apple developer email address

    Once the upload succeeds, you should get an identifier back along the lines of

    ```text
    RequestUUID = 2EFE2717-52EF-43A5-96DC-0797E4CA1041
    ```

    You can query the status of the notarization by running the following command (there is a modification to the command you can can run by the `RequestUUID` parameter you got earlier).

    ```bash
    xcrun altool --notarization-history 0 -u $APPLE_DEVELOPER_EMAIL -p "@keychain:AC_PASSWORD"
    ```

    If there is an error, it will return a URL you can look at and see more specific issues.

    > **GOTCHA:** PySide 5.12.0, 5.12.1, 5.12.2, 5.12.3, 5.12.4 and 5.13.0 on PyPi had some of their binaries build linking to an incompatable SDK. They are fully functional, but if you use the wheels them, notarization will fail. I reported the issue to their [bug-tracker here](https://bugreports.qt.io/browse/PYSIDE-1066) and they have resolved it in time for the next release (5.12.5 and 5.13.1). If you are using any of these libraries you will need to build the pyside2 wheels yourself, and when installing dependencies of your app you will need to install those custom built wheels, not the ones from PyPI. See the Appendix for instructions

    If it the query status says the package is accepted, you can "staple" the notarization with the following command

    ```bash
    xcrun stapler staple "MyApp.app"
    xcrun stapler staple "MyApp.dmg"
    ```

    ## Appendix

    ### Building Custom PySide2 Wheels

    Still need to write this up

    ### Querying Git Commits for Versioning

    Don't do this at runtime...
    ### Logging
    logging filehandler takes relative paths?
  9. j9ac9k created this gist Aug 31, 2019.
    184 changes: 184 additions & 0 deletions macOS_app_creation.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,184 @@
    # Building PyQt5/PySide2 macOS Application

    ## Preface

    As part of my professional duties, I have been tasked with developing a desktop application to analyze audio files, and make use of data generated by other internal tools. This desktop application would need to be cross platform compatable (Windows, macOS and Ubuntu), and the end users may or may not have much in the way of dependencies installed. I naturally turned to Qt (and more specifically, Qt for Python) due to my past experience with it, and the ease that you can create desktop applications with it. One of the requirements was to deploy this application to users machines that may or may not have developer dependencies (such as `python` itself) already on there. This made things more complicated; so I decided it may be worth trying to make native installers for each operating system (`deb` packages for linux, `.app` bundles and `.dmg` files for macOS, and `setup.exe` files for Windows). What I had no idea on, how do I build a PyQt5/PySide2 application, and bundle it inside a native installers.

    In short, here were the objectives I aimed to have

    * Cross platform compatable codebase
    * Deployment to users via native installers (no assumption of python being on the target machine)
    * Installable with binary wheel

    This post details the steps I took to create the installer and some issues I came across the way. I will be focusing on the macOS component as I did some extra steps such as code-signing my application, as well as notarizing it, which I will detail how to do so as well.

    What this blog post will not contain is instructions on building and running a Qt application in Python in general. To get a sample minimum working example after you install `fbs` you can run `fbs startproject`.

    ### Code Signing - What Is It

    Codesigning is the process of telling your end users that, you, or your organization is the one that developed the application. To codesign an application, you will need an [Apple Developer ID Application Certificate](https://developer.apple.com/support/certificates/) which costs $100/year (you get more than the cert, but the cert is what is needed here). While codesigning is not a requirement for an end user to install an application, it is a requirement to get apple to let you by [GateKeeper](https://support.apple.com/en-us/HT202491).

    > Note: `fbs` does have some codesign support but at the time of this writing it is only for `.deb` packages on linux. For macOS we will have to codesign ourselves.
    ## Shoulders of Giants

    There are a number of libraries that are used along the way. I wanted my application to be installable as a python binary wheel, and I wanted to deploy native installers for end users. The python library that has an easy to use API and fantastic documentation generate the installers is [fbs](https://build-system.fman.io/). `fbs` makes heavy use of `pyinstaller` under the hood but provides a nice, cross-platform compatable API, but it requires some specific directory strcture that is not intuitive, and takes a little work to make `setuptools` happy with it for creating the wheel.

    The work here could not be done without the hard work of maintainers and contributors of `setuptools`, `pyinstaller`, and `fbs` libraries. Furthermore, the `pyinstaller` [issue tracker](https://github.com/pyinstaller/pyinstaller/issues) has been a gold mine of information when I am getting unexplained behavior.

    I also want to point out the [Qt for Python developer gitter channel](https://gitter.im/PySide/pyside2) is a great place to ask for help if needed (these are the PySide2 devs, keep in mind they're pretty busy, I try to go here as a last resort). The folks I've interacted with there have been very helpful, and have verified bugs I have come across.

    ## Considerations As We Start

    > **TIP:** Pin your dependencies
    Pin your dependencies folks. `pyinstaller` is fragile, having pinned dependency versions lets you backtrack in your git history (bonus points if you use `git bisect`) to isolate what change in dependency suddenly broke the app bundle. I have had version changes in `numpy` (1.16.4 > 1.17.0) and `fbs` (0.8.2 > 0.8.3), that completely break the `pyinstaller` bundles.

    With pinned dependencies, you can go back in git history and recreate the environment at the time of the commit to a very precise level. You can pin dependencies via `pipenv` or making use of the output of `pip freeze` and `requirements.txt` files. Initially I gravitated toward `pipenv` but using the output of `pip freeze` is sufficient, however with `pipenv` I ran into my first gotcha, you have to be careful with _which_ python binary it grabs.

    > **GOTCHA:** PySide 5.12 (and maybe more recent versions) has an issue where if you are using a conda distributed version of python, your application will not respect macOS's dark mode setting (dark mode will not work for your app). This is problematic with `pipenv` as when `pipenv` creates a virtual environment, it searches through `$PATH` for the first python binary that matches the version requirements, if that version of python is a conda version, you have _silenty_ broken dark mode on your application. In short, if you use `pipenv` and have a `conda` distribution of python on your system, be very careful.
    ## Project Directory Structure

    `fbs` is very particular about the [directory structure](https://build-system.fman.io/manual/#directory-structure) of your project. Key here is to follow this structure, and we will then conform our `setup.py` to make it work. I would suggest having your project code inside the `src/main/python/my_project/` directory, as this makes the name-space a bit easier to work with. I will not repeat `fbs`'s documentation here, but I will suggest to users

    > **TIP** Do not just pack all your requirements into `setup.py`'s `install_requires`, use a `requirements.txt` file as that is the same as `./requirements/base.txt`, so you can just copy it over as part of your build process. The trick will be to keep _only_ runtime dependencies in your `requirements.txt` file, so you will need to put developer dependencies elsewhere (like `requirements-dev.txt` or `setup.py`'s `extra_requires`).
    To make `setup.py` conform to this directory structure we just need to add the following lines inside our `setup` call.

    ```python
    from setuptools import find_packages, setup

    ...

    setup(
    ...
    packages=find_packages("src/main/python"),
    package_dir={"": "src/main/python"},
    ...
    )
    ```

    The main reason I wanted to make my application into an installable package is that makes things much easier come testing, I can import my modules via `import MyApp` instead of `from src.main.python import MyApp`.

    ## Pre-Bundle Build Steps

    Before building a bundle, there are a few checks we want to do to ensure things run correctly.

    1. Ensure `fbs run` actually launches the application.

    This has been somewhat of an issue for me as I want to keep `fbs` as a non-runtime dependency [due to it's GPLv3 license](https://github.com/mherrmann/fbs/blob/master/LICENSE), and from the [`fbs` manual](https://build-system.fman.io/manual/#api)

    > When you use fbs, you typically add references to fbs_runtime to your source code. For example, the default main.py does this with ApplicationContext from this package.
    What I do here is I have a `src/main/python/main.py` file that includes the following command:

    ```python
    import sys
    from fbs_runtime.application_context.PySide2 import (
    ApplicationContext,
    cached_property,
    )

    from my_app.App import MyApp

    class AppContext(ApplicationContext):
    @cached_property
    def app(self) -> MyApp:
    app = MyApp()
    return app.qtapp

    def run(self) -> int:
    self.app.showMainWindow()
    return self.app.exec_()

    def main() -> None:
    appctxt = AppContext()
    exit_code = appctxt.run()
    sys.exit(exit_code)


    if __name__ == "__main__":
    main()
    ```

    > What you usually don't want to do is to refer to fbs from your application's implementation (src/main/). If at all, you should only refer to fbs from build scripts (build.py and/or src/build/python/).
    What this lets you do is you can have `fbs run` but we are not required to have `fbs` as part of runtime dependencies. That means, in addition to `fbs run` we want another way to launch the application, which can be of the form of of running `python src/main/python/my_app/__main__.py`, and having a corresponding file with something along the lines of

    ```python
    import os
    import sys


    def main() -> None:
    os.environ["QT_API"] = "pyside2"
    from App import MyApp

    app = MyApp()
    app.display()
    sys.exit(app.qtapp.exec_())


    if __name__ == "__main__":
    main()
    ```

    2. Update `./src/build/settings/base.json`

    Here is the version I have, I do want to highlight one import field, that is not placed there by default, the `hidden_imports` field.

    > **TIP:** The hidden imports field is the place you will add modules to that pyinstaller did not grab on its own as part of the bundling process. For my applications, I had to include `PySide2.QtMultimediaWidgets` when I added audio playback capability to my application.
    ```json
    {
    "app_name": "my_app_name",
    "author": "my_company_name",
    "version": "0.1.0",
    "main_module": "src/main/python/main.py",
    "public_settings": ["app_name", "author", "version"],
    "hidden_imports": []
    }
    ```

    3. Create an `Info.plist` file

    ... and place it in `./src/freeze/mac/Contents/Info.plist`. The following is what I creaated, mostly getting tips from google. The `${}` fields are grabbed from the `build.json` file above under the `public_settings` key.

    ```xml
    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
    <dict>
    <key>CFBundleExecutable</key>
    <string>${app_name}</string>
    <key>CFBundleDisplayName</key>
    <string>${app_name}</string>
    <key>CFBundlePackageType</key>
    <string>APPL</string>
    <key>CFBundleInfoDictionaryVersion</key>
    <string>6.0</string>
    <key>CFBundleIconFile</key>
    <string>Icon.icns</string>
    <key>CFBundleIdentifier</key>
    <string>com.mycompany.mygroup.app_name</string>
    <key>CFBundleDevelopmentRegion</key>
    <string>English</string>
    <key>LSBackgroundOnly</key>
    <string>0</string>
    <key>CFBundleShortVersionString</key>
    <string>${version}</string>
    <key>CFBundleVersion</key>
    <string>${version}</string>
    <key>CFBundleName</key>
    <string>${app_name}</string>
    <!-- Enable Retina support on OS X: -->
    <key>NSPrincipalClass</key>
    <string>NSApplication</string>
    <key>NSRequiresAquaSystemAppearance</key>
    <string>NO</string>
    <key>NSHighResolutionCapable</key>
    <string>YES</string>
    </dict>
    </plist>
    ```