rsms/macos-distribution.md

## macos-distribution.md

      
    Raw
  

              macos-distribution.md
            
          
    Hurdles of macOS distribution


Code signing
Notarization
Quarantine
Distribution vehicles
Notes

Code signing

Executable code must be cryptographically signed on macOS. The bare minimum required is "ad-hoc codesigning" which is essentially just a checksum of the executable. Clang will do this automatically when compiling for macOS.
Ad-hoc code signing

Ad-hoc signed code only works (without user intervention) on the local machine that built it. If you copy a ad-hoc signed executable to another computer, macOS will kill it before it even launches when run in a terminal. An ad-hoc signed executable needs to be interactively opened via Finder, once. If the user just tries to launch an ad-hoc signed app, an error dialog is displayed:
“Playbit.app” cannot be opened because
the developer cannot be verified.
macOS cannot verify that this app is free
from malware.
                 [Move to Trash] [Cancel]

However if the user right-clicks (or ctrl-clicks) the app in Finder and select "Open", the same dialog is shown but this time it includes an "Open" option:
macOS cannot verify the developer of “Playbit.app”.
Are you sure you want to open it?
By opening this app, you will be overriding system
security which can expose your computer and
personal information to malware that may harm your
Mac or compromise your privacy.
                 [Move to Trash] [Open] [Cancel]

For this to be possible, the user must have the setting "Allow apps downloaded from" in "System preferences" → "Security & Privacy" set to the option "App Store and identified developers"
"Developer ID Application" code signing

By paying Apple $99 a year they give you a certificate recognized by their certificate authority which allows macOS to know that the signature is from a known developer who has accepted a legally-binding agreement with Apple and can therefore be somewhat trusted.
Signing executables this way is significantly more complicated than ad-hoc signing. First, you have to create an Apple Developer account and do everything that goes with it (and pray that they don't get some of your personal details wrong in their database or you'll need another email address to open another developer account.) You also need to acquire your "Developer ID Application" certificate. It's not as simple as just a file; there are a few ways to get your cert. The easiest way requires you to 1) use a mac, and 2) install Xcode. Once you've installed Xcode and signed in to your developer account inside it, go to "Preferences" → "Account" → "Apple ID" → "Manage Certificates..." → "+" → "Developer ID Application". Next, open "Keychain Access.app" and search for "Developer ID Application". If you need to export a file with your certificate, you can do that here.
Next we need to find the identifier of the certificate, which we'll need during the next step.
$ security find-identity -v -p codesigning
1) AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
   "Developer ID Application: Your Name (XXXXXXXXXX)"
2) BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
   "Apple Development: Your Name (YYYYYYYYYY)"
3) CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
   "Developer ID Application: Your Name (ZZZZZZZZZZ)"
3 valid identities found

Make a note of the "Developer ID Application" ID (i.e. "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" in the example above.)
If you have more than one, as in the above example, pick any.
(It doesn't seem to matter; I've successfully notarized an app signed with both a cert from many years ago and one brand new one.)
Next step is to use the codesign command-line program. Codesigning modifies the thing you codesign, so when used in a build system like make you will need to take special measures to track mtime. For example by copying the unsigned app, codesigning the copy, checking for failure (and remove the signed app if so.)
Trivial code signing example

Here's an example shell session showing ad-hoc signature
and "Developer ID Application" signature:
$ clang hello.c -o hello
$ # Display signature: we expect it to be ad-hoc signed by clang/ld:
$ codesign -vv --display hello 2>&1 | grep -E '(Authority|Signature)='
Signature=adhoc
$ # Sign with your "Developer ID Application" signature:
$ codesign -s AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA hello
$ codesign -vv --display hello 2>&1 | grep -E '(Authority|Signature)='
Authority=Developer ID Application: Your Name (XXXXXXXXXX)
Authority=Developer ID Certification Authority
Authority=Apple Root CA
We can also verify the signature, but this is only useful for testing basic integrity (like comparing the hash of the file contents to an expected checksum.) codesign --verify does not care if the signature is ad-hoc or not.
Signing .app bundles

Signing a "bundle" is a little more complicated and it's easy to make a mistake here that becomes very hard to debug (Apple's verification tools will essentially just tell you "fail" and then you go figure out what is wrong.)
A few caveats:

Don't use --deep. Although codesign's manual page says
"--deep  When signing a bundle, specifies that nested code content such as helpers,
frameworks, and plug-ins, should be recursively signed in turn," it seem to not work
well in practice. Additionally, --deep applies only to executable (mach-o) files, not
other "resources", which are included in the signature regardless of this option.
Codesign "bottom up" in the file hierarchy.
I.e. codesign X.app/Contents/MacOS/X and then codesign X.app, no the other way around.
This applies to the entire file tree, including "frameworks" and other embedded bundles.

Example:
codesign -f -s $ID -o runtime --entitlements helper-entitlements.plist \
  X.app/Contents/MacOS/Helper
codesign -f -s $ID -o runtime \
  X.app/Contents/Frameworks/Sparkle.framework/Versions/B/XPCServices/Installer.xpc
codesign -f -s $ID -o runtime --preserve-metadata=entitlements \
  X.app/Contents/Frameworks/Sparkle.framework/Versions/B/XPCServices/Downloader.xpc
codesign -f -s $ID -o runtime \
  X.app/Contents/Frameworks/Sparkle.framework/Versions/B/Autoupdate
codesign -f -s $ID -o runtime \
  X.app/Contents/Frameworks/Sparkle.framework/Versions/B/Updater.app
codesign -f -s $ID -o runtime \
  X.app/Contents/Frameworks/Sparkle.framework
codesign -f -s $ID -o runtime X.app/Contents/MacOS/X
codesign -f -s $ID -o runtime X.app
Notarization

Even "Developer ID Application" signed programs will not "just work" when distributed to other people's macOS. You also need to "notarize" a program, which essentially means that you send a copy of a signed program to Apple. Their servers then analyze your program, verify its signature, that you developer account is valid, and that a number of other condition are met. Once "approved", you can download a cryptographic "stamp" from Apple's server which can then be "stapled" to (i.e. embedded in) your program.
At this point you can send a copy of that program to another macOS computer and the user will be able to run your program, assuming they have the setting "Allow apps downloaded from" in "System preferences" → "Security & Privacy" set to the option "App Store and identified developers" (the default for regular macOS installations.)
However, depending on how you distribute your program it may be subject to quarantine.
Notarizing an app requires the following conditions:

An active Apple developer subscription
The app be codesigned with a "Developer ID Application"
The app be codesigned with the "Hardened Runtime" option (codesign -o runtime)
An app-specific password linked to the same developer account that the "Developer ID Application" certificate is owned by.

Using notarytool

A few things to note about the notarytool:

Not in PATH; you need to use xcrun or use its absolute path.
Exits with "OK" (status 0) on failure (lol what), so you need to look at the output
to determine if an error occurred.
Writes its informational output to stderr, not stdout, so if you want to capture the
output, you may want to redirect stderr to stdout (2>&1).
Will fail to notarize some zip files, depending on how they were created.
For example, a zip created with zip -qr Playbit.app.zip Playbit.app will cause
notarization to fail with "The signature of the binary is invalid", but
a zip created with ditto -c -k --keepParent Playbit.app Playbit.app.zip will succeed.

First thing we need to do is to setup an
app-specific password in macOS Keychain.
Go to https://appleid.apple.com/account/manage and setup an app-specific password.
Next, create an entry in macOS Keychain:
$ xcrun notarytool store-credentials playbit-notarytool-password \
  --team-id XXXXXXXXXX \
  --apple-id "YOUR_APPLE_ID_EMAIL" \
  --password "APP_SPECIFIC_PASSWORD"
Now we can notarize stuff. Here's an example:
$ ditto -c -k --keepParent Playbit.app Playbit.app.zip
$ xcrun notarytool submit Playbit.app.zip \
  --keychain-profile playbit-notarytool-password \
  --wait
Conducting pre-submission checks for Playbit.app.zip and
initiating connection to the Apple notary service...
Submission ID received
  id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Successfully uploaded file8 MB of 8 MB)
  id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
  path: /Users/rsms/Playbit.app.zip
Waiting for processing to complete.
Current status: Accepted...........
Processing complete
  id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
  status: Accepted
Make a note of the "id:" (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx);
we'll need it for the next step.
Annoyingly, notarytool won't simply show you errors and other details.
If notarization fails (or if you are just curious about the result), you need to fetch
the logs with notarytool log ID. (See example below)
Finally, once notarytool submit succeeded, we need to "staple a ticket" to our app:
$ xcrun stapler staple Playbit.app
Important: If we want to distribute the app in a zip archive, remember to create a new zip file with the "stapled" app; the zip file we submitted to Apple for notarization is not stapled.
You can test a file's notarization status
using Apple's spctl ("SecAssessment system policy security") tool:
$ spctl -a -vvv -t install Playbit.app
Playbit.app: accepted
source=Notarized Developer ID
origin=Developer ID Application: Your Name (XXXXXXXXXX)
Example of fetching the notarization log:
$ xcrun notarytool log xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
  --keychain-profile playbit-notarytool-password \
  notarytool-log.json
$ cat notarytool-log.json
{ "logFormatVersion": 1,
  "jobId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "status": "Invalid",
  ...
  "issues": [
    { "severity": "error",
      "path": "Playbit.app.zip/Playbit.app/Contents/MacOS/Playbit",
      "message": "The signature of the binary is invalid.",
      "docUrl": "https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution/resolving_common_notarization_issues#3087735",
      "architecture": "arm64"
    }
  ]
}
Hardened runtime

"Hardened runtime" is a requirement for notarization
and is enabled during code signing (codesign -o runtime).

The Hardened Runtime, along with System Integrity Protection (SIP), protects the runtime integrity of your software by preventing certain classes of exploits, like code injection, dynamically linked library (DLL) hijacking, and process memory space tampering.
The Hardened Runtime doesn’t affect the operation of most apps, but it does disallow certain less common capabilities, like just-in-time (JIT) compilation. If your app relies on a capability that the Hardened Runtime restricts, add an entitlement to disable an individual protection.
https://developer.apple.com/documentation/security/hardened_runtime

As mentioned, some capabilities are restricted for "hardened runtime" programs.
To make an exception we need to provide the codesign tool with an "entitlements" file.
For example, we need the following exceptions for our virtual machine program:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>com.apple.security.hypervisor</key><true/>
  <key>com.apple.security.virtualization</key><true/>
  <key>com.apple.security.cs.allow-jit</key><true/>
  <key>com.apple.security.device.audio-input</key><true/>
</dict>
</plist>
Then pass this file to codesign which will embed these "entitlements" and include them in the cryptographic signature: codesign --entitlements entitlements.xml
Quarantine

Files downloaded from the scary internet are assigned an xattr com.apple.quarantine.
When a quarantined zip archive is unarchived, all files inside it are assigned this xattr as well.
Trying to launch an app with xattr com.apple.quarantine causes macOS to show an interstitial dialog saying
Playbit.app is an app downloaded from the Internet.
Are you sure you want to open it?
                                    [Cancel] [Open]

There's no way to avoid this with a zip file.
Note that even though codesign -s ID app.zip seems to be working, it does not.
macOS 13+


"macOS 13 has changed the process of app quarantine, with the addition of a new extended attribute to track the origin of quarantined apps in their provenance."
https://eclecticlight.co/2023/03/13/ventura-has-changed-app-quarantine-with-a-new-xattr/

Distribution vehicles

Mac apps are usually distributed in one of the following vehicles:

Installer (.pkg)
Archive (.zip, .tar, .xip)
Disk image (.dmg)

Installer as a distribution vehicle

Installers are the scariest to users, so this is the least popular vehicle as it's least likely that a user will follow through with the installation. Also, issues may arise during installation (like a failing pre- or post script.)
Archive as a distribution vehicle

Zip archives seems to be about as popular a vehicle as DMGs these days. Zip archives used to be more popular, but since they are unconditionally subject to quarantine they come with "tripping hazards" for non-advanced users; i.e. scary dialogs saying the app might be dangerous deter some people from even opening it.
“Playbit.app” is an app downloaded from the Internet
Are you sure you want to open it?
Chrome downloaded this file today at 10:55. Apple checked
it for malicious software and none was detected.
                                          [Cancel] [Open]

Additionally, since a quarantined application (even a code-signed, notarized one) runs in a sandbox when the user says "Open", your app may or may not crash or behave unpredictably.
For example, say that the first time your app launches you ask the user to answer some questions and some make choices, like signing in. You write that state to ~/Library/Application Support so that your users don't have to go through the same questions every time they launch your app. Now, this data is written to a "translocation" sandbox, a temporary location (like a virtual machine), not to the actual ~/Library/Application Support directory. If the user now installs (copies) the app to /Applications and runs it a second time, it will be like they never ran it before. That's an awkward user experience.
To deal with this scenario of translocation (or when running apps from disk images; see below), you can test for this scenario when your app launches and help the user install the application (or just let them know that they are doing a weird thing.)
if ([NSBundle.mainBundle.bundlePath rangeOfString:@"/AppTranslocation/"].location != NSNotFound) {
  NSLog(@"Running in translocation sandbox");
  NSAlert* alert = [NSAlert new];
  [alert setMessageText:@"Quarantined application"];
  [alert setInformativeText:
    @"This application may not behave correctly."
    " Move or copy it to your Applications directory"
    " and launch it from there."];
  [alert addButtonWithTitle:@"Open Anyways"];
  [alert addButtonWithTitle:@"Quit"];
  if (![NSApp isActive])
    [NSApp activateIgnoringOtherApps:YES];
  if ([alert runModal] != NSAlertFirstButtonReturn)
    exit(0);
}
See github.com/potionfactory/LetsMove
for a more comprehensive solution.
If you just want to know if a file has the quarantine flag set (which triggers macOS to launch an app under translocation), you can read the xattr with xattr -l X.app or check for it programmatically:
#import <sys/xattr.h>
bool is_file_quarantined(const char* filename) {
  return getxattr(filename, "com.apple.quarantine", NULL, 0, 0, 0) >= 0;
}
Disk image as a distribution vehicle

DMG disk images are a little awkward to create and they require the user to take one additional step during installation: to manually copy your app to a persistent directory like /Applications. This can be aided with an alias to the Applications directory in the disk image, reducing the ask from the user to just a drag-and-drop operation.
Two upsides of disk images:

They can be code signed
Avoids quarantine "scary file" dialog in the common case,
as a result from the user having to copy the app;
the first time the app launches outside the disk image it is not quarantined.

A downside with disk images is that users can run your app "inside" the disk image.
Even code-signed & notarized disk images are subject to quarantine (but not translocation.)
This means the users will see an interstitial warning dialog when trying to open your app inside a disk image:
“Playbit.app” is an app downloaded from the Internet.
Are you sure you want to open it?
This item is on the disk image “Playbit.dmg”. Chrome
downloaded this disk image today at 10:46. Apple checked
it for malicious software and none was detected.
                       [Cancel] [Show Disk Image] [Open]

Note that the default option is "Open" which makes it far more likely that a user opens an app in this scenario compared to the Archive vehicle.
Running an app inside a (signed & notarized) disk image is acceptable for most apps as it will be running with full privileges, i.e. it is not sandboxed/translocated.
However, you may want to consider the "marketing" or "business" downside of promoting this way of running your app since disk images are unmounted on reboot and not subject to things like backup or Spotlight etc.
A common pattern is—at launch—to check if the app is running from a disk image and offer the user to install it into /Applications for them. github.com/potionfactory/LetsMove is a copy-paste-able solution.
Notes

Signing & notarizing outside of macOS

We build the Playbit macOS app hermetically on Linux since our entire toolchain is managed.
However code signing currently requires using macOS.
These examples use Apple's official tools which only runs on macOS (and only on very recent versions of macOS).
github.com/indygreg/apple-platform-rs is an alternative open-source set of tools with the promise of enabling code signing and notarization on non-macOS platforms. However I have not been able to make these tools work in practice, in particular rcodesign notary-submit is failing for me with the same inputs as those that succeed with Apple's notarytool submit.
Another problem with apple-platform-rs is that it is written in Rust; getting a rust compiler on a non-standard Linux system like Playbit is currently Very Complicated™.