Skip to content

Instantly share code, notes, and snippets.

@talkingmoose
Last active June 26, 2023 05:37
Show Gist options
  • Star 58 You must be signed in to star a gist
  • Fork 9 You must be signed in to fork a gist
  • Save talkingmoose/e9ed319226c6da30dd633725e48a97b0 to your computer and use it in GitHub Desktop.
Save talkingmoose/e9ed319226c6da30dd633725e48a97b0 to your computer and use it in GitHub Desktop.
The pkgbuild binary and Jamf Composer don't support adding single files of 8 GB or more to a package. Some apps like Install macOS Big Sur.app include files larger than 8 GB in their bundles. This script will create a deployable PKG file from apps whose bundles include those large files.
#!/bin/zsh
:<<ABOUT_THIS_SCRIPT
-------------------------------------------------------------------------------
Written by:William Smith
Professional Services Engineer
Jamf
bill@talkingmoose.net
https://gist.github.com/e9ed319226c6da30dd633725e48a97b0
Originally posted: October 15, 2020
Updated: December 2, 2020
Updated: December 11, 2020
Purpose: Creates a deployable PKG file from an app whose app bundle
contains one or more files larger than the 8 GB limit supported by pkgbuild
or Jamf Composer.
Instructions:
1. Save the script to a file named MegaPKGr.zsh on your Mac.
2. If necessary, run 'chmod +x /path/to/MegaPKGr.zsh' to make it execu-
table.
3. To run the script, open Terminal and enter:
/path/to/MegaPKGr.zsh /path/to/large.app
Except where otherwise noted, this work is licensed under
http://creativecommons.org/licenses/by/4.0/
"If you try to save wisdom until the world is wise, Father,
the world will never have it."
-------------------------------------------------------------------------------
ABOUT_THIS_SCRIPT
function logmessage() {
echo -ne "$1\r"
/bin/sleep 1
}
function logresult() {
if [ $? = 0 ] ; then
echo "$1"
/bin/date "+%Y-%m-%d %H:%M:%S $1" >> "$logFile"
/bin/sleep 1
else
echo "$2"
/bin/date "+%Y-%m-%d %H:%M:%S $2" >> "$logFile"
echo "Aborting script"
/bin/date "+%Y-%m-%d %H:%M:%S Aborting script" >> "$logFile"
cleanup
exit 1
fi
}
function cleanup() {
logmessage "Removing temporary items..."
/bin/rm -Rf "$tempDirectory"
logresult "Succeeded removing temporary items" "Failed removing temporary items"
}
# need to run this script with elevated privileges
if [ $EUID -ne 0 ]; then
echo
echo "This script requires elevated privileges. Run it again with \"sudo\"."
echo
exit 0
fi
# define log file location
logFile="/Library/Logs/MegaPKGr.log"
# choose app for packaging
appPath="$1"
while [[ "$appPath" = "" ]]
do
echo
echo "Drag the app for packaging into this Terminal window and press return."
echo "To abort this script, press Control-C."
echo
read -p "App path: " appPath
done
app=$( /usr/bin/basename "$appPath" )
echo
# the time right now
startTime=$( /bin/date '+%s' )
# prompt for folder for the final package
theCommand='set chooseFolder to POSIX path of (choose folder with prompt "Save package to this folder...")'
chooseDirectory=$( /usr/bin/osascript -e "$theCommand" )
# verify chosen directory is writable
logmessage "Verifying chosen folder..."
/usr/bin/touch "$chooseDirectory/.test"
logresult "Chosen folder is writable" "Chosen folder is not writable"
/bin/rm "$chooseDirectory/.test"
# verify disk space needed to package app
appSize=$( /usr/bin/du -md 0 "$appPath" | /usr/bin/awk '{ print $1 }' )
logmessage "Size of \"$app\" in MB: $appSize"
availableSpace=$( /bin/df -g /System/Volumes/Data | /usr/bin/grep dev | /usr/bin/awk '{ print $4 }' )
logmessage "Available disk space: $availableSpace GB "
neededSpace=$(( appSize * 3 / 1024 ))
logmessage "Disk space required to build package: $neededSpace GB "
[ $availableSpace -gt $neededSpace ]
logresult "Disk has enough free space to build \"$app.pkg\"" "Disk does not have enough free space to build \"$app.pkg\""
echo "Log file is located in /Library/Logs/MegaPKGr.log"
echo
# create temporary working directory
logmessage "Creating temporary working directory..."
tempDirectory=$( /usr/bin/mktemp -d "/private/tmp/MegaPKGr.XXXXXX" )
logresult "Succeeded creating temporary working directory" "Failed creating temporary working directory"
# create dmg from the app to create a single file
logmessage "Creating disk image (DMG) archive of \"$app\"..."
/usr/bin/hdiutil create -quiet "$tempDirectory/$app.dmg" -ov -volname MegaPKGr -fs APFS -srcfolder "$appPath"
logresult "Succeeded creating DMG archive of $app" "Failed creating DMG archive of $app"
# create folder structure for pkgbuild
logmessage "Ceating pkgbuild ROOT directory..."
/bin/mkdir -p "$tempDirectory/ROOT/private/tmp/MegaPKGrFileParts" && cd "$tempDirectory/ROOT/private/tmp/MegaPKGrFileParts"
logresult "Succeeded creating pkgbuild ROOT directory" "Failed creating pkgbuild ROOT directory"
logmessage "Creating pkgbuild Scripts directory"
/bin/mkdir -p "$tempDirectory/Scripts"
logresult "Succeeded creating pkgbuild Scripts directory" "Failed creating pkgbuild Scripts directory"
# split archive file into 500 MB file chunks
logmessage "Creating 500 MB file chunks from archive file..."
/usr/bin/split -b 500m "$tempDirectory/$app.dmg"
logresult "Succeeded creating 500 MB file chunks from archive file" "Failed creating 500 MB file chunks from archive file"
# create postinstall script for pkgbuild
logmessage "Creating pkgbuild postinstall script..."
/bin/cat << 'EOF' > "$tempDirectory/Scripts/postinstall"
#!/bin/bash
function logresult() {
if [ $? = 0 ] ; then
echo "$1"
else
echo "$2"
exit 1
fi
}
function cleanup() {
/bin/rm -Rf "$tempDirectory/" "/private/tmp/MegaPKGr"*
logresult "[RESULT: $?] Succeeded removing temporary items" "[ERROR: $?] Failed removing temporary items"
}
# create temporary working directory
echo "Creating temporary working directory"
tempDirectory=$( /usr/bin/mktemp -d "/private/tmp/MegaPKGr.XXXXXX" )
logresult "[RESULT: $?] Succeeded creating temporary working directory" "[ERROR: $?] Failed creating temporary working directory"
# change directory to temporary working directory
echo "Changing directory to working directory '$tempDirectory'"
cd "$tempDirectory"
logresult "[RESULT: $?] Succeeded changing to temporary working directory" "[ERROR: $?] Failed changing to temporary working directory"
# recombine file parts
echo "Recombining file parts"
/bin/cat /private/tmp/MegaPKGrFileParts/x* > "$tempDirectory/MegaPKGrFileParts.dmg"
logresult "[RESULT: $?] Succeeded recombining MegaPKGrFileParts file parts" "[ERROR: $?] Failed recombining MegaPKGrFileParts file parts"
# copy the application to original location
echo "Extracting archived file"
# mounting recombined DMG
/usr/bin/hdiutil attach -nobrowse "$tempDirectory/MegaPKGrFileParts.dmg"
logresult "[RESULT: $?] Succeeded mounting DMG" "[ERROR: $?] Failed mounting DMG"
# dittoing DMG contents to local disk
/usr/bin/ditto -rsrc "/Volumes/MegaPKGr/" /Applications
logresult "[RESULT: $?] Succeeded dittoing files to specified location" "[ERROR: $?] Failed dittoing files to specified location"
# unmounting recombined DMG
/sbin/umount "/Volumes/MegaPKGr"
logresult "[RESULT: $?] Succeeded unmounting DMG" "[ERROR: $?] Failed unmounting DMG"
# delete temporary files
echo "Deleting temporary items"
cleanup
exit 0
EOF
/bin/chmod +x "$tempDirectory/Scripts/postinstall"
logresult "Succeeded creating pkgbuild postinstall script" "Failed creating pkgbuild postinstall script"
# create installer package
logmessage "Creating package \"$app.pkg\"..."
/usr/bin/pkgbuild --quiet --identifier net.talkingmoose.MegaPKGr --version "1.0.0" --scripts "$tempDirectory/Scripts" --root "$tempDirectory/ROOT" "$chooseDirectory/$app.pkg"
logresult "Succeeded creating package \"$app.pkg\" in $chooseDirectory" "Failed creating package \"$app.pkg\" in \"$chooseDirectory\""
# remove temporary files
logmessage "Removing temporary items..."
cleanup
echo
# calculate and report script run time
stopTime=$( /bin/date '+%s' )
seconds=$(( $stopTime-$startTime ))
formattedTime=$( /bin/date -j -f "%s" "$seconds" '+%M minutes and %S seconds' )
logresult "Packaging time: $formattedTime"
echo
exit 0
@dev-yeet
Copy link

dev-yeet commented Dec 7, 2020

look at the comment I made above. you need to provide the full path of the script so like /Users/shabeer3989/MegaPKGr.zsh

@Shabeer3989
Copy link

@dev-yeet, i just provided the full path of the script but i have no .pkg

bash-3.2# bash Users/shabeerahmed/Documents/Script/MegaPKGr.zsh /Applications/Install\ macOS\ Big\ Sur.app/

Disk has enough free space to build "Install macOS Big Sur.app.pkg"
Log file is located in /Library/Logs/MegaPKGr.log

Succeeded creating temporary working directory
Succeeded creating DMG archive of Install macOS Big Sur.app.app"...
Succeeded creating pkgbuild ROOT directory
Succeeded creating pkgbuild Scripts directory
Succeeded creating 500 MB file chunks from archive file
Succeeded creating pkgbuild postinstall script
pkgbuild: error: directory "Users/shabeerahmed/Documents/Script" does not exist for writing "Install macOS Big Sur.app.pkg".
Failed creating package "Install macOS Big Sur.app.pkg" in the same folder as this script
Aborting script
Succeeded removing temporary items
bash-3.2#

@dev-yeet
Copy link

dev-yeet commented Dec 8, 2020

/Users/shabeerahmed/Documents/Script/MegaPKGr.zsh /Applications/Install\ macOS\ Big\ Sur.app

try running that instead, don't specify bash in the beginning.

@Shabeer3989
Copy link

@dev-yeet, it is working fine. thank you

@dstranathan
Copy link

dstranathan commented Dec 11, 2020

Great script William - Thanks!

For giggles, I ran MegaPKGr against the Apple Big Sur 11.0.1 installer app (~12GB) on a brand new 2020 Intel MacBook Pro 13" and a brand new M1 Mac mini (Apple Silicon). Results:

-Intel: ~14 minutes. Fans kicked in for a few minutes early on.
-Apple M1: ~10 minutes. No fan activity or heat that I could detect.

Not as exciting as I was hoping for but still interesting.

@talkingmoose
Copy link
Author

For folks who've been calling the script in Terminal with an indeterminate path like ./MegaPKGr.zsh, I've changed the behavior of the script to display an osascript prompt to request a folder for placing the final package. This should alleviate missing packages at the end of running the script regardless of how it's invoked.

@pairjal
Copy link

pairjal commented Dec 15, 2020

I have this error:

sudo bash /Users/faizal.kamaruddin/Desktop/MegaPKG/MegaPKGr.zsh /Applications/Install\ macOS\ Big\ Sur.app

Chosen folder is writable.
Disk has enough free space to build "Install macOS Big Sur.app.pkg"
Log file is located in /Library/Logs/MegaPKGr.log

Succeeded creating temporary working directory
Succeeded creating DMG archive of Install macOS Big Sur.app.app"...
Succeeded creating pkgbuild ROOT directory
Succeeded creating pkgbuild Scripts directory
Succeeded creating 500 MB file chunks from archive file
Succeeded creating pkgbuild postinstall script
pkgbuild: error: Cannot write package to "/Users/faizal.kamaruddin/Desktop/MegaPKG//Install macOS Big Sur.app.pkg".
Failed creating package "Install macOS Big Sur.app.pkg" in "/Users/faizal.kamaruddin/Desktop/MegaPKG/"
Aborting script
Succeeded removing temporary items

@talkingmoose
Copy link
Author

@pairjal, are you possibly using iCloud integration and syncing your Desktop? Although you have enough disk space, for some reason the script doesn't want to write to your Desktop folder. See what happens if you write to /Users/Shared instead.

@pairjal
Copy link

pairjal commented Dec 15, 2020

thanks @talkingmoose , work now

@sjha967
Copy link

sjha967 commented Dec 24, 2020

Thank you. It successfully created the PKG and uploaded to JAMF Cloud. But while using Cache & install Cached it throws error in Install Cached. Error Says:

Installation failed. The installer reported: installer: Package name is Install macOS Big Sur11.1

    installer: Installing at base path /

    installer: The install failed. (The Installer encountered an error that caused the installation to fail. Contact the software manufacturer for assistance. An error occurred while running scripts from the package “Install macOS Big Sur11.1.pkg”.)

@pythoninthegrass
Copy link

osascript isn't launching when called from ${chooseDirectory} over SSH. When the script is run from Terminal.app over ARD, the prompt appears as expected. Was also thinking about appending a line in /etc/synthetic.conf for /.test (host is macOS 10.15.3), but this is a remote box that may not come back up if rebooted to implement the changes.

Not sure if it's worth controlling for, but was about to hard-code paths or substitute read lines to work around it if the prompt never came up ¯\_(ツ)_/¯

ISSUE

sudo /Users/<REDACTED>/Downloads/MegaPKGr.zsh /Applications/Install\ macOS\ Big\ Sur.app/
Password:

2021-01-08 15:28:39.491 osascript[94591:88812601] -[__NSCFConstantString objectAtIndex:]: unrecognized selector sent to instance 0x7fff8a093b90
2021-01-08 15:28:39.498 osascript[94591:88812601] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[__NSCFConstantString objectAtIndex:]: unrecognized selector sent to instance 0x7fff8a093b90'
*** First throw call stack:
(
	0   CoreFoundation                      0x00007fff3147a8ab __exceptionPreprocess + 250
	1   libobjc.A.dylib                     0x00007fff6759b805 objc_exception_throw + 48
	2   CoreFoundation                      0x00007fff314f9b61 -[NSObject(NSObject) __retain_OA] + 0
	3   CoreFoundation                      0x00007fff313deadf ___forwarding___ + 1427
	4   CoreFoundation                      0x00007fff313de4b8 _CF_forwarding_prep_0 + 120
	5   CoreFoundation                      0x00007fff313ad23c CFArrayContainsValue + 197
	6   HIServices                          0x00007fff2f603a59 TransformProcessType + 927
	7   osascript                           0x000000010f8f603c osascript + 12348
	8   HIToolbox                           0x00007fff2fff25bd AEInteractWithUser + 53
	9   StandardAdditions                   0x00000001136f21a5 AEVTsysostdf + 1461
	10  AE                                  0x00007fff32778092 _AppleEventsCheckInAppWithBlock + 18070
	11  AE                                  0x00007fff32787ac4 AESendMessage + 2843
	12  AE                                  0x00007fff32793dad aeSend + 355
	13  osascript                           0x000000010f8f4b21 osascript + 6945
	14  AppleScript                         0x00007fff44aca4d2 _Z13ComponentSendPK6AEDescPS_ii + 485
	15  AppleScript                         0x00007fff44adbfa2 _ZN15TUASApplication4SendEP25TStackFrame_UASRemoteSendP6AEDescS3_hhh + 2320
	16  AppleScript                         0x00007fff44afcdbe _Z13UASRemoteSendhhhhhPh + 548
	17  AppleScript                         0x00007fff44ad53ba _Z13UASActor_Sendhhh + 383
	18  AppleScript                         0x00007fff44ae03e2 _Z11UASExecute1v + 1127
	19  AppleScript                         0x00007fff44ab3b47 _Z14ASExecuteEventPK6AEDescjiPj + 614
	20  AppleScript                         0x00007fff44aac6e1 AppleScriptComponent + 1677
	21  AppleScript                         0x00007fff44ac5dcb _ZN12AGenericCall8DelegateEP23ComponentInstanceRecord + 37
	22  AppleScript                         0x00007fff44ac5d91 _ZN15AGenericManager13HandleOSACallEP19ComponentParameters + 57
	23  AppleScript                         0x00007fff44ac539c GenericComponent + 156
	24  OpenScripting                       0x00007fff302353f2 OSAExecuteEvent + 50
	25  osascript                           0x000000010f8f55a3 osascript + 9635
	26  libdyld.dylib                       0x00007fff689097fd start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException
touch: /.test: Read-only file system
Chosen folder is not writable
Aborting script
Succeeded removing temporary items

WORKAROUND (TERMINAL.APP)

<REDACTED> Downloads % sudo ./MegaPKGr.zsh /Applications/Install\ macOS\ Big\ Sur.app
Password:

Chosen folder is writable.
Disk has enough free space to build "Install macOS Big Sur.app.pkg" 
Log file is located in /Library/Logs/MegaPKGr.log

Succeeded creating temporary working directory
Creating disk image (DMG) archive of "Install macOS Big Sur.app"...

@TJonesR2
Copy link

So, this script technically works for me. In the sense that it does create a package. But the package is the same size as if I just used Composer.
sudo /Users/name/Documents/MegaPKGr.zsh /Applications/Install\ macOS\ Big\ Sur.app
Password:
Chosen folder is writable.
Disk has enough free space to build "Install macOS Big Sur.app.pkg"
Log file is located in /Library/Logs/MegaPKGr.log

Succeeded creating temporary working directory
Succeeded creating DMG archive of Install macOS Big Sur.app.app"...
Succeeded creating pkgbuild ROOT directory
Succeeded creating pkgbuild Scripts directory
Succeeded creating 500 MB file chunks from archive file
Succeeded creating pkgbuild postinstall script
Succeeded creating package "Install macOS Big Sur.app.pkg" in /Users/theresa.jones/Documents/
Succeeded removing temporary items

Packaging time: 18 minutes and 31 seconds

I'm trying to package a Big Sur installer. And with this, it's still 12.2 GB
Any thoughts?

@TJonesR2
Copy link

@sjha967 - Did you figure out a way to get it to install or cache?

@talkingmoose
Copy link
Author

@TJonesR2, the intent of the script isn't to reduce the size but to make the installer app deployable. For example, because of its size, Composer cannot make a PKG file of the Big Sur installer app.

@talkingmoose
Copy link
Author

@pythoninthegrass, the script was written to be interactive and because of the osascript prompt (necessary because folks were calling the script from a relative path instead of providing the full path) it'll require you be remote controlling the Mac. Unless you modify it to hard code some paths, SSH and ARD will likely be problematic.

@pythoninthegrass
Copy link

Understood @talkingmoose. Thanks for writing this up!

@kuhnamie
Copy link

hmm i have well over 200gb of free space but keep getting "Disk does not have enough free space to build "Install macOS Big Sur.app.pkg" "

I see others have had this issue but i am providing full paths for the script and the chosen folder is writable. Any suggestions?

@talkingmoose
Copy link
Author

@kuhnamie, is your chosen folder something like Desktop or Documents that may be syncing with iCloud? Try saving to some place like /Users/Shared and see what happens.

@kuhnamie
Copy link

So the folder i chose now is /Users/Shared and i get:
'df: /System/Volumes/Data: No such file or directory
/path/to/MegaPKGr.zsh:[:123: unknown condition: -gt
Disk does not have enough free space to build "Install macOS Big Sur.app.pkg" '

@kuhnamie
Copy link

So i tried the exact same process on a different computer and it worked fine. ¯_(ツ)_/¯

@salihzett
Copy link

Same issue:

Disk does not have enough free space to build "Install macOS Big Sur.app.pkg"
Aborting script
Succeeded removing temporary items

I have 99,45 GB available. Thats strange.

@rhype
Copy link

rhype commented Jan 25, 2021

Hi @talkingmoose
I need some help here, I keep getting this error: Failed creating DMG archive of Install macOS Big Sur.appSur.app"
here is what I am running in the terminal:
sudo /Users/Shared/MegaPKGr.zsh /Applications/Install\ macOS\ Big\ Sur.app
I am not sure what the work around is as I have done this on another device prior and it ran successfully for me.

@JamesV93
Copy link

Script works really well, I was just wondering if anyone else had the same error as @sjha967. I too am getting the same thing but can't figure out what might be causing it to fail. Any input is appreciated, Thank You!

Thank you. It successfully created the PKG and uploaded to JAMF Cloud. But while using Cache & install Cached it throws error in Install Cached. Error Says:

Installation failed. The installer reported: installer: Package name is Install macOS Big Sur11.1

    installer: Installing at base path /

    installer: The install failed. (The Installer encountered an error that caused the installation to fail. Contact the software manufacturer for assistance. An error occurred while running scripts from the package “Install macOS Big Sur11.1.pkg”.)

@talkingmoose
Copy link
Author

@rhype, a few folks have reported issues running the same command on different Macs and receiving different results. Not sure what could be different between the machines other than one is enabled for iCloud and the other is not.

Because you and @JamesV93 are trying to do this with Big Sur, I suggest using Armin Briegel's fetch-installer-pkg script instead. It downloads the same Apple installer package that Software Update would download from Apple and it's already in PKG format.

While my script should work, I haven't been able to reproduce any of the failures with it because I'm likely not able to reproduce the environmental conditions causing them.

@rhype
Copy link

rhype commented Jan 27, 2021

@talkingmoose
Thanks for this, I was able to get around the error "Failed creating DMG archive of Install macOS Big Sur.app" by uninstalling the Jamf Pro tools and re-installing it. However, I am now getting the error: "Installation failed. The package could not be verified." I have tried this on multiple devices, on different networks but the issue still persists. I am going to try the Armin Briegel's fetch-installer-pkg script you suggested. I will provide updates. Thanks for your help.

@dv8nrg
Copy link

dv8nrg commented Feb 2, 2021

@talkingmoose

Hi Bill!

We've met before when you came onsite to Cadence Design Systems (in San Jose,Ca) years ago to help with my environment back then, just wanted to say hi again, how you doing?!

Anyhow, this might be a small syntax error somewhere but I am getting the following error when running your script:

% sudo ./MegaPKGr.zsh /Applications/Install\ macOS\ Big\ Sur.app

35:93: execution error: User canceled. (-128)
touch: /.test: Read-only file system
Chosen folder is not writable
Aborting script
Succeeded removing temporary items

Let me know if you can shed a light, thanks and stay safe!

-Dave

@dv8nrg
Copy link

dv8nrg commented Feb 2, 2021

Never mind what I said, for some reason, I couldn't even run ls -la in any of my directories, changed to a clean system and was able to run the command just fine:

Desktop % ls -la
ls: .: Operation not permitted

Sorry for the noise...thank you and have a good year!

@Kuldeep-Github
Copy link

Kuldeep-Github commented May 26, 2021

@talkingmoose

thank you writing this script, this really helps us.
My question, can we put the .app under /private/var ? instead /Application directory.
if yes, then where we need to make change in script.
Actually, I am trying to deploy the package via Jamf and want to cache the package under /private/var/ ,so that, its not visible for users to do any thing.

@dev-yeet
Copy link

dev-yeet commented May 26, 2021

@Kuldeep-Github

replace the only reference to "/Applications" with the new destination; i just tested with /private/var/tmp and it placed the installer there for me instead.

@Kuldeep-Github
Copy link

Thank you, @dev-yeet.
Yes it drops the location where we want by just replacing "/Application" to new_path

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment