Skip to content

Instantly share code, notes, and snippets.

@knightsc
Created February 18, 2020 15:08
Show Gist options
  • Save knightsc/abbee4ac48b0a43198f5ae39c0f8843f to your computer and use it in GitHub Desktop.
Save knightsc/abbee4ac48b0a43198f5ae39c0f8843f to your computer and use it in GitHub Desktop.
A script to build XNU version 6153.11.26 (macOS Catalina 10.15).
#! /bin/bash
#
# build-xnu-6153.11.26.sh
# Scott Knight
#
# Based on the script by Brandon Azad
# https://gist.github.com/bazad/654959120a423b226dc564073b435453
#
# A script showing how to build XNU version 6153.11.26 on macOS Catalina
# 10.15 with Xcode 11.13.1.
#
# Note: This process will OVERWRITE files in Xcode's MacOSX10.15.sdk. Make a
# backup of this directory first! Or set BACKUP_SDK=1
#
# Set the working directory.
WORKDIR="${WORKDIR:-build-xnu-6153.11.26}"
# Set a permissive umask just in case.
umask 022
# Print commands and exit on failure.
set -ex
# Get the SDK path and toolchain path.
SDKPATH="$(xcrun --sdk macosx --show-sdk-path)"
TOOLCHAINPATH="$(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain"
[ -d "${SDKPATH}" ] && [ -d "${TOOLCHAINPATH}" ]
# Create the working directory.
mkdir "${WORKDIR}"
cd "${WORKDIR}"
# Back up the SDK if that option is given.
if [ -n "${BACKUP_SDK}" ]; then
sudo ditto "${SDKPATH}" "$(basename "${SDKPATH}")"
fi
# Download XNU and some additional sources we will need to help build.
curl https://opensource.apple.com/tarballs/xnu/xnu-6153.11.26.tar.gz | tar -xf-
curl https://opensource.apple.com/tarballs/dtrace/dtrace-338.0.1.tar.gz | tar -xf-
curl https://opensource.apple.com/tarballs/AvailabilityVersions/AvailabilityVersions-45.tar.gz | tar -xf-
curl https://opensource.apple.com/tarballs/libplatform/libplatform-220.tar.gz | tar -xf-
curl https://opensource.apple.com/tarballs/libdispatch/libdispatch-1173.0.3.tar.gz | tar -xf-
# Build and install ctf utilities. This adds the ctf tools to
# ${TOOLCHAINPATH}/usr/local/bin.
cd dtrace-338.0.1
cd include/llvm-Support
rm PointerLikeTypeTraits.h
curl https://gist.githubusercontent.com/knightsc/cf46670ea023168cdfe98b4a295f2cf4/raw/00f0b13c00983e4010ba0019eeeecb0ba9a381e7/PointerLikeTypeTraits.h > PointerLikeTypeTraits.h
curl https://gist.githubusercontent.com/knightsc/fe2cbe276a006fe601b704cd5286047f/raw/bb44a1b8cbfa22a8d814368189a193394b9cfe4c/DataTypes.h > DataTypes.h
cd ../..
mkdir -p obj dst sym
xcodebuild install -target ctfconvert -target ctfdump -target ctfmerge -UseModernBuildSystem=NO ARCHS="x86_64" SDKROOT=macosx SRCROOT="${PWD}" OBJROOT="${PWD}/obj" SYMROOT="${PWD}/sym" DSTROOT="${PWD}/dst"
# TODO: Get the XcodeDefault.toolchain path programmatically.
sudo ditto "${PWD}/dst/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain" "${TOOLCHAINPATH}"
cd ..
# Install AvailabilityVersions. This writes to ${SDKPATH}/usr/local/libexec.
cd AvailabilityVersions-45
mkdir -p dst
make install SRCROOT="${PWD}" DSTROOT="${PWD}/dst"
sudo ditto "${PWD}/dst/usr/local" "${SDKPATH}/usr/local"
cd ..
# Install the XNU headers we'll need for libdispatch. This OVERWRITES files in
# MacOSX10.14.sdk!
cd xnu-6153.11.26
mkdir -p BUILD.hdrs/obj BUILD.hdrs/sym BUILD.hdrs/dst
make installhdrs SDKROOT=macosx ARCH_CONFIGS=X86_64 SRCROOT="${PWD}" OBJROOT="${PWD}/BUILD.hdrs/obj" SYMROOT="${PWD}/BUILD.hdrs/sym" DSTROOT="${PWD}/BUILD.hdrs/dst"
xcodebuild installhdrs -project libsyscall/Libsyscall.xcodeproj -sdk macosx -UseModernBuildSystem=NO ARCHS="x86_64" SRCROOT="${PWD}/libsyscall" OBJROOT="${PWD}/BUILD.hdrs/obj" SYMROOT="${PWD}/BUILD.hdrs/sym" DSTROOT="${PWD}/BUILD.hdrs/dst"
# Set permissions correctly before dittoing over MacOSX10.13.sdk.
sudo chown -R root:wheel BUILD.hdrs/dst/
sudo ditto BUILD.hdrs/dst "${SDKPATH}"
cd ..
# Install libplatform headers to ${SDKPATH}/usr/local/include.
cd libplatform-220
sudo ditto "${PWD}/include" "${SDKPATH}/usr/local/include"
sudo ditto "${PWD}/private" "${SDKPATH}/usr/local/include"
cd ..
# Build and install libdispatch's libfirehose_kernel target to
# ${SDKPATH}/usr/local.
cd libdispatch-1173.0.3
mkdir -p obj sym dst
xcodebuild install -project libdispatch.xcodeproj -target libfirehose_kernel -sdk macosx -UseModernBuildSystem=NO ARCHS="x86_64" SRCROOT="${PWD}" OBJROOT="${PWD}/obj" SYMROOT="${PWD}/sym" DSTROOT="${PWD}/dst"
sudo ditto "${PWD}/dst/usr/local" "${SDKPATH}/usr/local"
cd ..
# Build XNU.
cd xnu-6153.11.26
make SDKROOT=macosx ARCH_CONFIGS=X86_64 KERNEL_CONFIGS="DEBUG"
@Shaneee
Copy link

Shaneee commented Mar 4, 2020

Not the cleanest but I was able to build and start testing on AMD machine with these changes, https://www.dropbox.com/s/lc7kryryqo2y7yf/Catalina_Fixed.diff.zip?dl=0

@knightsc
Copy link
Author

knightsc commented Mar 4, 2020

I had a set of changes here I used to compile

knightsc/darwin-xnu@429747b

Couldn't get it to boot in a VM though. Just crashed. I'll take a look at your diff and see if I can figure out what's wrong with mine.

@trulyspinach
Copy link

I can get it to boot into login screen but mouse and keyboard doesn't work :(.
A lot of kext reports missing symbol bpf_tap_packet_out and bpf_tap_packet_in.

@jhftss
Copy link

jhftss commented Mar 31, 2020

I can get it to boot into login screen but mouse and keyboard doesn't work :(.
A lot of kext reports missing symbol bpf_tap_packet_out and bpf_tap_packet_in.

Me too, but wait for serval minutes, I can use my mouse and keyboard again.

@knightsc
Copy link
Author

@phyon-xnu there are multiple patches mentioned above. The source released by Apple on it's own does not compile

https://www.dropbox.com/s/lc7kryryqo2y7yf/Catalina_Fixed.diff.zip?dl=0
knightsc/darwin-xnu@429747b

@phyon-xnu
Copy link

thank you :)

@anarcheuz
Copy link

Hey guys, how did you manage to get to the logging screen ? The boot process just keeps hanging with "still waiting for root device". Specifically it hangs at IOService::waitForService() for "boot-uuid-media". I'm on VMware Fusion.

@wjllz
Copy link

wjllz commented Jan 20, 2022

Hey, I try to build "xnu-7195.81.3". And I use vmware fusion.
My target is "11.2".

Here is my script(I install dtrace by macport, and scan-build by pip).

#! /bin/bash
# build xnu script by @18楼梦想改造家 
# Thanks to @afrojer, @knightsc and @Brandon
# You could read their original blogpost/script at here
#	[+] https://kernelshaman.blogspot.com/2021/02/building-xnu-for-macos-112-intel-apple.html
#	[+] https://gist.github.com/knightsc/abbee4ac48b0a43198f5ae39c0f8843f
#	[+] https://gist.github.com/bazad/654959120a423b226dc564073b435453


XNU_VERSION="xnu-7195.81.3"
WORKDIR="build-"$XNU_VERSION

# Set a permissive umask just in case.
umask 022

# Print commands and exit on failure.
set -ex

# Get the SDK path and toolchain path.
SDKPATH="$(xcrun --sdk macosx --show-sdk-path)"
TOOLCHAINPATH="$(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain"
[ -d "${SDKPATH}" ] && [ -d "${TOOLCHAINPATH}" ]

# Now to build the worker path
echo "${WORKDIR}"
mkdir "${WORKDIR}"
cd "${WORKDIR}"

# download related source code
TARBALLS=https://opensource.apple.com/tarballs
curl -O ${TARBALLS}/dtrace/dtrace-370.40.1.tar.gz
curl -O ${TARBALLS}/AvailabilityVersions/AvailabilityVersions-70.tar.gz
curl -O ${TARBALLS}/libdispatch/libdispatch-1271.40.12.tar.gz
curl -O ${TARBALLS}/xnu/$XNU_VERSION.tar.gz

#	build ctf tools
tar zxf dtrace-370.40.1.tar.gz
cd dtrace-370.40.1
mkdir -p obj dst sym
#xcodebuild install -sdk macosx -target ctfconvert -target ctfdump -target ctfmerge ARCHS=x86_64 VALID_ARCHS=X86_64 DSTROOT=/Users/18f/Desktop/18_floor/source_code/build-xnu-7195.81.3/dtrace-370.40.1/dst
xcodebuild install -sdk macosx -target ctfconvert \
-target ctfdump -target ctfmerge \
ARCHS='x86_64' VALID_ARCHS='X86_64' DSTROOT=$PWD/dst
sudo ditto "${TOOLCHAINPATH}" "${PWD}/dst/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain"
cd ..

tar zxf AvailabilityVersions-70.tar.gz
cd AvailabilityVersions-70
make install
sudo ditto "$PWD/dst/usr/local/libexec" \
"$(xcrun -sdk macosx -show-sdk-path)/usr/local/libexec"
cd ..

tar zxf $XNU_VERSION.tar.gz
cd $XNU_VERSION
make SDKROOT=macosx ARCH_CONFIGS="X86_64" installhdrs
sudo ditto "$PWD/BUILD/dst" "$(xcrun -sdk macosx -show-sdk-path)"
cd ..

tar zxf libdispatch-1271.40.12.tar.gz
cd libdispatch-1271.40.12
xcodebuild install -sdk macosx ARCHS='x86_64' \
VALID_ARCHS='x86_64' -target libfirehose_kernel \
PRODUCT_NAME=firehose_kernel DSTROOT=$PWD/dst
sudo ditto "$PWD/dst/usr/local" \
"$(xcrun -sdk macosx -show-sdk-path)/usr/local"
cd ..

cd $XNU_VERSION
make SDKROOT=macosx ARCH_CONFIGS=X86_64 KERNEL_CONFIGS="DEBUG" USE_WERROR=0 CONCISE=1 LOGCOLORS=y

if KERNEL_CONFIGS=DEVELOPMENT, every thing is ok...
However, if KERNEL_CONFIGS=DEBUG, I can't boot in the virtual machine.
Could u give me any tips, thank u so much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment