Skip to content

Instantly share code, notes, and snippets.

@eddieh
Created July 28, 2021 19:43
Show Gist options
  • Star 18 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save eddieh/b3c04436eb6b28826939cd57f1b704bb to your computer and use it in GitHub Desktop.
Save eddieh/b3c04436eb6b28826939cd57f1b704bb to your computer and use it in GitHub Desktop.
chroot Environments on macOS (draft)

chroot Environments on macOS

This is a draft.

macOS doesn’t have many of the advanced Linux or UNIX features that have come about in the past 20 years. So getting a proper chroot environment up and running takes a little more work.

The basics of chroot is to run some command (a shell in our case) such that its filesystem root is an arbitrary directory on the system.

For this explainer and on my system I put all my chroot environment directories in my home under rts and each directory has .root appended to the name. This convention is purely for bookkeeping.

So the absolute first thing to try is creating a empty chroot environment, we’ll call it ~/rts/null.root

$ mkdir -p rts/null.root
$ chroot rts/null.root

Fails with the following error

chroot: rts/null.root: Operation not permitted

We’ve just learned our first chroot leason. Only the superuser can create a chroot environment (something, something privilege escalation).

Now try to chroot our empty root using sudo.

$ sudo chroot rts/null.root

Again this fails with a different error

chroot: /bin/sh: No such file or directory

Our empty root does not have access to anything since it is empty. Depending on your needs copying over the necessary files may be appropriate. For other circumstances it may be more appropriate to mount a directory from elsewhere on you system inside the new root.

macOS does not provide an out-of-the-box way to bind a directory inside of another so we’ll turn to the venerable usespace filesystem package FUSE. Specifically we’re going to use bindfs (one of the many FUSE filesystems) to replicate directories.

As above we need sh from /bin so we’ll mount (or rather bind) /bin to a new root that well place in rts/base.root.

But first we need to install bindfs. If you’re using Homebrew (lord help you). This is what you need to do.

$ brew install macfuse
$ brew edit bindfs

Locate the block that calls disable!.

on_macos do
  disable! date: "2021-04-08", because: "requires closed-source macFUSE"
end

Kill that line with the call to disable!, so it looks like the following snippet now

on_macos do
end

Save and quit your editor. Now Homebrew will allow you to install bindfs.

$ brew install bindfs

We’re only going to bind /bin so we can demonstrate another error you may encounter. After this we will bind enough to replicate a base CLI environment.

$ mkdir -p rts/base.root
$ mkdir -p rts/base.root/bin
$ bindfs -r /bin rts/base.root/bin
$ sudo chroot rts/base.root

You’ll see this fail with the cryptic Killed: 9 and nothing else. /bin/sh depends on a shared library that also needs to be available in the chroot. We can check this with otool

$ otool -L /bin/sh
/bin/sh:
	/usr/lib/libSystem.B.dylib ...

Every binary on macOS needs to link libSystem.B.dylib, even void main() {} compiled with cc test.c

$ otool -L a.out
a.out:
	/usr/lib/libSystem.B.dylib

So we can see that our chroot needs to bind /usr/lib so that any binary can run.

$ mkdir -p rts/base.root/usr/lib
$ bindfs -r /usr/lib rts/base.root/usr/lib
$ sudo chroot rts/base.root

Then we bind the rest of the system

$ R=rts/base.root
$ mkdir -p $R/usr/libexec
$ mkdir -p $R/usr/sbin
$ mkdir -p $R/usr/share
$ mkdir -p $R/System
$ mkdir -p $R/Library
$ bindfs -r /usr/libexec $R/usr/libexec
$ bindfs -r /usr/libexec $R/usr/libexec
$ bindfs -r /usr/sbin $R/usr/sbin
$ bindfs -r /usr/share $R/usr/share
$ bindfs -r /System $R/System
$ bindfs -r /Library $R/Library

$ mkdir -p $R/etc $R/sbin $R/home $R/tmp $R/var
$ bindfs -r /etc $R/etc
$ bindfs -r /sbin $R/sbin

$ mkdir -p $R/dev
$ bindfs -o dev /dev $R/dev

$ mkdir -p $R/var/run
$ bindfs -r /var/run $R/var/run

$ mkdir -p $R/var/db
$ bindfs -r /var/db $R/var/db

This works okay for basic UNIX stuff. There are some gotchas I haven’t worked out. e.g. host, dig, and nslookup can resolve names, but curl fails.

If you want to compile stuff you’ll need to have installed the Command Line Tools (outside of the chroot environment since we’ve made it readonly) or you’d have to bind /Applications/Xcode.app too.

Oh, to exit the chroot, just type exit at the prompt. To unbind the directories use umount. So umount rts/base.root/bin would remove the bin binding from the base.root environment.

@WaterJuice
Copy link

Hi, I was wondering if you have been able to get chroot to work on macOS Monterey. I have not had any success so far

@eddieh
Copy link
Author

eddieh commented Apr 9, 2022

I have not tried on Big Sur or Monterey. I've been a curmudgeon and sticking with Catalina.

You might have to disable SIP and you have to create the chroot in your home (even with SIP disabled or else get around the readony root).

I gave up on using bindfs too. It would fail when building LLVM and other large projects. So I switched to using a sparseimage with shadow files. Here is the script I use in pkgmgr: https://github.com/pkgmgr/pkgmgr/blob/master/scripts/darwin-chroot-alt

@eddieh
Copy link
Author

eddieh commented Apr 9, 2022

You'll probably also want some version of this login script: https://github.com/pkgmgr/pkgmgr/blob/master/scripts/homes/Darwin/.profile-alt

I should have inlined it in the chroot script using a heredoc...

@stephen-fox
Copy link

Sadly, it appears Apple has killed support for chroot due to whatever the "hardened runtime" does / requires: https://threedots.ovh/blog/2021/01/chroot-on-modern-macos-disallowed/

@tomalbrc
Copy link

Sadly, it appears Apple has killed support for chroot due to whatever the "hardened runtime" does / requires: https://threedots.ovh/blog/2021/01/chroot-on-modern-macos-disallowed/

Good to know which macOS version is affected. (Watch out - Sarcasm)

@cachius
Copy link

cachius commented Apr 6, 2023

@slonopotamus
Copy link

You might also be interested in https://macoscontainers.org/

@mikebeaton
Copy link

Hi, I was wondering if you have been able to get chroot to work on macOS Monterey. I have not had any success so far

Disable SIP

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment