Set up Mac self-hosted CI runner on AWS

mps-self-hosted.md

Amazon EC2:

• Create a Dedicated Host. Wait for it to become "Available".
• Select the host. Click "Actions" and select "Launch instance(s) onto host".
• Create the instance (Ventura on arm). Attach a secondary drive to be used as hub cache. I chose 1.5 TB.
• Connect to the instance when it's ready

Mac VM:

• Enable vnc access

  sudo defaults write /var/db/launchd.db/com.apple.launchd/overrides.plist com.apple.screensharing -dict Disabled -bool false\n
  sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.screensharing.plist

• Enable a password for ec2-user, only used for vnc connection sudo /usr/bin/dscl . -passwd /Users/ec2-user

• Open ssh tunnel for vnc (ssh -L5900:localhost:5900 <name>) and connect to vnc://localhost:5900

• Use Disk Utility to format the disk, name it hfcache so it mounts at /Volumes/hfcache

  • Or ask GPT-4 about how to format a drive from the command line, create an apfs container, add a new apfs volume...

• symlink from /System/Volumes/Data/mnt/cache. This is the location used in the workflow yml files.

  sudo chown -R ec2-user /System/Volumes/Data/mnt
  ln -s /Volumes/hfcache /System/Volumes/Data/mnt/cache
  

  (alternative: change the HF_HOME location in the github workflows)

Configure Runner (notes from Anton)

• Follow the instructions in https://github.com/huggingface/diffusers/settings/actions/runners (you need admin access to the repo) Note: don't start the service at the end of the setup, it hangs on Arm64! The runner needs the tag apple-m1, you can give it during creation or later on in the GitHub UI.
• Run the service manually in the background, e.g. with tmux

  brew install tmux
  tmux
  ./run.sh        # Observe new jobs being executed
  Ctrl+B, D
  exit