Skip to content

Instantly share code, notes, and snippets.

@jhg03a

jhg03a/CustomVMwareSecurebootIso.md

Created October 20, 2022 21:44
Show Gist options
  • Save jhg03a/7eed0b0f97c91a46b9cf65e22da7ab9f to your computer and use it in GitHub Desktop.
Save jhg03a/7eed0b0f97c91a46b9cf65e22da7ab9f to your computer and use it in GitHub Desktop.
Download ZIP
Some instructions on how to make an UEFI SecureBoot capable custom ISO with menus for Kickstart
Raw
CustomVMwareSecurebootIso.md

I couldn't find instructions on the internet anywhere for how to craft a UEFI secure bootable VMware ESXi installer disk iso. I worked it out using the rEFInd OSS project. This is not perfect or optimal, but it does the job.

Inspirational Links of note

Requirements

  • VMware secure boot public certs (Optional unless you couldn't secure boot the official VMware ISO before)
  • An official VMware ESXi install ISO
  • The rEFInd release iso file
  • The efi shim from a linux distributions.
    • Since I was already using a Centos7.9 docker image, I just installed shim-x64 inside docker and copied out /boot/efi/EFI/centos/shim.efi & /boot/efi/EFI/centos/MokManager.efi to the host volume mount
  • Build environment binaries:
    • xorriso
    • unzip
    • mkisofs (genisoimage flavor)
    • isohybrid
    • implantisomd5

Setup

For my purposes, I used a CentOS 7.9 docker image with a host volume mount to prep the iso. I'm also testing against a HPE ProLiant DL325 Gen10 V2 Plus.

Directory setup

vmware (volume host mounted into docker)
│   source.iso (official VMware ISO)
│   rEFInd.iso (official rEFInd release ISO)
│   shim.efi
│   MokManager.efi
└─── esxi_7u3 (workingdir effectively)
     └─── KS (directory for VMware Kickstart files)

Build

  1. cd into the vmware directory
  2. Extract the rEFInd zip into the vmware dir unzip refind-cd-0.13.3.1.zip
  3. Mount the rEFInd iso: mkdir /tmp/refind.iso && mount -o loop,ro refind-cd-0.13.3.1.iso /tmp/refind.iso/
  4. cd into the esxi_7u3 directory
  5. Use xorriso to extract the official VMware ISO: xorriso -osirrox on -indev ../source.iso -extract / .
  6. Move ESXi bootloader to the side: mv EFI/BOOT/BOOTX64.EFI EFI/BOOT/ESXBOOT64.EFI
  7. Move the shim.efi into its place mv ../shim.efi EFI/BOOT/BOOTX64.EFI
  8. Move the mokmanager alongside mv ../MokManager.efi EFI/BOOT/MokManager.efi
  9. Move the rEFInd EFI into place under grub's name mv /tmp/refind.iso/refind/refind_x64.efi EFI/BOOT/grubx64.efi
  10. Create whatever kickstart files you want for VMware in the KS directory, for example 0.KS
    • Reminder that when using secure boot you cannot use the %firstboot directive in the ks file
      • From my own experiments and poking around the bootbank, it seems VMware did try and plug as many holes as they could to keep you from hooking cron or init.rc to do what the %firstboot directive already does with SecureBoot off. The last possible way maybe I didn't check is that in /bootbank/onetime.tgz appears to be some basic things like the hostname and password configs. There's also a var/lib/vmware/firstboot/firstboot.json file with a copyFiles key that's a list of source/dests. Maybe you could augment the existing onetime.tgz during the %post to include an extra self-deleting shell script and add that to the copyFiles list into /etc/init.rc.d/. I basically stopped short here because at this point you're really getting into something more volitile from release to release and it's probably better to just use real vCenter Host Profiles at this point.
  11. (Optional) Craft an older ISOLINUX.CFG file to make a menu for legacy bios booting; see links of note above
  12. Craft whatever you want the rEFInd config file to be at EFI/BOOT/refind.conf: see Appendix A: refind.conf
  13. Copy over the rEFInd keys directory for convenience or maybe to simplify things later cp -r /tmp/refind.iso/keys .
  14. Create an empty file of size necessary to contain the build directory: truncate -s $(du -sm . | tr -dc '[:alnum:]\n\r')M refind.boot.img
  15. Format the empty file with a dos filesystem: mkdosfs -n 'Custom VMware Install Image' refind.boot.img
  16. Mount this new boot image: mount -o rw,sync refind.boot.img /mnt
  17. Mount the original rEFInd boot image: mkdir /tmp/refind.boot && mount -o ro,sync /tmp/refind.iso/refind-bin-0.13.3.1.img /tmp/refind.boot
  18. Copy over the rEFInd image as a base: rsync --delay-updates -F --compress --archive --out-format=<<CHANGED>>%i %n%L /tmp/refind.boot/ /mnt/
  19. Copy the current workingdir on top of it: rsync --delay-updates -F --compress --archive --exclude=refind.boot.img --out-format=<<CHANGED>>%i %n%L . /mnt/
  20. Unmount everything unmount /mnt && unmount /tmp/refind.boot && unmount /tmp/refind.iso
  21. Build the ISO: mkisofs -V 'custom_esxi_7u3.iso' -R -J -v -T -o '../esxi_7u3.iso' -relaxed-filenames -b ISOLINUX.BIN -c BOOT.CAT -no-emul-boot -boot-load-size 4 -boot-info-table -eltorito-alt-boot -e refind.boot.img -no-emul-boot .
  22. (Optional) Make it an hybrid iso to work with both legacy bios and efi: isohybrid --uefi '../custom_esxi_7u3.iso'
  23. (Optional) Implant the ISO MD5: implantisomd5 '../custom_esxi_7u3.iso'

Important things to know

  • Depending on your BIOS, it may be possible to simply enroll the keys/refind.cer into the secureboot trust db. If you can, that should bypass the need to use the shim at all and you can change the rEFInd efi to BOOTX64.EFI instead. There's no harm in leaving the shim in place if you only want one iso for everything.
    • Be aware that if you trust the rEFInd signing cert, that means any past/future binaries produced with that key will also be trusted automatically.
  • The ElTorito image file used during makeisofs is just a dos formatted filesystem and it's not tied to secureboot. Secureboot primarily cares about the binary files like *.efi and not other stuff like refind.conf or additional rEFInd graphics.
  • The shim.efi should be signed by Microsoft already and serves as an intermediary for other distros to sign their own boot files. This should already be present in most modern BIOS SecureBoot trust stores.
  • The specific names and paths to the EFI inside the iso are important. By default UEFI looks to /EFI/BOOT/BOOTX64.EFI when booting a disk iso. Likewise the shim was designed to be used with Grub so it expects the real efi to be at /EFI/BOOT/grubx64.efi.
  • This particular overlay approach to the ISO creation and use process is nice because it's not limited to any particular version of ESXi or flavor of ESXi such as the official dell/hp images or even a custom iso built via the default VMware tooling.
  • This approach is also nice because it avoids having to create your own uefi signing certs and resigning everything, although it doesn't preclude it if you want.
  • Since the boot images are dos formatted, you'll want to follow 8.3 filename conventions.

Usage

Host is in BIOS boot mode

The build ISO should work no differently than usual for a potentially customized isolinux boot

Host is in UEFI boot mode and SecureBoot is disabled

Nothing out of the ordinary to do to get into rEFInd

Host is in UEFI boot mode and SecureBoot is enabled and you're able to enroll the rEFInd key into the system BIOS SecureBoot trust

Nothing out of the ordinary to do to get into rEFInd

Host is in UEFI boot mode and SecureBoot is enabled and you're not able to enroll the rEFInd key into the system BIOS SecureBoot trust

During the boot process it's likely going to complain about security violations, but if you continue through that it will give you the opportunity to enroll binaries into the MOK for the shim to use. Every efi file under /EFI/BOOT should be enrolled. A reboot may be required, but now when it boots into the iso it should skip past all the MOK enrollment pieces and into rEFInd to pick what install kickstart you want.

Appendix A: refind.conf

It's usually a good idea to start with the official refind.conf template as a base.

The important bits I changed were:

  • timeout 0 to disable choosing a default option
  • scanfor manual or at least be sure manual is in your list
  • Some menuentry for each ks

Multiple root menu example

menuentry "Install VMware ESXi - Somehost" {
  loader \EFI\BOOT\ESXBOOT64.EFI
  options "ks=cdrom:/KS/0.KS"
}

Submenu example

If you use the default option in this method, you'll get a regular install process without kickstart involved.

menuentry "Install Custom VMware ESXi" {
  loader \EFI\BOOT\ESXBOOT64.EFI
  submenuentry "Somehost 0" {
    options "ks=cdrom:/KS/0.KS"
  }
  submenuentry "Somehost 1" {
    options "ks=cdrom:/KS/1.KS"
  }
}

The important menuentry options to include are to chainload the official ESXi bootloader and to append the ks to the kerneloptions. You can verify this did what you think during the ESXi boot using [Shift]+O to view the boot parameters. Don't worry about the stuff that comes at the beginning of the boot parameters, the additional ks= bit at the end is what's important. Also note that the ks= kernel parameter doesn't just accept the iso as the only place kickstart files can live. If you want and your rEFInd instance has a functional network, you could reference an outside source for the kickstart files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment