Is Virtual Box's EFI firmware failing to keep data between VM restarts? Have you tried both manual configuration and efibootmgr
to no avail?
Make a file called startup.nsh
containing your kernel boot line, e.g.
vmlinuz-linux root=/dev/sdXY initrd=/initramfs-linux.img
This will use EFISTUB to boot the kernel directly, see https://wiki.archlinux.org/index.php/EFISTUB#Using_UEFI_Shell. This still isn't really causing persisting changes (e.g. config changes via the NVRAM still don't persist) but at least this lets you boot EFI without fiddling with settings all the time.