Vagrant and VMWare Tech Preview 21H1 on Apple M1 Pro

vagrant-vmware-tech-preview-apple-m1-pro.md

Vagrant and VMWare Tech Preview 21H1 on Apple M1 Pro

UPDATE November 20, 2022: VMWare Fusion 13

VMWare Fusion 13 is now released. Read Vagrant and VMWare Fusion 13 Player on Apple M1 Pro for the latest.

Summary

This document summarizes notes taken while to make the VMWare Tech preview work on Apple M1 Pro, it originated from discussions in hashicorp/vagrant-vmware-desktop#22

Created on: November 1, 2021

Updated on: November 20, 2022

Installing Rosetta

First install Rosetta if not already done, this is needed to run x86 code:

/usr/sbin/softwareupdate --install-rosetta --agree-to-license

Installing Vagrant

Install Vagrant via brew or install it manually. The version is important. DO NOT use 2.2.19

brew install vagrant@2.3.0

Installing VMWare Fusion Tech Preview

You will need to create an account on vmware as it needs user and key information that are user specific. The registration process is kinda convoluted. Be careful about passwords as the password needs to be less than 20 characters and there are no error messages for this.

You can download the tech preview via the download page.

Once this is installed you will NEED to create a symlink as the vagrant vmware utility etc.. assumes that vmware is installed in a specific directory and the tech preview is installed in a different one.

ln -s /Applications/VMWare\ Fusion\ Tech\ Preview.app /Applications/VMWare\ Fusion.app

Installing Vagrant VMWare provider

It requires two steps. This is detailed in the documentation but follow the steps below:

It is now available via brew, so you can do

brew install --cask vagrant-vmware-utility

Otherwise, go to Vagrant vmware Utility and download the binary and install it. The direct link is vagrant-vmware-utility_1.0.21_x86_64.dmg.

It needs to be version 1.0.21

Next install the provider:

vagrant plugin install vagrant-vmware-desktop

Create a Vagrant file

Create a file Vagrantfile

Vagrant.configure("2") do |config|
  config.vm.box = "spox/ubuntu-arm"
  config.vm.box_version = "1.0.0"
end

Run vagrant

vagrant up

and then

vagrant ssh

Hopefully this should work and you should find yourself with mostly everything working.

Troubleshooting

I have observed various issues that makes the whole experience unstable or creating conflicts.

"Vagrant encountered an unexpected communications error" after an os update

Everytime macOS install a software update, this will break your vmware utility driver. You will need to reinstall the vmware utility driver as documented above.

Vagrant encountered an unexpected communications error with the
Vagrant VMware Utility driver. Please try to run the command
again. If this error persists, please contact support@hashicorp.com

vagrant 2.2.19 is broken

I have had issues trying to run vagrant 2.2.19. Use 2.3.0

vagrant-vmware provider forwarded ports bound

If you declare forwarding port on your box, for some reasons the provider persists them and keeps it bound in LISTEN state even after you do a vagrant halt.

You can check this using something like:

sudo lsof -i -P | grep LISTEN | grep 'vagrant-v'

You can see the ports are stored in:

cat /opt/vagrant-vmware-desktop/settings/portforwarding.json

To stop the provider use:

sudo launchctl unload -w /Library/LaunchDaemons/com.vagrant.vagrant-vmware-utility.plist

To start it again, use load instead of unload.

nfs exports conflicts

If your vagrant box uses nfs, it seems to pollute the /etc/exports file with duplicate or stale entries over time which will cause vagrant to get angry at you at some point. You may need to prune the entries. It seems related to vagrant#11418

This is an excellent guide. Thanks. By the way, the latest vagrant - 2.2.19 does work on my side. You probably need to do vagrant plugin expunge --reinstall to fix the potential issue.

@fredngo even I am facing a similar issue, I am unable to fix it. Destroying & Rebuilding is the only solution for me now.

Btw, I am using bytesguy/ubuntu-server-20.04-arm64 box

@swateek

Yes, I am also using bytesguy/ubuntu-server-20.04-arm64 box.

I've been able to get this to work:

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  ...
  config.vm.network 'private_network', ip: '172.16.221.111'
  ...
end

But occasionally I also still have to destroy and rebuild. There are definitely some stability issues.

Hi All,

I have installed Vmware fusion tech and vagrant (version 2.2.18) still not able to up it. I am getting the below error.

Vagrant encountered an unexpected communications error with the
Vagrant VMware Utility driver. Please try to run the command
again. If this error persists, please contact support@hashicorp.com
amansrivastava@Amans-MacBook-Air amansrivastava %

It was written to use version 2.2.18 but sadly its not working for me. Can someone please look into this, it will be great help for me.

@aman09-git Check this: hashicorp/vagrant#12695 (comment)

I am not able to create the vagrant file by the command below:

Vagrant.configure("2") do |config|
config.vm.box = "spox/ubuntu-arm"
config.vm.box_version = "1.0.0"
end

See the error message below when I run the above command

Vagrant.configure("2") do |config|
-bash: syntax error near unexpected token `"2"'
A-MBP:~ aali$ config.vm.box = "bento/ubuntu-20.04-arm64"
-bash: config.vm.box: command not found
A-MBP:~ aali$ config.vm.box_version = "202112.19.0"
-bash: config.vm.box_version: command not found
A-MBP:~ aali$ end

@abbasali31 you cannot paste this in the shell and expect to tp create a Vagrantfile.

Please paste the snippet in a file and name it Vagrantfile and use vagrant up

Thank you Swateek! I am not very experience with bash. Thank you ! It

Any ideas folks?

➜  vagrant vagrant up
Bringing machine 'focal64' up with 'vmware_desktop' provider...
==> focal64: Checking if box 'bento/ubuntu-22.04' version '202206.13.0' is up to date...
==> focal64: Verifying vmnet devices are healthy...
==> focal64: Preparing network adapters...
==> focal64: Starting the VMware VM...
An error occurred while executing `vmrun`, a utility for controlling
VMware machines. The command and output are below:

Command: ["start", "/Users/adminuser/Documents/Nordcloud/vagrant/.vagrant/machines/focal64/vmware_desktop/7c2dffe5-2cbc-4a38-8087-ec98a30da515/ubuntu-22.04-amd64.vmx", "nogui", {:notify=>[:stdout, :stderr], :timeout=>45}]

Stdout: 2022-07-04T15:47:36.391| ServiceImpl_Opener: PID 56917
Error: The operation was canceled

Stderr: 

and here's my Vagrantfile

Vagrant.configure("2") do |config|

  config.vm.provider :vmware_desktop do |vmware|
    vmware.vmx["ethernet0.pcislotnumber"] = "160"
  end
  
  config.vm.define "focal64" do |focal64|
    # VM-1 Ubuntu 22.04
    focal64.vm.box = "bento/ubuntu-22.04"
    focal64.vm.box_version = '>=202206.13.0'
    focal64.vm.hostname = 'linux'
    focal64.vm.network "private_network", ip: "192.168.2.10"
    focal64.vm.synced_folder "/Users/adminuser/Documents/nc/vagrant/vagrant_data/focal64", "/vagrant_data"
  end
end

Thank you very much!

I'm getting an error similar to the one reported by @825i

$ vagrant -v
Vagrant 2.2.18

$ ls /Applications/VMWare\ Fusion.app
Contents

$ /opt/vagrant-vmware-desktop/bin/vagrant-vmware-utility -v
1.0.21

$ vagrant plugin list
vagrant-vmware-desktop (3.0.1, global)

$ cat Vagrantfile 
Vagrant.configure("2") do |config|
  config.vm.box = "spox/ubuntu-arm"
  config.vm.box_version = "1.0.0"
end

$ vagrant up
Bringing machine 'default' up with 'vmware_desktop' provider...
==> default: Checking if box 'spox/ubuntu-arm' version '1.0.0' is up to date...
==> default: Verifying vmnet devices are healthy...
==> default: Preparing network adapters...
==> default: Starting the VMware VM...
An error occurred while executing `vmrun`, a utility for controlling
VMware machines. The command and output are below:

Command: ["start", "/Users/james/git/code/vagrant-vmware/.vagrant/machines/default/vmware_desktop/3742772b-9aa9-4eef-a984-56be119670be/Other Linux 5.x kernel 64-bit Arm.vmx", "nogui", {:notify=>[:stdout, :stderr], :timeout=>45}]

Stdout: 2022-07-30T19:21:14.422| ServiceImpl_Opener: PID 2044
Error: The operation was canceled

Stderr: 

and my VMWare Fusion version:

F.Y.I. you can also install the VMware Fusion Tech Preview as a homebrew cask brew install vmware-fusion-tech-preview

For those getting an error like:

An error occurred while executing `vmrun`, a utility for controlling
VMware machines. The command and output are below:

Command: ["start", "/Users/james/git/code/vagrant-vmware/.vagrant/machines/default/vmware_desktop/3742772b-9aa9-4eef-a984-56be119670be/Other Linux 5.x kernel 64-bit Arm.vmx", "nogui", {:notify=>[:stdout, :stderr], :timeout=>45}]

Stdout: 2022-07-30T19:21:14.422| ServiceImpl_Opener: PID 2044
Error: The operation was canceled

Stderr: 

Try running w/ GUI enabled, like:

config.vm.provider :vmware_fusion do |v, o|
  v.gui = true
  # ...other config...
end

Just saying that Canonical's Multipass works on Apple M1 with no hacks required, does basically the same stuff (but worse), and is easy to set up

Getting the same error as @jtconnor @825i :(

Vagrant Version :

vagrant --version
Vagrant 2.3.0

Fusion soft link :

$ ll /Applications/VMWare\ Fusion.app
$ /Applications/VMWare Fusion.app -> /Applications/VMWare Fusion Tech Preview.app

Vagrant File, even tried @benalavi's solution still throws error :

Vagrant.configure("2") do |config|
  #config.vm.box = "nickschuetz/ubuntu-21.10-arm64"
  #config.vm.provider :vmware_fusion do |v, o|
  #  v.gui = true
  # end
	config.vm.box = "spox/ubuntu-arm"
	config.vm.box_version = "1.0.0"
end

Error:

An error occurred while executing `vmrun`, a utility for controlling
VMware machines. The command and output are below:

Command: ["start", "/Users/rushi/Documents/MS_VT/FALL22/LKP/vagrant/.vagrant/machines/default/vmware_desktop/e0ad98a9-d633-482c-a98a-13540e4eff4e/box.vmx", "nogui", {:notify=>[:stdout, :stderr], :timeout=>45}]

Stdout: 2022-08-25T22:24:17.374| ServiceImpl_Opener: PID 35755
Error: The operation was canceled

Stderr:

Well what fixed for me is, I tried running vmrun utility manually with command to the file vbox created which I picked up from the output. Now from second time, vagrant up also works

After upgrading to Monterey 12.5.1, I'm experiencing a problem similary aman09-git, but none of the previous fixes seem to work. The error is:

Vagrant encountered an unexpected communications error with the
Vagrant VMware Utility driver. Please try to run the command
again. If this error persists, please contact support@hashicorp.com

I'm fairly inexperienced when it comes the inner workings of Vagrant, but I have followed all of the steps laid out above and I've even uninstalled and repeated installation 4 or 5 times just to be certain with no success.

Is anyone else seeing this issue post upgrade on an M1 Mac?

Well what fixed for me is, I tried running vmrun utility manually with command to the file vbox created which I picked up from the output. Now from second time, vagrant up also works

Thank you, I had exactly your same problem and I managed to solve it running the vmrun utility manually.

More specifically, I executed the following command: vmrun -T ws start "path/to/file.vmx". In the error's message is also mentioned the option nogui, but I didn't try to add it.

---

M1 Pro Mac with Monterey 12.5.1, Vagrant 2.3.0 and bytesguy/ubuntu-server-20.04-arm64 box.

Hello,
I tried to build my own version of CentOS Stream 9 Arm vagrant box. It works on VMWare, but, when running it from Vagrant, I get stuck on:

default: Waiting for the VM to receive an address

I tried most of the solutions above.

Vagrant file:

Vagrant.configure("2") do |config|
  config.vm.box = "vmware-centos9-arm64"
  config.vm.provider :vmware_desktop do |vmware|
    vmware.gui = true
    vmware.linked_clone = false
    vmware.cpus = 2
  end
end

Logs:

DEBUG vmware_driver: Trying to get MAC address for ethernet0
DEBUG vmware_driver: No explicitly set MAC, looking or auto-generated one...
DEBUG vmware_driver:  -- MAC: XX:XX:XX:XX:XX:XX
 INFO vmware_driver: Reading DHCP lease for 'XX:XX:XX:XX:XX:XX' on 'vmnet8'
DEBUG vagrant_utility: request METHOD=GET PATH=/vmnet/vmnet8/dhcplease/XX:XX:XX:XX:XX:XX RESPONSE=400 ERROR=No entry found for MAC XX:XX:XX:XX:XX:XX
 INFO vmware_driver: Trying vmrun getGuestIPAddress...
 INFO subprocess: Starting process: ["/Applications/VMware Fusion.app/Contents/Library/vmrun", "getGuestIPAddress", "/Users/home/Virtual Machines.localized/vmware-centos9-arm64.vmwarevm/test/.vagrant/machines/default/vmware_desktop/a4fd56be-1a1d-4888-9687-c9764ccc1e5f/vmware-centos9-arm64.vmx"]
DEBUG subprocess: Command not in installer, not touching env vars.
 INFO subprocess: Command not in installer, restoring original environment...
DEBUG subprocess: Selecting on IO
DEBUG subprocess: Waiting for process to exit. Remaining to timeout: 32000
DEBUG subprocess: Exit status: 255
 INFO vmware_driver: vmrun getGuestIPAddress failed: VMRunError

Whenever I manually run vmrun getGuestIPAddress on the base VM (directly run from vmware), it works, I get the corresponding IP. However, whenever I run it on the VM started with vagrant, I get above error.

Whenever I run your other arm boxes (https://app.vagrantup.com/bytesguy/boxes/ubuntu-server-22.04-arm64), it works without any issue (just need to run with the GUI, otherwise it hangs. However, apart the gui issue, with ubuntu it works).

Do you encounter the same issue ? Any idea on how to fix it/debug it ? Did you manage to start a centos stream 9 box ?

Thanks for all the work, it's really useful 😇

Versions:

VMWare tech fusion preview (current latest version)
vagrant-vmware-desktop (3.0.1, global)

@grouville It's probably the Linux Kernel. Check the links I provided here: hashicorp/vagrant#12695 (comment)

Vagrant encountered an unexpected communications error with the
Vagrant VMware Utility driver. Please try to run the command
again. If this error persists, please contact support@hashicorp.com
Is anyone else seeing this issue post upgrade on an M1 Mac?

@certaincents I added a note on the troubleshooting section. Any macOS update will break. After each update, you will need to reinstall vmware utility driver and this should solve it.

Worked perfectly! Thank you so much!

I run the vagrant up with the following vagrant file,

Vagrant.configure("2") do |config|
  config.vm.box = "bytesguy/ubuntu-server-20.04-arm64"
  config.vm.box_version = "1.0.0"
  config.vm.provider :vmware_desktop do |vmware|
    vmware.gui = true
    vmware.cpus = 2
  end
end

But I got the hanging issue with the following message. I tried different options and searched for the solution for a couple of weeks with no luck. Any suggestion would be much appreciated.

vagrant up
Bringing machine 'default' up with 'vmware_desktop' provider...
==> default: Cloning VMware VM: 'bytesguy/ubuntu-server-20.04-arm64'. This can take some time...
==> default: Checking if box 'bytesguy/ubuntu-server-20.04-arm64' version '1.0.0' is up to date...
==> default: Verifying vmnet devices are healthy...
==> default: Preparing network adapters...
WARNING: The VMX file for this box contains a setting that is automatically overwritten by Vagrant
WARNING: when started. Vagrant will stop overwriting this setting in an upcoming release which may
WARNING: prevent proper networking setup. Below is the detected VMX setting:
WARNING: 
WARNING:   ethernet0.pcislotnumber = "33"
WARNING: 
WARNING: If networking fails to properly configure, it may require this VMX setting. It can be manually
WARNING: applied via the Vagrantfile:
WARNING: 
WARNING:   Vagrant.configure(2) do |config|
WARNING:     config.vm.provider :vmware_desktop do |vmware|
WARNING:       vmware.vmx["ethernet0.pcislotnumber"] = "33"
WARNING:     end
WARNING:   end
WARNING: 
WARNING: For more information: https://www.vagrantup.com/docs/vmware/boxes.html#vmx-allowlisting
==> default: Starting the VMware VM...
==> default: Waiting for the VM to receive an address...

The versions I am using are,

$vagrant --version
Vagrant 2.3.1
$ vagrant plugin list
vagrant-vmware-desktop (3.0.1, global)
$ /opt/vagrant-vmware-desktop/bin/vagrant-vmware-utility -v
1.0.21

I tried with a fedora image and got the same hanging issue.

Vagrant.configure("2") do |config|
  config.vm.box = "jacobw/fedora35-arm64"
  config.vm.box_version = "1.0.0"
  config.vm.provider :vmware_desktop do |vmware|
    vmware.gui = true
    vmware.cpus = 2
  end
end

$ vagrant up
Bringing machine 'default' up with 'vmware_desktop' provider...
==> default: Checking if box 'jacobw/fedora35-arm64' version '3.6.8' is up to date...
==> default: Verifying vmnet devices are healthy...
==> default: Preparing network adapters...
==> default: Starting the VMware VM...
==> default: Waiting for the VM to receive an address...

@michael-shl your problem looks like the issue I documented on my new gist Vagrant and VMWare Fusion 13 Player on Apple M1 Pro). The key might be to add

vmware.vmx["ethernet0.virtualdev"] = "vmxnet3"

in your Vagrantfile

works... 😳

it was working but then it [vagrant up] stopped detecting the provider...

% vagrant up
Bringing machine 'default' up with 'vmware_desktop' provider...
Vagrant could not detect VirtualBox! Make sure VirtualBox is properly installed.
Vagrant uses the `VBoxManage` binary that ships with VirtualBox, and requires
this to be available on the PATH. If VirtualBox is installed, please find the
`VBoxManage` binary and add it to the PATH environmental variable.

the $PATH variable contains the VMWare fusion as it used to when it was working... I don't understand...

% echo $PATH
/Users/administrateur/miniconda3/bin:/Users/administrateur/miniconda3/condabin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware Fusion Tech Preview.app/Contents/Public:/Library/Apple/usr/bin

Direct download link (without registration)

2023-04-25 | 13.0.2 | 672.09 MB | dmg

wget https://download3.vmware.com/software/FUS-1302/VMware-Fusion-13.0.2-21581413_universal.dmg

Brew formulae is available to explore ( No Registration required for download)

brew install --cask vmware-fusion

Formulae link: https://formulae.brew.sh/cask/vmware-fusion

Direct download link (without registration)

2023-04-25 | 13.0.2 | 672.09 MB | dmg

wget https://download3.vmware.com/software/FUS-1302/VMware-Fusion-13.0.2-21581413_universal.dmg

Thanks, bypassed signup! 👍

There's a new arm64 version of the image -

  config.vm.box = "bento/ubuntu-22.04-arm64"