We have a Ruby application running on Ubuntu x86 Linux. Development is done
locally on the developer's machines using Vagrant managed VirtualBox VMs. File
synchronization is handled by vagrant rsync. NFS and the built in VirtualBox
file sync were not performant.
By coincidence all developers use Macs – they choose their hardware. And they wanted to use new Apple Silicon (Arm 64) machines. These new machines can not emulate X86 hardware with reasonable performance.
So we needed to find a path forward.
One consideration was moving to cloud based development. X86 VMs or containers would be spun up for each developer. Github CodeSpaces was one possible solution. Costs aside Codespaces would not preserve databases between sessions. This is a requirement since our data can be complex and time consuming to setup.
Codespaces would have cost about $1200 annually per developer. A DIY EC2 solution would have been around $600 annually per developer plus the unknown costs of supporting this solution.
Since our developers were getting new machines, which is enough of a cost, we decided to look to local solutions.
We only need to support Ubuntu based VMs. So we decided to try out Multipass – https://multipass.run. This is a well supported tool for spinning up Ubuntu virtual machines on Mac, Windows or Linux.
One issue we had was Multipass does not let you assign an IP address to the
machine, or at least not easily. Vagrant/Virtualbox does. As part of our
provisioning phase the IP address is grabbed from the VM and inserted into the
etc/hosts file. This way the machines can be referenced as
dev.foobar.local.
The second issue was the built in file sync was not performance. This was an
issue with VirtualBox as well, which is why we used vagrant rsync.
In researching a sync solution we settled on Mutagen – https://mutagen.io. We have found this tool to be reliable, easy to install and understand.
Our production machines are still X86 and will be for at least another year. We will explore Arm based servers in the future.
To ensure our application is still X86 ready we have a CI server running on X86 Ubuntu that runs the full test suite before deploy.
The final hurdle is with Ruby development there are native Gems (X86 or Arm versions.)
The Gemfile.lock now has three PLATFORMS
PLATFORMS
arm64-linux
ruby
x86_64-linux
This has worked without issues, except for two gems that are not supported on Arm Linux – yet.
Fortunately they are not required for production and are optional for development. But this could be an issue for some applications. We got lucky.
In our Gemfile we optionally include them
install_if -> { RUBY_PLATFORM.include?('x86_64') } do
gem 'sorbet-static-and-runtime'
gem 'tapioca', require: false, group: :development
end
MacOS allows up to 8 cores to be assigned to VMs. The developers all have new M2 Macs with 12 cores. So we gave over 8 cores to the VM.
Our 2019 Intel Macs with 6 cores ran our test suites in 3 minutes. The new M2 machines run our test suites in 1:15 seconds.
This solution is cloud based only the cloud is running on the developer's machine. We could switch the target to a remote machine and enable port forwarding in Mutagen. This gives a nice option to change our development in the future, if we so choose.