ISO Boot in vSphere Veewee Provider

iso_boot.md

ISO Boot to Remote vSphere Server

Example Veewee definitions utilize ISOs (either full or network install) to initiate the operating system installation. This is straightforward on a local workstation, but requires additional work on remote hypervisors.

Note: The below approach has been tested using CentOS net boot ISOs to execute the installation.

Approach

My approach involved uploading the ISOs to a datastore visible to the target vSphere API endpoint, attaching the ISO as a CD/DVD device to the VM during the create portion of a build, and utilize VNC to issue the boot command.

ISO Upload

To use the ISO image to boot a VM requires uploading the ISO to a datastore visible to the server you intend to build the VM on, and to configure the VM to attach. In manual operation, these steps can be achieved through the vSphere Client application. To automate the process, we can take advantage of the Datastore APIs to upload the ISOs, and use the vSphere APIs to configure our VM to point to that location.

1. Download and verify ISO per the Core provider's build process
2. Add a load_iso method to the VSphere::Box.create call
   1. load_iso calls vSphere APIs using RbVmomi to see if the destination ISO directory exists on the datastore targeted for the build command, creating the folder if it is not already defined
   2. load_iso calls vSphere APIs using RbVmomi to see if the ISO already exists. If the ISO doesn't exist, it uploads the ISO using the vSphere HTTP API for interacting with the datastore API
3. Add a CD/DVD device to the VM when creating, pointing to the uploaded ISO

Note: Current implementation utilizes vmware/rbvmomi, which provides a ruby API allowing uploads from a Datastore object. This implementation depends on curl being available on the host system. vmware/rvc implements this same capability using NET::HTTP. I intend to submit a patch to that bring rbvmomi inline with the rvc approach.

Assumptions and Limitations

• ISOs are uploaded to the datastore to a folder named "ISOs" used for building the given VM (specified as part of the build process)
  • Enhancement - Notionally this could be specified explicitly in whole or part as part of a definition file
• ISOs are validated before upload, but not after.
  • TODO - This probably needs to be modified, assuming there is a reasonable way to check the ISO against a checksum to ensure it doesn't change.
• Loading ISOs as part of the create process feels like a hack, but there didn't appear to be a better place to implement without copying the Core provider's build.rb. definition.verify_iso would make sense, as it already handles ISO download if required, but that class does not have access to the provider to make the remote hypervisor connection.
  • Recommendation - As Veewee front page says, adding new provider's needs some love. Additional hooks could provide a way to allow more flexible extension, or a middleware composition approach ala Vagrant would allow custom recomposition of separate steps in core as desired.

Configure VNC

After the VM is created, we enabled VNC access to the VMs console through VMware. This is implemented by setting extra configuration parameters to the VM's definition. After this configuration is complete, the calls to send the boot sequence mirrors the VMware Fusion provider (along with the same limitations based on the timing of when the keystrokes are provided).

Note: Current implementation adapts the approach used by vmware/rvc to setup VNC and vmware/rbvmomi to interact with the vSphere APIs.

Assumptions and Limitations

• Veewee core VNC support assumes VNC ports are specified as an integer above 5900 (i.e., if the VNC port is 5901, Veewee's code to connect to the port expects to see 1). This behavior required the vSphere provider implementation to calculate the Veewee expected port and replicate for VNC to work.
  • Recommendation - Modify Veewee core VNC code to use the actual port number, rather than a display number relative to 5900.
• VNC support in current implementation is not password protected, and is thus insecure. This behavior works fine in a workstation environment, and may be sufficient in an isolated build environment, but be disabled when not needed. the vSphere provider build.rb disables VNC after the build process completes.

load_iso.rb

def load_iso datastore_name
  env.ui.info "Loading ISO to Host"
  filename = definition.iso_file
  local_path=File.join(env.config.veewee.iso_dir,filename)
  File.exists?(local_path) or fail "ISO does not exist"
  dc = vim.serviceInstance.find_datacenter
  datastore = dc.find_datastore datastore_name
  unless datastore.exists? "isos/"+filename
    unless datastore.exists? "isos/"
      vim.serviceContent.fileManager.MakeDirectory :name => "[#{datastore_name}] isos", :datacenter => dc
    end
    datastore.upload "isos/"+filename, local_path
  end
end

vnc.rb

module Veewee
  module Provider
    module Vsphere
      module BoxCommand

        attr_reader :vnc_port
        attr_reader :vnc_host

        def enable_vnc
          vm = raw
          @vnc_host = reachable_ip raw.collect('runtime.host')[0]
          extraConfig, = raw.collect('config.extraConfig')
          already_enabled = extraConfig.find { |x| x.key == 'RemoteDisplay.vnc.enabled' && x.value.downcase == 'true' }
          if already_enabled
            puts "VNC already enabled"
            real_vnc_port = extraConfig.find { |x| x.key == 'RemoteDisplay.vnc.port' }.value
            #Modify the port to work around the way Veewee's VNC class is implemented
            @vnc_port = real_vnc_port - 5900
          else
            real_vnc_port = unused_vnc_port vnc_host
            #Modify the port to work around the way Veewee's VNC class is implemented
            @vnc_port = real_vnc_port - 5900
            vm.ReconfigVM_Task(:spec => {
              :extraConfig => [
                { :key => 'RemoteDisplay.vnc.enabled', :value => 'true' },
                { :key => 'RemoteDisplay.vnc.port', :value => real_vnc_port.to_s }
              ]
            }).wait_for_completion
          end
        end

        def reachable_ip host
          ips = host.collect('config.network.vnic')[0].map { |x| x.spec.ip.ipAddress }
          ips.find do |x|
            begin
              Timeout.timeout(1) { TCPSocket.new(x, 443).close; true }
            rescue
                false
            end
          end or err("could not find IP for server #{host.name}")
        end

        def unused_vnc_port ip
          10.times do
            port = 5901 + rand(64)
            unused = (TCPSocket.connect(ip, port).close rescue true)
            return port if unused
          end
          err "no unused port found"
        end

        def close_vnc
          vm = raw
          vm.ReconfigVM_Task(:spec => {
            :extraConfig => [
              { :key => 'RemoteDisplay.vnc.enabled', :value => 'false' },
              { :key => 'RemoteDisplay.vnc.password', :value => '' },
              { :key => 'RemoteDisplay.vnc.port', :value => '' }
            ]
          }).wait_for_completion
        end

        def vnc_enabled?
          extraConfig, = raw.collect('config.extraConfig')
          return extraConfig.find { |x| x.key == 'RemoteDisplay.vnc.enabled' && x.value.downcase == 'true' }
        end

      end
    end
  end
end

nice example of setting extraConfig. thanks!