Skip to content

Instantly share code, notes, and snippets.

@gitkodak
Last active May 12, 2022 19:39
Show Gist options
  • Save gitkodak/68de4aa4693572c4632effc710d1e118 to your computer and use it in GitHub Desktop.
Save gitkodak/68de4aa4693572c4632effc710d1e118 to your computer and use it in GitHub Desktop.
Mapping vmware vsphere disks to linux devices with vsphere API
I'm trying to automatically add disks to Linux VMs on a vmware hypervisor through the vsphere API.
This isn't as simple as it sounds. Linux SCSI devices, as you probably know, use the format:
CONTROLLER:BUS:SCSI_ID:LUN
for addressing SCSI disks. For example:
[3:0:2:0] disk VMware Virtual disk 1.0 /dev/sdk
The problem is that the *controller* number doesn't necessarily match between Linux and vmware. A lot of tutorials you'll find online assume a single controler, or controllers that are added in a certain order. In my case I've have systems that have the IDE controller show up both before and after the PVSCSI controllers making the PVSCSI controller 0 or 2. Or maybe we have a lot of disks on a box and we have more than one PVSCSI controller. Whatever the case, I want to absolutely nail the disk I just added to a specific controller and SCSI_ID.
I've used Dann Bohn's example python script (Github: https://github.com/whereismyjetpack) to add the disk. The script finds the first scsi controller, then loops through the devices on that controller until it finds an unused device. Then it runs reconfigurevm_task() using a generated spec.
The only thing I get back from this method is the controller object, and this object contains the physical slot number of the controller. Since I know what the disk ID is (I gave it to the reconfigure task) all I need to do is map the physical slot to the logical scsi device on the Linux box.
First, I can find the PCI slot from the physical slot using lspci. lspci -vmm will produce output similar to (this is trimmed):
Slot: 0b:00.0
Class: Serial Attached SCSI controller
Vendor: VMware
Device: PVSCSI SCSI Controller
SVendor: VMware
SDevice: PVSCSI SCSI Controller
PhySlot: 192
Rev: 02
Assuming 192 is the physical slot I care about, I can see that the associated PCI slot is 0b:00.0. Armed with that information I can use lshw. "lshw -c storage" gives me (again, trimmed):
description: Serial Attached SCSI controller
product: PVSCSI SCSI Controller
vendor: VMware
physical id: 0
bus info: pci@0000:0b:00.0
logical name: scsi3
I can see that the PCI slot (ob:00.0) is associated with the logical controller scsi3.
Now it's simply a matter of doing lsscsi and finding the disk with controller 3 and the ID of whatever disk I added back at the begining. Let's say it's disk ID 1, lsscsi gives me:
[2:0:14:0] disk VMware Virtual disk 1.0 /dev/sdn
[2:0:15:0] disk VMware Virtual disk 1.0 /dev/sdo
[3:0:0:0] disk VMware Virtual disk 1.0 /dev/sdp
[3:0:1:0] disk VMware Virtual disk 1.0 /dev/sdq
Now I know that the controller on slot 192 with the scsi ID of 1 is /dev/sdq.
This is easy enough to automate. Combine this and Dann's script I can construct a command that will add, format, mount and set permissions all at once.
I'm including a script below that I just knocked out today. It's completely unpolished, very little sanity checking is done. Basically I need to comptely rewrite it, but I wanted to get it out before I forgot about it.
#!/bin/bash
#
# this script will return the device name (sans /dev)
# when given the physical slot number of a SCSI controller
# (such as is returned by the vsphere reconfigure_vm() task
# when adding a disk) and given the ID of the disk.
#
# for example, given a physical slot number of 224 and id 2
# and assuming that physical slot 224 maps to controller 3,
# input of this script:
#
# ./scriptname 224 3
# would return something like:
# sdk
#
# this is useful because the vsphere api will only return the slot
# number and we need to map an added disk to a physical device on
# linux
set -o pipefail
SLOT_NUMBER="$1"
SCSI_ID="$2"
PATH="$PATH:/usr/sbin:/sbin"
if [[ ! $UID -eq 0 ]] ; then
echo Must be run as root, failing.
exit 99
fi
PCISLOT=$(lspci -vmm \
| grep -E -B 6 "PhySlot:\s+$SLOT_NUMBER" \
| head | grep -E "^Slot:" \
| sed -e 's/\t/ /g'| awk {'print $2'})
if [[ "${PIPESTATUS[@]}" =~ [^0\ ] ]] ; then
echo Unable to map the physical slot to the PCI address, failing.
exit 1
fi
CONTROLLER_NUMBER=$(lshw -quiet -c storage \
| grep -A1 $PCISLOT | grep "logical name:" \
| sed -e 's/ //g' | awk -F":" {'print $2'} \
| sed -e 's/scsi//')
if [[ "${PIPESTATUS[@]}" =~ [^0\ ] ]] ; then
echo Unable to map the PCI address to the controller number, failing.
exit 2
fi
DEVICE_NAME=$(lsscsi | grep -E "${CONTROLLER_NUMBER}:0:${SCSI_ID}:0")
if [[ ! "${PIPESTATUS[@]}" =~ [^0\ ] ]] ; then
echo $DEVICE_NAME | awk -F"/" {'print $3'}
else
echo Unable to map disk $SCSI_ID on controller $CONTROLLER_NUMBER to device, failing.
exit 3
fi
---end script---
This is an example object as returned by vsphere after adding a disk:
(vim.vm.device.ParaVirtualSCSIController) {
dynamicType = <unset>,
dynamicProperty = (vmodl.DynamicProperty) [],
key = 1001,
deviceInfo = (vim.Description) {
dynamicType = <unset>,
dynamicProperty = (vmodl.DynamicProperty) [],
label = 'SCSI controller 1',
summary = 'VMware paravirtual SCSI'
},
backing = <unset>,
connectable = <unset>,
slotInfo = (vim.vm.device.VirtualDevice.PciBusSlotInfo) {
dynamicType = <unset>,
dynamicProperty = (vmodl.DynamicProperty) [],
pciSlotNumber = 224
},
controllerKey = 100,
unitNumber = 4,
busNumber = 1,
device = (int) [
2016,
2017
],
hotAddRemove = true,
sharedBus = 'noSharing',
scsiCtlrUnitNumber = 7
}
and here's the spec we send to ReconfigureVM_Task():
(vim.vm.device.VirtualDeviceSpec) {
dynamicType = <unset>,
dynamicProperty = (vmodl.DynamicProperty) [],
operation = 'add',
fileOperation = 'create',
device = (vim.vm.device.VirtualDisk) {
dynamicType = <unset>,
dynamicProperty = (vmodl.DynamicProperty) [],
key = 0,
deviceInfo = <unset>,
backing = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) {
dynamicType = <unset>,
dynamicProperty = (vmodl.DynamicProperty) [],
fileName = '',
datastore = <unset>,
backingObjectId = <unset>,
diskMode = 'persistent',
split = <unset>,
writeThrough = <unset>,
thinProvisioned = <unset>,
eagerlyScrub = <unset>,
uuid = <unset>,
contentId = <unset>,
changeId = <unset>,
parent = <unset>,
deltaDiskFormat = <unset>,
digestEnabled = <unset>,
deltaGrainSize = <unset>
},
connectable = <unset>,
slotInfo = <unset>,
controllerKey = 1001,
unitNumber = 2,
capacityInKB = 1048576,
capacityInBytes = <unset>,
shares = <unset>,
storageIOAllocation = <unset>,
diskObjectId = <unset>,
vFlashCacheConfigInfo = <unset>
},
profile = (vim.vm.ProfileSpec) []
}
So we use "pciSlotNumber" (224) from the returned controller object and the "unitNumber" that we put in the spec with the script above (run on the guest -- hopefully that's obvious) to get the device:
[root@stllxts6 ~]# ./get_device_from_slot_and_id 224 2
sdk
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment