Last active
May 12, 2022 19:39
-
-
Save gitkodak/68de4aa4693572c4632effc710d1e118 to your computer and use it in GitHub Desktop.
Mapping vmware vsphere disks to linux devices with vsphere API
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I'm trying to automatically add disks to Linux VMs on a vmware hypervisor through the vsphere API. | |
This isn't as simple as it sounds. Linux SCSI devices, as you probably know, use the format: | |
CONTROLLER:BUS:SCSI_ID:LUN | |
for addressing SCSI disks. For example: | |
[3:0:2:0] disk VMware Virtual disk 1.0 /dev/sdk | |
The problem is that the *controller* number doesn't necessarily match between Linux and vmware. A lot of tutorials you'll find online assume a single controler, or controllers that are added in a certain order. In my case I've have systems that have the IDE controller show up both before and after the PVSCSI controllers making the PVSCSI controller 0 or 2. Or maybe we have a lot of disks on a box and we have more than one PVSCSI controller. Whatever the case, I want to absolutely nail the disk I just added to a specific controller and SCSI_ID. | |
I've used Dann Bohn's example python script (Github: https://github.com/whereismyjetpack) to add the disk. The script finds the first scsi controller, then loops through the devices on that controller until it finds an unused device. Then it runs reconfigurevm_task() using a generated spec. | |
The only thing I get back from this method is the controller object, and this object contains the physical slot number of the controller. Since I know what the disk ID is (I gave it to the reconfigure task) all I need to do is map the physical slot to the logical scsi device on the Linux box. | |
First, I can find the PCI slot from the physical slot using lspci. lspci -vmm will produce output similar to (this is trimmed): | |
Slot: 0b:00.0 | |
Class: Serial Attached SCSI controller | |
Vendor: VMware | |
Device: PVSCSI SCSI Controller | |
SVendor: VMware | |
SDevice: PVSCSI SCSI Controller | |
PhySlot: 192 | |
Rev: 02 | |
Assuming 192 is the physical slot I care about, I can see that the associated PCI slot is 0b:00.0. Armed with that information I can use lshw. "lshw -c storage" gives me (again, trimmed): | |
description: Serial Attached SCSI controller | |
product: PVSCSI SCSI Controller | |
vendor: VMware | |
physical id: 0 | |
bus info: pci@0000:0b:00.0 | |
logical name: scsi3 | |
I can see that the PCI slot (ob:00.0) is associated with the logical controller scsi3. | |
Now it's simply a matter of doing lsscsi and finding the disk with controller 3 and the ID of whatever disk I added back at the begining. Let's say it's disk ID 1, lsscsi gives me: | |
[2:0:14:0] disk VMware Virtual disk 1.0 /dev/sdn | |
[2:0:15:0] disk VMware Virtual disk 1.0 /dev/sdo | |
[3:0:0:0] disk VMware Virtual disk 1.0 /dev/sdp | |
[3:0:1:0] disk VMware Virtual disk 1.0 /dev/sdq | |
Now I know that the controller on slot 192 with the scsi ID of 1 is /dev/sdq. | |
This is easy enough to automate. Combine this and Dann's script I can construct a command that will add, format, mount and set permissions all at once. | |
I'm including a script below that I just knocked out today. It's completely unpolished, very little sanity checking is done. Basically I need to comptely rewrite it, but I wanted to get it out before I forgot about it. | |
#!/bin/bash | |
# | |
# this script will return the device name (sans /dev) | |
# when given the physical slot number of a SCSI controller | |
# (such as is returned by the vsphere reconfigure_vm() task | |
# when adding a disk) and given the ID of the disk. | |
# | |
# for example, given a physical slot number of 224 and id 2 | |
# and assuming that physical slot 224 maps to controller 3, | |
# input of this script: | |
# | |
# ./scriptname 224 3 | |
# would return something like: | |
# sdk | |
# | |
# this is useful because the vsphere api will only return the slot | |
# number and we need to map an added disk to a physical device on | |
# linux | |
set -o pipefail | |
SLOT_NUMBER="$1" | |
SCSI_ID="$2" | |
PATH="$PATH:/usr/sbin:/sbin" | |
if [[ ! $UID -eq 0 ]] ; then | |
echo Must be run as root, failing. | |
exit 99 | |
fi | |
PCISLOT=$(lspci -vmm \ | |
| grep -E -B 6 "PhySlot:\s+$SLOT_NUMBER" \ | |
| head | grep -E "^Slot:" \ | |
| sed -e 's/\t/ /g'| awk {'print $2'}) | |
if [[ "${PIPESTATUS[@]}" =~ [^0\ ] ]] ; then | |
echo Unable to map the physical slot to the PCI address, failing. | |
exit 1 | |
fi | |
CONTROLLER_NUMBER=$(lshw -quiet -c storage \ | |
| grep -A1 $PCISLOT | grep "logical name:" \ | |
| sed -e 's/ //g' | awk -F":" {'print $2'} \ | |
| sed -e 's/scsi//') | |
if [[ "${PIPESTATUS[@]}" =~ [^0\ ] ]] ; then | |
echo Unable to map the PCI address to the controller number, failing. | |
exit 2 | |
fi | |
DEVICE_NAME=$(lsscsi | grep -E "${CONTROLLER_NUMBER}:0:${SCSI_ID}:0") | |
if [[ ! "${PIPESTATUS[@]}" =~ [^0\ ] ]] ; then | |
echo $DEVICE_NAME | awk -F"/" {'print $3'} | |
else | |
echo Unable to map disk $SCSI_ID on controller $CONTROLLER_NUMBER to device, failing. | |
exit 3 | |
fi | |
---end script--- | |
This is an example object as returned by vsphere after adding a disk: | |
(vim.vm.device.ParaVirtualSCSIController) { | |
dynamicType = <unset>, | |
dynamicProperty = (vmodl.DynamicProperty) [], | |
key = 1001, | |
deviceInfo = (vim.Description) { | |
dynamicType = <unset>, | |
dynamicProperty = (vmodl.DynamicProperty) [], | |
label = 'SCSI controller 1', | |
summary = 'VMware paravirtual SCSI' | |
}, | |
backing = <unset>, | |
connectable = <unset>, | |
slotInfo = (vim.vm.device.VirtualDevice.PciBusSlotInfo) { | |
dynamicType = <unset>, | |
dynamicProperty = (vmodl.DynamicProperty) [], | |
pciSlotNumber = 224 | |
}, | |
controllerKey = 100, | |
unitNumber = 4, | |
busNumber = 1, | |
device = (int) [ | |
2016, | |
2017 | |
], | |
hotAddRemove = true, | |
sharedBus = 'noSharing', | |
scsiCtlrUnitNumber = 7 | |
} | |
and here's the spec we send to ReconfigureVM_Task(): | |
(vim.vm.device.VirtualDeviceSpec) { | |
dynamicType = <unset>, | |
dynamicProperty = (vmodl.DynamicProperty) [], | |
operation = 'add', | |
fileOperation = 'create', | |
device = (vim.vm.device.VirtualDisk) { | |
dynamicType = <unset>, | |
dynamicProperty = (vmodl.DynamicProperty) [], | |
key = 0, | |
deviceInfo = <unset>, | |
backing = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) { | |
dynamicType = <unset>, | |
dynamicProperty = (vmodl.DynamicProperty) [], | |
fileName = '', | |
datastore = <unset>, | |
backingObjectId = <unset>, | |
diskMode = 'persistent', | |
split = <unset>, | |
writeThrough = <unset>, | |
thinProvisioned = <unset>, | |
eagerlyScrub = <unset>, | |
uuid = <unset>, | |
contentId = <unset>, | |
changeId = <unset>, | |
parent = <unset>, | |
deltaDiskFormat = <unset>, | |
digestEnabled = <unset>, | |
deltaGrainSize = <unset> | |
}, | |
connectable = <unset>, | |
slotInfo = <unset>, | |
controllerKey = 1001, | |
unitNumber = 2, | |
capacityInKB = 1048576, | |
capacityInBytes = <unset>, | |
shares = <unset>, | |
storageIOAllocation = <unset>, | |
diskObjectId = <unset>, | |
vFlashCacheConfigInfo = <unset> | |
}, | |
profile = (vim.vm.ProfileSpec) [] | |
} | |
So we use "pciSlotNumber" (224) from the returned controller object and the "unitNumber" that we put in the spec with the script above (run on the guest -- hopefully that's obvious) to get the device: | |
[root@stllxts6 ~]# ./get_device_from_slot_and_id 224 2 | |
sdk |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment