Skip to content

Instantly share code, notes, and snippets.

@williamcaban

williamcaban/Simulating Bare Metal with VirtualBMC.md

Last active April 16, 2024 19:14
Show Gist options
  • Star 23 You must be signed in to star a gist
  • Fork 5 You must be signed in to fork a gist
  • Save williamcaban/aba796f856264799326d554ac11a4a66 to your computer and use it in GitHub Desktop.
Save williamcaban/aba796f856264799326d554ac11a4a66 to your computer and use it in GitHub Desktop.
Download ZIP
Bare Metal IPMI for VMs with Virtual BMC
Raw
Simulating Bare Metal with VirtualBMC.md

Simulate Bare-Metal IPMI for VMs in RHEL8 Libvirt

  • Install the Virtual BMC package in the machine to use as the vBMC server. Note1: When using OpenStack repos the python3-virtualbmc RPM might be available. These instrucctions do not use that package.

    pip3 install virtualbmc
    • The vBMC server can be running in any machine. When using a remote libvirt server it is recommended to have passwordless authentication from the vBMC server to the libvirt machines.
    • For this document the vBMC Server export vBMCServerIP=192.168.1.13

  • For each libvirt server, identify the exact names of the VMs to configure in the vBMC.

    # virsh list --all
 Id    Name                           State
----------------------------------------------------
 7     rhcos-aio                      shut off
 8     ocp4-master-0                  shut off
 9     ocp4-master-1                  shut off
 10    ocp4-master-2                  shut off
 11    ocp4-worker-0                  shut off
 12    ocp4-worker-1                  shut off
 13    ocp4-worker-2                  shut off
 14    ocp4-bootstrap                shut off

  • The format for the vBMC server is as follows:

    $ vbmc add -h
usage: vbmc add [-h] [--username USERNAME] [--password PASSWORD] [--port PORT]
                [--address ADDRESS] [--libvirt-uri LIBVIRT_URI]
                [--libvirt-sasl-username LIBVIRT_SASL_USERNAME]
                [--libvirt-sasl-password LIBVIRT_SASL_PASSWORD]
                domain_name

Create a new BMC for a virtual machine instance

positional arguments:
domain_name           The name of the virtual machine

optional arguments:
-h, --help            show this help message and exit
--username USERNAME   The BMC username; defaults to "admin"
--password PASSWORD   The BMC password; defaults to "password"
--port PORT           Port to listen on; defaults to 623
--address ADDRESS     The address to bind to (IPv4 and IPv6 are supported);
                        defaults to ::
--libvirt-uri LIBVIRT_URI
                        The libvirt URI; defaults to "qemu:///system"
--libvirt-sasl-username LIBVIRT_SASL_USERNAME
                        The libvirt SASL username; defaults to None
--libvirt-sasl-password LIBVIRT_SASL_PASSWORD
                        The libvirt SASL password; defaults to None
    • An essential parameters is the libvirt-uri:
      • When using local libvirt server the default syntax is --libvirt-uri qemu:///system <name-of-vm>
      • When using remote libvirt server with SSH key authentication use the syntax --libvirt-uri qemu+ssh://root@<remote-libvirt-server>/system <name-of-vm>
    • The --port parameter must be different for each VM using the same vBMC server IP ${vBMCServerIP}. To map multiple vBMC to the same IP address, increase the listening port for each VM.

  • Example using local libvirt:

# export vBMCServerIP=192.168.1.13

vbmc add --username admin --password password --port 6230 --address ${vBMCServerIP} --libvirt-uri qemu:///system ocp4-bootstrap
vbmc add --username admin --password password --port 6231 --address ${vBMCServerIP} --libvirt-uri qemu:///system ocp4-master-0
vbmc add --username admin --password password --port 6232 --address ${vBMCServerIP} --libvirt-uri qemu:///system ocp4-master-1
vbmc add --username admin --password password --port 6233 --address ${vBMCServerIP} --libvirt-uri qemu:///system ocp4-master-2
vbmc add --username admin --password password --port 6234 --address ${vBMCServerIP} --libvirt-uri qemu:///system ocp4-worker-0
vbmc add --username admin --password password --port 6235 --address ${vBMCServerIP} --libvirt-uri qemu:///system ocp4-worker-1
vbmc add --username admin --password password --port 6236 --address ${vBMCServerIP} --libvirt-uri qemu:///system ocp4-worker-2

# vbmc list
+----------------+--------+--------------+------+
| Domain name    | Status | Address      | Port |
+----------------+--------+--------------+------+
| ocp4-bootstrap | down   | 192.168.1.13 | 6230 |
| ocp4-master-0  | down   | 192.168.1.13 | 6231 |
| ocp4-master-1  | down   | 192.168.1.13 | 6232 |
| ocp4-master-2  | down   | 192.168.1.13 | 6233 |
| ocp4-worker-0  | down   | 192.168.1.13 | 6234 |
| ocp4-worker-1  | down   | 192.168.1.13 | 6235 |
| ocp4-worker-2  | down   | 192.168.1.13 | 6236 |
+----------------+--------+--------------+------+

firewall-cmd --zone=public --permanent --add-port=6230-6250/udp
firewall-cmd --reload

Note: Increase the listening port to map multiple vBMC to the same address

  • Using vBMC to start/stop a VM
# vbmc add --username admin --password password --port 6250 --address ${vBMCServerIP} --libvirt-uri qemu:///system rhcos-aio

# vbmc start rhcos-aio
2020-01-02 10:58:02,223.223 10797 INFO VirtualBMC [-] Started vBMC instance for domain rhcos-aio

# vbmc list
+----------------+---------+--------------+------+
| Domain name    | Status  | Address      | Port |
+----------------+---------+--------------+------+
| rhcos-aio      | running | 192.168.1.13 | 6250 |
+----------------+---------+--------------+------+

# vbmc stop rhcos-aio
2020-01-02 10:58:14,254.254 10797 INFO VirtualBMC [-] Terminated vBMC instance for domain rhcos-aio

#vbmc list
+----------------+--------+--------------+------+
| Domain name    | Status | Address      | Port |
+----------------+--------+--------------+------+
| rhcos-aio      | down   | 192.168.1.13 | 6250 |
+----------------+--------+--------------+------+

Using IPMITOOL

  • Using ipmitool to start/stop a VM
firewall-cmd --zone=public --permanent --add-port=6250/udp
firewall-cmd --reload

export vBMCServerIP=192.168.1.13

vbmc add --username admin --password password --port 6250 --address ${vBMCServerIP} --libvirt-uri qemu:///system rhcos-aio


ipmitool -vv -I lanplus -H ${vBMCServerIP} -p 6250 -U admin -P password chassis status

ipmitool -I lanplus -H ${vBMCServerIP} -p6250 -Uadmin -Ppassword chassis power on

ipmitool -I lanplus -H ${vBMCServerIP} -p6250 -Uadmin -Ppassword chassis status

ipmitool -I lanplus -H ${vBMCServerIP} -p6250 -Uadmin -Ppassword chassis power off

Sample virtualbmc.conf

Using a custom virtualbmc.conf path export VIRTUALBMC_CONFIG=~/vbmc/virtualbmc.conf

Sample virtualbmc.conf

[default]
show_passwords = true
# default 50891
server_port = 12345
# server_spawn_wait default 3000 milliseconds
server_spawn_wait = 3000
# server_response_timeout default 5000 milliseconds
server_response_timeout = 5000
config_dir = /root/vbmc
pid_file = /root/vbmc/vbmc.pid

[log]
debug = true 
logfile = /root/vbmc/vbmc.log

[ipmi]
# default 1 seconds
session_timeout = 30

Running vbmcd as foreground service (for troubleshooting)

vbmcd --foreground

Testing OpenShift Bare-Metal IPI

https://github.com/openshift-kni/baremetal-deploy/blob/master/install-steps.md

Credits

  • Thanks to Benjamin Schmaus for the original pointers on how to configure virtual BMC for VMs.
@bionic-steve
Copy link

thanks, great post.

For vbmc - I used the systemd unit from here : https://www.cloudnull.io/2019/05/vbmc/
I put the config file in /etc/, then made a small change in the unit file :
Environment="VIRTUALBMC_CONFIG=/etc/vbmcd.conf"
ExecStart = /usr/local/bin/vbmcd --foreground

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment