Skip to content

Instantly share code, notes, and snippets.

@steve-todorov

steve-todorov/Simulating Bare Metal with VirtualBMC.md

Forked from williamcaban/Simulating Bare Metal with VirtualBMC.md
Created August 28, 2021 10:45
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save steve-todorov/e6370dd490b6fbd6c60158ca32763f12 to your computer and use it in GitHub Desktop.
Save steve-todorov/e6370dd490b6fbd6c60158ca32763f12 to your computer and use it in GitHub Desktop.
Download ZIP
Bare Metal IPMI for VMs with Virtual BMC
Raw
Simulating Bare Metal with VirtualBMC.md

Simulate Bare-Metal IPMI for VMs in RHEL8 Libvirt

  • Install the Virtual BMC package in the machine to use as the vBMC server. Note1: When using OpenStack repos the python3-virtualbmc RPM might be available. These instrucctions do not use that package.

    pip3 install virtualbmc
    • The vBMC server can be running in any machine. When using a remote libvirt server it is recommended to have passwordless authentication from the vBMC server to the libvirt machines.
    • For this document the vBMC Server export vBMCServerIP=192.168.1.13

  • For each libvirt server, identify the exact names of the VMs to configure in the vBMC.

    # virsh list --all
 Id    Name                           State
----------------------------------------------------
 7     rhcos-aio                      shut off
 8     ocp4-master-0                  shut off
 9     ocp4-master-1                  shut off
 10    ocp4-master-2                  shut off
 11    ocp4-worker-0                  shut off
 12    ocp4-worker-1                  shut off
 13    ocp4-worker-2                  shut off
 14    ocp4-bootstrap                shut off

  • The format for the vBMC server is as follows:

    $ vbmc add -h
usage: vbmc add [-h] [--username USERNAME] [--password PASSWORD] [--port PORT]
                [--address ADDRESS] [--libvirt-uri LIBVIRT_URI]
                [--libvirt-sasl-username LIBVIRT_SASL_USERNAME]
                [--libvirt-sasl-password LIBVIRT_SASL_PASSWORD]
                domain_name

Create a new BMC for a virtual machine instance

positional arguments:
domain_name           The name of the virtual machine

optional arguments:
-h, --help            show this help message and exit
--username USERNAME   The BMC username; defaults to "admin"
--password PASSWORD   The BMC password; defaults to "password"
--port PORT           Port to listen on; defaults to 623
--address ADDRESS     The address to bind to (IPv4 and IPv6 are supported);
                        defaults to ::
--libvirt-uri LIBVIRT_URI
                        The libvirt URI; defaults to "qemu:///system"
--libvirt-sasl-username LIBVIRT_SASL_USERNAME
                        The libvirt SASL username; defaults to None
--libvirt-sasl-password LIBVIRT_SASL_PASSWORD
                        The libvirt SASL password; defaults to None
    • An essential parameters is the libvirt-uri:
      • When using local libvirt server the default syntax is --libvirt-uri qemu:///system <name-of-vm>
      • When using remote libvirt server with SSH key authentication use the syntax --libvirt-uri qemu+ssh://root@<remote-libvirt-server>/system <name-of-vm>
    • The --port parameter must be different for each VM using the same vBMC server IP ${vBMCServerIP}. To map multiple vBMC to the same IP address, increase the listening port for each VM.

  • Example using local libvirt:

# export vBMCServerIP=192.168.1.13

vbmc add --username admin --password password --port 6230 --address ${vBMCServerIP} --libvirt-uri qemu:///system ocp4-bootstrap
vbmc add --username admin --password password --port 6231 --address ${vBMCServerIP} --libvirt-uri qemu:///system ocp4-master-0
vbmc add --username admin --password password --port 6232 --address ${vBMCServerIP} --libvirt-uri qemu:///system ocp4-master-1
vbmc add --username admin --password password --port 6233 --address ${vBMCServerIP} --libvirt-uri qemu:///system ocp4-master-2
vbmc add --username admin --password password --port 6234 --address ${vBMCServerIP} --libvirt-uri qemu:///system ocp4-worker-0
vbmc add --username admin --password password --port 6235 --address ${vBMCServerIP} --libvirt-uri qemu:///system ocp4-worker-1
vbmc add --username admin --password password --port 6236 --address ${vBMCServerIP} --libvirt-uri qemu:///system ocp4-worker-2

# vbmc list
+----------------+--------+--------------+------+
| Domain name    | Status | Address      | Port |
+----------------+--------+--------------+------+
| ocp4-bootstrap | down   | 192.168.1.13 | 6230 |
| ocp4-master-0  | down   | 192.168.1.13 | 6231 |
| ocp4-master-1  | down   | 192.168.1.13 | 6232 |
| ocp4-master-2  | down   | 192.168.1.13 | 6233 |
| ocp4-worker-0  | down   | 192.168.1.13 | 6234 |
| ocp4-worker-1  | down   | 192.168.1.13 | 6235 |
| ocp4-worker-2  | down   | 192.168.1.13 | 6236 |
+----------------+--------+--------------+------+

firewall-cmd --zone=public --permanent --add-port=6230-6250/udp
firewall-cmd --reload

Note: Increase the listening port to map multiple vBMC to the same address

  • Using vBMC to start/stop a VM
# vbmc add --username admin --password password --port 6250 --address ${vBMCServerIP} --libvirt-uri qemu:///system rhcos-aio

# vbmc start rhcos-aio
2020-01-02 10:58:02,223.223 10797 INFO VirtualBMC [-] Started vBMC instance for domain rhcos-aio

# vbmc list
+----------------+---------+--------------+------+
| Domain name    | Status  | Address      | Port |
+----------------+---------+--------------+------+
| rhcos-aio      | running | 192.168.1.13 | 6250 |
+----------------+---------+--------------+------+

# vbmc stop rhcos-aio
2020-01-02 10:58:14,254.254 10797 INFO VirtualBMC [-] Terminated vBMC instance for domain rhcos-aio

#vbmc list
+----------------+--------+--------------+------+
| Domain name    | Status | Address      | Port |
+----------------+--------+--------------+------+
| rhcos-aio      | down   | 192.168.1.13 | 6250 |
+----------------+--------+--------------+------+

Using IPMITOOL

  • Using ipmitool to start/stop a VM
firewall-cmd --zone=public --permanent --add-port=6250/udp
firewall-cmd --reload

export vBMCServerIP=192.168.1.13

vbmc add --username admin --password password --port 6250 --address ${vBMCServerIP} --libvirt-uri qemu:///system rhcos-aio


ipmitool -vv -I lanplus -H ${vBMCServerIP} -p 6250 -U admin -P password chassis status

ipmitool -I lanplus -H ${vBMCServerIP} -p6250 -Uadmin -Ppassword chassis power on

ipmitool -I lanplus -H ${vBMCServerIP} -p6250 -Uadmin -Ppassword chassis status

ipmitool -I lanplus -H ${vBMCServerIP} -p6250 -Uadmin -Ppassword chassis power off

Sample virtualbmc.conf

Using a custom virtualbmc.conf path export VIRTUALBMC_CONFIG=~/vbmc/virtualbmc.conf

Sample virtualbmc.conf

[default]
show_passwords = true
# default 50891
server_port = 12345
# server_spawn_wait default 3000 milliseconds
server_spawn_wait = 3000
# server_response_timeout default 5000 milliseconds
server_response_timeout = 5000
config_dir = /root/vbmc
pid_file = /root/vbmc/vbmc.pid

[log]
debug = true 
logfile = /root/vbmc/vbmc.log

[ipmi]
# default 1 seconds
session_timeout = 30

Running vbmcd as foreground service (for troubleshooting)

vbmcd --foreground

Testing OpenShift Bare-Metal IPI

https://github.com/openshift-kni/baremetal-deploy/blob/master/install-steps.md

Credits

  • Thanks to Benjamin Schmaus for the original pointers on how to configure virtual BMC for VMs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment