Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save AskAlice/ad571425e6c3c81081e7715b7d9d1f2b to your computer and use it in GitHub Desktop.
Save AskAlice/ad571425e6c3c81081e7715b7d9d1f2b to your computer and use it in GitHub Desktop.

Haven't seen anyone do this before, would be intereseted if anyone is able to figure out how to handle multiple VMs. At the moment if you pin with two VMs then shutdown one, the system will no longer pin any CPU. I call this from the base qemu hook located here

#!/bin/bash

set -x
VM_NAME="$1"
VM_ACTION="$2/$3"

# Initialize HOST_CORES with a discrete list of core indices from the range "0-5,18-31"
TOTAL_CORES='0-31'
TOTAL_CORES_MASK=FFFFFFFF
ranges=$4
IFS=',' read -r -a range_parts <<< "$ranges"
HOST_CORES=()
for range in "${range_parts[@]}"; do
    if [[ $range =~ ^([0-9]+)-([0-9]+)$ ]]; then
        for ((i=${BASH_REMATCH[1]}; i<=${BASH_REMATCH[2]}; i++)); do
            HOST_CORES+=($i)
        done
    elif [[ $range =~ ^[0-9]+$ ]]; then
        HOST_CORES+=($range)
    fi
done

# Generate the bitmask for host cores
bitmask=0
for core in "${HOST_CORES[@]}"; do
    ##############
    # should match total_cores
    bitmask=$((bitmask | (1 << (31-core)))) # CHANGE ME
    ##############
done
HOST_CORES_MASK=$(printf '%X\n' $bitmask)

# Calculate VIRT_CORES by excluding the host cores from the full range of 0-31
full_range=($(seq 0 31))
echo  ${HOST_CORES[@]}
VIRT_CORES=()
for i in "${full_range[@]}"; do
    if [[ ! " ${HOST_CORES[@]} " =~ " $i " ]]; then
        VIRT_CORES+=($i)
    fi
done
VIRT_CORES=$(IFS=,; echo "${VIRT_CORES[*]}")
HOST_CORES=$(IFS=,; echo "${HOST_CORES[*]}")

echo $(date) libvirt hook $VM_NAME $VM_ACTION $HOST_CORES_MASK $VIRT_CORES ${HOST_CORES[@]} >> /var/log/libvirthook.log




 if [[ "$VM_ACTION" == "prepare/begin" ]]; then
     echo $(date) Reserving CPUs $VIRT_CORES for VM $VM_NAME >> /var/log/libvirthook.log
     systemctl set-property --runtime -- user.slice AllowedCPUs=$HOST_CORES
     systemctl set-property --runtime -- system.slice AllowedCPUs=$HOST_CORES
     systemctl set-property --runtime -- init.scope AllowedCPUs=$HOST_CORES

     echo $HOST_CORES_MASK > /sys/bus/workqueue/devices/writeback/cpumask
     echo 0 > /sys/bus/workqueue/devices/writeback/numa

     echo $(date) Successfully reserved CPUs $VIRT_CORES >> /var/log/libvirthook.log

 elif [[ "$VM_ACTION" == "started/begin" ]]; then
     if pid=$(pidof qemu-system-x86_64); then
       chrt -fifo -p 1 $pid
       echo $(date) Changing scheduling to fifo for pid $pid >> /var/log/libvirthook.log
     fi

 elif [[ "$VM_ACTION" == "release/end" ]]; then
     echo $(date) Releasing CPUs $VIRT_CORES from VM $VM_NAME >> /var/log/libvirthook.log
     systemctl set-property --runtime -- user.slice AllowedCPUs=$TOTAL_CORES
     systemctl set-property --runtime -- system.slice AllowedCPUs=$TOTAL_CORES
     systemctl set-property --runtime -- init.scope AllowedCPUs=$TOTAL_CORES

     echo $TOTAL_CORES_MASK > /sys/bus/workqueue/devices/writeback/cpumask
     echo 1 > /sys/bus/workqueue/devices/writeback/numa

     echo $(date) Successfully released CPUs $VIRT_CORES >> /var/log/libvirthook.log
fi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment