ilian

Ignore missing VT-d / HVM

1. Change the virtualization mode of sys-net and sys-usb to PV
2. Add qubes.enable_insecure_pv_passthrough to GRUB_CMDLINE_LINUX in /etc/default/grub
3. Run sudo grub2-mkconfig -o /boot/efi/EFI/qubes/grub.cfg
4. Reboot

Disable PV L1TF mitigations

Symptom: PV qubes won't start, VM logs are empty, /var/log/xen/console/hypervisor.log contains

L1TF-vulnerable L1e - Crashing

ilian

Enable verbose boot

1. Enter recovery mode
2. Run nvram boot-args="-v"

Set custom network gateway for a specific user

Add to /etc/pf.conf where 192.168.1.123 is the ip-address of the custom network gateway

pass out on en0 route-to (en0 192.168.1.123) from any to any user { myusername }

Perform similar changes to IPv6 stack or disable IPv6.

ilian

{
  # Use the systemd-boot EFI boot loader.
  boot.loader.efi.canTouchEfiVariables = true;

  boot.loader.grub.enable = true;
  boot.loader.grub.device = "nodev";
  boot.loader.grub.efiSupport = true;
  boot.loader.grub.extraConfig = ''
    insmod setpci;
    setpci -s "00:17.0" 3e.b=8;

ilian

# git clone https://github.com/cilium/cilium
# cd cilium
# git checkout v1.14.3
## build docker plugin since there is no dockerhub image for 1.14.3 for it
# docker buildx build -f images/cilium-docker-plugin/Dockerfile -t cilium-docker-plugin-local .
# docker compose up

## in other shell:
# docker network create my_network --ipam-driver cilium --driver cilium
# docker run --rm --net my_network -it alpine

ilian

How to reproduce consistently

1. Run watch -d -n 0.1 cat /proc/interrupts and find on what core periodic acpi interrupts happen
2. Pin a latency-sensitive application on that CPU core using taskset -c $CORE $APP where $CORE is the cpu number found in step 1. Observe that there are periodic interruptions in the real-time app (every ~0.5 seconds).

How to reproduce (seems random)

1. Run watch -d -n 0.1 cat /proc/interrupts and find on what core a lot of interrupts happen when holding your finger on the touchpad
2. Pin a latency-sensitive application on that CPU core using taskset -c $CORE $APP where $CORE is the cpu number found in step 1

Options to mitigate

• Pin the application on a different core

ilian

Our goal is to route a packet from a particular interface or source address range to the desired interface, even if that interface is not our default interface.

Source based routing with ip rule

We have virtual machines bridged to the virbr1 interface and we want to route trafic of these VMs to a virtual network that is accessible via tun0. Assuming default NAT routing has been set up for the VM instances, we can route the traffic as follows:

ip rule add dev virbr1 table 100
ip route add default dev tun0 table 100

ilian

mkdir -p ~/wine-dirs/wine-build
git clone git://source.winehq.org/git/wine.git ~/wine-dirs/wine-source
sudo dnf install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm
sudo dnf install mingw32-gcc mingw64-gcc pulseaudio-libs-devel dbus-libs fontconfig-devel freetype-devel gnutls-devel libjpeg-turbo-devel libpng-devel libtiff-devel mesa-libGL-devel libunwind-devel libxml2-devel libxslt-devel libFAudio-devel gstreamer1-devel gstreamer1-plugins-base-devel mpg123-devel SDL2-devel libXcursor-devel libXi-devel libXxf86vm-devel libXrandr-devel libXinerama-devel libXcomposite-devel mesa-libOSMesa-devel libv4l-devel jxrlib-devel
cd wine-dirs/wine-build
../wine-source/configure --enable-win64