Skip to content

Instantly share code, notes, and snippets.

@sandip4n
Created August 26, 2021 05:58
Show Gist options
  • Save sandip4n/f427bb21774296485e117cd467d939e7 to your computer and use it in GitHub Desktop.
Save sandip4n/f427bb21774296485e117cd467d939e7 to your computer and use it in GitHub Desktop.

Qemu Quick Start

- Create a new virtual disk image
	(host) $ qemu-img create -f qcow2 <path-to-qcow2-image> 30G

- Resize an existing virtual disk image
	(host) $ qemu-img resize <path-to-qcow2-image> +5G

- Resize guest root partition (LVM)
	(guest) # parted
		(parted) print free
		(parted) resizepart <partition-number>
		(parted) quit

	N.B. This resizes the physical volume first and then the logical
	     volume corresponding to the root filesystem. Names or identities
	     of these should be apparent from the output of pvs, vgs and lvs.

	(guest) # pvs && vgs && lvs
	(guest) # pvresize <physical-volume>
	(guest) # lvextend -r -l +100%FREE <logical-volume>
	(guest) # reboot

- Resize guest root partition (non-LVM)
	(guest) # parted
		(parted) print
		(parted) resizepart <partition-number>
		(parted) quit

	N.B. This resizes an existing partition. However, use only one of the
	     following three commands based on the target filesystem.
	     For XFS, use xfs_growfs.
	     For EXT4, use resize2fs.
	     For BTRFS, use btrfs filesystem resize.

	(guest) # xfs_growfs /
	(guest) # resize2fs <path-to-partition-device-file>
	(guest) # btrfs filesystem resize max /
	(guest) # reboot

- Use KVM without superuser access
	(host) $ sudo usermod -aG kvm $USER

- Build and run upstream QEMU
	(host) $ git clone https://git.qemu.org/git/qemu.git
	(host) $ cd qemu
	(host) $ mkdir -p build && cd build
	(host) $ ../configure --target-list=ppc64-softmmu
	(host) $ make -j32
	(host) $ ./ppc64-softmmu/qemu-system-ppc64 ...

- Setup a new guest on a virtual disk image using QEMU
	(host) $ qemu-system-ppc64			\
		-enable-kvm -m 8G -smp 8,threads=1	\
		-machine cap-htm=off			\
		-vga none -nographic			\
		-net nic,model=virtio -net user		\
		-drive file=<path-to-qcow2-image>	\
		-cdrom <path-to-iso-image>

- Boot a guest from a virtual disk image using QEMU
	(host) $ qemu-system-ppc64			\
		-enable-kvm -m 8G -smp 8,threads=1	\
		-machine pseries,cap-htm=off		\
		-vga none -nographic			\
		-net nic,model=virtio -net user		\
		-drive file=<path-to-qcow2-image>

- Boot a guest with a custom kernel and a virtual disk image using QEMU
	(host) $ qemu-system-ppc64			\
		-enable-kvm -m 8G -smp 8,threads=1	\
		-machine pseries,cap-htm=off		\
		-kernel <path-to-vmlinux>		\
		-append "root=/dev/vda<partition-id>"	\
		-device virtio-blk-pci,drive=disk	\
		-vga none -nographic			\
		-net nic,model=virtio -net user		\
		-drive file=<path-to-qcow2-image>,if=none,id=disk

- Boot a guest with a custom NUMA topology using QEMU
	(host) $ numactl -H
		available: 2 nodes (0,8)
		node 0 cpus: 0 8 16 24 32 40 48 56
		node 0 size: 261759 MB
		node 0 free: 244357 MB
		node 8 cpus: 64 72 80 88 96 104 112 120
		node 8 size: 261292 MB
		node 8 free: 252974 MB
		node distances:
		node   0   8 
		  0:  10  40 
		  8:  40  10 

	(host) $ qemu-system-ppc64								\
		-enable-kvm -m 16G								\
		-object memory-backend-ram,size=8G,policy=bind,host-nodes=0,id=ram-node0	\
		-numa node,nodeid=0,cpus=0-3,memdev=ram-node0					\
		-object memory-backend-ram,size=8G,policy=bind,host-nodes=8,id=ram-node1	\
		-numa node,nodeid=1,cpus=4-7,memdev=ram-node1					\
		-smp sockets=1,cores=8,threads=1						\
		-machine pseries,cap-htm=off							\
		-vga none -nographic								\
		-net nic,model=virtio -net user							\
		-drive file=<path-to-qcow2-image>

	N.B. This exposes two virtual NUMA nodes to the guest kernel. Each of
	     these nodes have 4 cpus and 8GB of memory. For guest node 0, the
	     memory comes from host node 0 and for guest node 1, the memory
	     comes from host node 8.

	(guest) $ numactl -H
		available: 2 nodes (0-1)
		node 0 cpus: 0 1 2 3
		node 0 size: 8148 MB
		node 0 free: 7907 MB
		node 1 cpus: 4 5 6 7
		node 1 size: 8180 MB
		node 1 free: 7318 MB
		node distances:
		node   0   1 
		  0:  10  40 
		  1:  40  10 

- Boot a guest with a custom NUMA topology having sparse physical memory using QEMU
	(host) $ qemu-system-ppc64						\
		-enable-kvm -m 64G,maxmem=512G,slots=16				\
		-smp 16,threads=1						\
		-object memory-backend-ram,size=16G,id=node0-ram0		\
		-object memory-backend-ram,size=16G,id=node0-ram1		\
		-object memory-backend-ram,size=16G,id=node1-ram0		\
		-object memory-backend-ram,size=16G,id=node1-ram1		\
		-object memory-backend-ram,size=16G,id=node2-ram0		\
		-object memory-backend-ram,size=16G,id=node2-ram1		\
		-object memory-backend-ram,size=16G,id=node3-ram0		\
		-object memory-backend-ram,size=16G,id=node3-ram1		\
		-numa node,nodeid=0,cpus=0-3,memdev=node0-ram0			\
		-numa node,nodeid=1,cpus=4-7,memdev=node1-ram0			\
		-numa node,nodeid=2,cpus=8-11,memdev=node2-ram0			\
		-numa node,nodeid=3,cpus=12-15,memdev=node3-ram0		\
		-device pc-dimm,memdev=node0-ram1,addr=0x0000000000,node=0	\
		-device pc-dimm,memdev=node1-ram1,addr=0x1400000000,node=1	\
		-device pc-dimm,memdev=node2-ram1,addr=0x2400000000,node=2	\
		-device pc-dimm,memdev=node3-ram1,addr=0x3400000000,node=3	\
		-machine pseries,cap-htm=off					\
		-vga none -nographic						\
		-net nic,model=virtio -net user					\
		-drive file=<path-to-qcow2-image>

	N.B. This configuration makes it possible to add holes in the physical
	     address space by specifying the start address. This is often seen
	     in real systems when all the DIMM slots are not populated.

	(guest) $ numactl -H
		available: 4 nodes (0-3)
		node 0 cpus: 0 1 2 3
		node 0 size: 32699 MB
		node 0 free: 32485 MB
		node 1 cpus: 4 5 6 7
		node 1 size: 32733 MB
		node 1 free: 32458 MB
		node 2 cpus: 8 9 10 11
		node 2 size: 32733 MB
		node 2 free: 32280 MB
		node 3 cpus: 12 13 14 15
		node 3 size: 32732 MB
		node 3 free: 32421 MB
		node distances:
		node   0   1   2   3 
		  0:  10  40  40  40 
		  1:  40  10  40  40 
		  2:  40  40  10  40 
		  3:  40  40  40  10 

	(guest) $ lsmem
		RANGE                                 SIZE  STATE REMOVABLE   BLOCK
		0x0000000000000000-0x00000017ffffffff  96G online       yes   0-383
		0x0000002400000000-0x00000027ffffffff  16G online       yes 576-639
		0x0000003400000000-0x00000037ffffffff  16G online       yes 832-895

- Boot a guest on a POWER8 system using QEMU
	(host) $ sudo ppc64_cpu --smt=off
	(host) $ qemu-system-ppc64 ...

- Boot a guest with hugepage backed memory using QEMU
	(host) $ sudo hugeadm --create-global-mounts
	(host) $ sudo hugeadm --pool-pages-min 16M:1024
	(host) $ qemu-system-ppc64					\
		-enable-kvm -m 16G					\
		-smp 8,threads=1					\
		-machine pseries,cap-htm=off,cap-hpt-max-page-size=16M	\
		-mem-path /var/lib/hugetlbfs/global/pagesize-16MB	\
		-vga none -nographic					\
		-net nic,model=virtio -net user				\
		-drive file=<path-to-qcow2-image>

	N.B. This allocates 1024 hugepages, each of size 16MB. This is the
	     minimum amount required to back the 16GB of memory that has been
	     requested for the guest.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment