The following table with 9 PRs summarize all the work I have done starting from the application date in early April. The objective is to make existing Libvirt API domain properties configurable in Salt Virt in order to support advanced performance tuning. 5 PRs have been merged so far and another 4 are under review. Significant refactorization was done after adding memory tune support. Although all basic unit tests have passed, I believe more rigorous tests are needed before using these features in production. Detailed explanation of each PR is shown after the table.
Sincere thanks to my mentors Cedric Bosdonnat (Opensuse), Pavel Hrdina (RedHat) and Tyler Jones (SaltStack) for their patient support and advice. Thank Pedro Algarvio (s0undt3ch), Wayne Werner (waynew), Daniel Wozniak (dwoz) Megan Wilhite (ch3ll) from SaltStack community for being very responsive on my questions and code reviews and Libvirt organisation for providing this opportunity for me to learn and grow.
| ID | Name | Date | Label | Status | Tests & doc |
| 1 | Boot parameter update fix | 07 April | Bug fix | merged | ✔️ |
| 2 | Boot VMs with UEFI | 12 April | Feature | merged | ✔️ |
| 3 | Polish parameter doc | 05 May | Documentation | merged | ✖️ |
| 4 | Firmware auto select | 21 May | Feature | merged | ✔️ |
| 5 | Map null to None in sls file | 25 May | Minior fix | merged | ✔️ |
| 6 | Memory Tuning | 11 June | Feature | merged | ✔️ |
| 7 | CPU model, topology and NUMA Node Tuning | 07 July | Feature | merged into ID 9 | ✔️ |
| 8 | Memory Backing | 12 August | Feature | merged into ID 9 | ✔️ |
| 9 | cpu tuning and Iothreads allocation | 14 August | Feature | merged | ✔️ |
This PR fixes a bug on updating existing boot parameters. If users attempt to update an VM with existing boot parameters, it will fail. The reason is that found_tag = desc.find(tag) will always return None since tag is not a direct child of desc. Moreover, found_tag and found_tag.text != boot_tag_value returns an ElementTree object which is not a Boolean value. As a result, if branch will evaluate to False and statements under it will never get executed .
This PR adds support boot VMs with UEFI. Users are expected to specify loader and nvram with the corresponding firmware path to boot the VM with UEFI. For instance:
- boot:
loader: /usr/share/OVMF/OVMF_CODE.fd
nvram: /usr/share/OVMF/OVMF_VARS.ms.fd
To remove existing boot parameters, pass None object instead.
- boot:
kernel: None
initrd: None
This PR improves the boot parameter documentation. Parameter definitions are linked to virt.init function using pydoc strings
This PR allows users to specify efi parameters so that uefi firmware can be auto selected.
A bug on removing loader and nvram path is also fixed. Previous code will not work since None is handled as a string in the sls file. We need to use boot_tag_value == "None" to check that. However, later we realize that None object can be directly mapped to null in YAML. Future change was done to incorporate that.
Passing None with boot parameters in the sls file is handled as a string. We could pass null so that it will be translated to Python None object by YAML. It makes more sense to do so.
This PR makes memory tuning options available which allow much greater control of memory allocation. configurable parameters are:
hard_limitsoft_limitswap_hard_limitmin_guarantee
In addition, _handle_unit function is added which uses regex to parse user input into the corresponding size.
Fixes a bug on adding sub elements to an existing XML tree. Element.append() should be used to append an element into an existing XML tree. When using ElementTree.SubElement() we must make sure the parent element is from the target domain XML tree.
Refactored XML diff code, added a few helper functions to manipulate xml nodes. Credits to my mentor Cedric.
More robust check is added if 'initrd' in boot and boot.initrd to ensure None object will not cause an exception if it is passed to initialise a new VM .
This PR adds the feature to support CPU model, topology and numa node tuning. configurable parameters are:
modematchcheckmodelvendortopologycachefeature
numa which is used to specify the attributes and sub-elements of numa guest. configurable parameters are:
cpusmemorymemAccessdistancesiblingid and values
To configure numa guest, numa host must be configured beforehand, numatune can be used to specify memory which tells how to allocate memory for the domain process on a NUMA host and memnode which specify memory allocation policies per each guest NUMA node.
Support Memory Backing which enables how virtual memory pages are backed by host pages. configurable parameters are:
hugepagesnosharepageslockedsourceaccessdiscard
hugepages can take a string to represent the size of the page eg. hugepages: "1g" or a dictionary contains the size and particular numa node set. For instance: hugepages: { "1g": "1-4,3"}. Still working on passing a dictionary.
Making IOThreads allocation available. These are dedicated event loop threads for supported disk devices to perform block I/O requests. Iothreads parameters specify the number of IOThreads to be assigned to the domain by supported target storage devices.
IOThreads can be pinned using iothreadpin parameter in cputune which provides details regarding the CPU tunable parameters for the domain. cputune contains the following parameters:
vcpupinemulatorpiniothreadpinsharesperiodquotaglobal_periodglobal_quotaemulator_periodemulator_quotaiothread_periodiothread_quotavcpusched, iothreadsched and emulatorschedcachetunememorytune