Skip to content

Instantly share code, notes, and snippets.

@WillyPillow

WillyPillow/main.md

Last active August 27, 2020 10:24
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save WillyPillow/b8a643ddbd9235a97bc187e6e44b16e4 to your computer and use it in GitHub Desktop.
Save WillyPillow/b8a643ddbd9235a97bc187e6e44b16e4 to your computer and use it in GitHub Desktop.
Download ZIP
[GSoC 2020] Qubes OS Template Manager Design
Raw
main.md

Initial project proposal (contains motivation and goals, while this document primarily focuses on the implementation):

Discussion threads:

Supported Operations

Command format should roughly follow that of DNF. See qvm-template --help for details.

  • qvm-template install
    • install template from repo/existing RPM
  • qvm-template download
    • download, but not install, template RPM from repo
  • qvm-template list
    • list available/installed templates
  • qvm-template info
    • list information about available/installed templates
  • qvm-template search
    • search for templates in repo
  • qvm-template upgrade
  • qvm-template downgrade
  • qvm-template reinstall
    • {upgrade,downgrade,reinstall} templates, overriding local changes
  • qvm-template remove
    • remove template (equivalent to qvm-remove)

Management Operations

  • qvm-template clean
    • clean downloaded RPMs
  • qvm-template repolist
    • list repos, e.g., qubes-templates-community

Other Possible Features

  • use kernel in template (i.e., set to PVGrub)
  • ability to output information in machine-readable format (JSON?), as mentioned in QubesOS/qubes-issues#2534
  • ability to install templates in alternative pools, again mentioned in #2534
  • when removing templates, possibly remove associated AppVMs or have their templates set to dummy1)
  • set template of AppVMs (merge with current qubes-template-manager?) (low priority)
  • GUI

Package Format

The package format is based on RPM to take advantage of existing tools and mechanisms such as repository management and signature verification.

For the package metadata, one needs to ensure that the character | does not appear. This is because we use the character as a separator internally. Failure to do so may result in qvm-template to fail with cryptic error messages. Note that as we already consider the repository metadata to be untrusted, this should not result in an issue security-wise.

The file structure should be quite similar to the current template RPMs. Namely, there should be the following files in the package:

  • var/lib/qubes/vm-templates/${template_name}/root.img.part.*
    • Split tarball of template root.img. Note that the file is no longer split, since newer releases of Fedora handles large files fine. However, the tarball is still kept due to RPM not preserving sparsity. The file is still split due to tools such as rpm2cpio not supporting large files.2
  • var/lib/qubes/vm-templates/${template_name}/template.conf
    • Stores custom package metadata (as RPM does not support custom attributes).
    • KEY=VALUE format
    • Fields (corresponding to qvm-prefs/qvm-features tags)
      • virt_mode Setting this to pv requires user confirmation.
      • kernel Only allowed to be set to "none", i.e., empty string.
      • no-monitor-layout
      • pci-e820-host
      • linux-stubdom
      • gui
      • gui-emulated
      • qrexec These six flags are only allowed to be set to "1" or "0", denoting "true" and "false" respectively.
      • net.fake-ip
      • net.fake-gateway
      • net.fake-netmask
  • var/lib/qubes/vm-templates/${template_name}/whitelisted-appmenus.list
  • var/lib/qubes/vm-templates/${template_name}/vm-whitelisted-appmenus.list
  • var/lib/qubes/vm-templates/${template_name}/netvm-whitelisted-appmenus.list
    • Same as current format. Namely, default appmenu entries of the template itself, VMs based on the template, and NetVMs based on the template are stored respectively.
    • Note that the contents of these files are stored in qvm-features upon installation. See section below for details.

Metadata Storage

The template manager needs to store metadata of installed templates such as version information and template origin. This data can be stored via qvm-features to keep things consistent when, e.g., qvm-remove is used. Besides, backups are also more easily handled. In addition, the fields also serve as an indication to whether a template is installed by qvm-template.

Fields

Most of the fields should be fairly self-explanatory.

  • template-name

  • template-epoch

  • template-version

  • template-release

  • template-reponame

  • template-buildtime

  • template-install-date

    • Dates are in ISO 8601 format, and can be parsed in Python with datetime.datetime.fromisoformat().

  • template-licence

  • template-url

  • template-summary

  • template-description

    • Note that the newlines in this field are converted to | to work better with existing tools like qvm-features.

  • menu-items

  • default-menu-items

  • netvm-menu-items

    • The entries store the contents of var/lib/qubes/vm-templates/${template_name}/whitelisted-appmenus.list var/lib/qubes/vm-templates/${template_name}/vm-whitelisted-appmenus.list var/lib/qubes/vm-templates/${template_name}/netvm-whitelisted-appmenus.list respectively.
    • Note that the newlines are converted to spaces, again for it to work better with existing tools like qvm-features. This should not cause ambiguity as the FreeDesktop specifications forbid spaces from file names of desktop files.

Interaction with Existing Tools

qvm-remove

Should work straightforwardly.

Renaming

A template is treated as non-RPM-installed once renamed. However, the origin is still stored in the VM features for future extension and a hint for the user.

Repositories

For UpdateVMs to access the template repository configuration, a package qubes-repo-templates is created that contains not only the repository configuration but also relevant PGP keys.

Permissions

To achieve features such as qtm purge, the template manager needs to have corresponding AdminAPI access to change the templates of other AppVMs. However, one might wish for the management VM not to be able to see/control all VMs.

More fine-grained rules can be created by automatically adjusting VM tags based on the template VM used -- similar to how GUI VMs are handled3 -- so that a management VM is able to control the VMs based on the templates that it control.

https://github.com/QubesOS/qubes-core-admin/blob/master/qubes/ext/gui.py#L55-L77

Qrexec Protocol

As discussed, qrexec calls for listing and downloading templates are needed. The following calls are thus proposed.

  • qubes.TemplateSearch: Wraps dnf repoquery
  • qubes.TemplateDownload: Wraps dnf download

Input

Both of the calls accept the following input format from standard input:

arg1
arg2
...
argN
package-file-spec
---
repo config

In other words, the input consists of two parts separated by the line ---. The first part contains some arguments and a parameter that indicates the pattern to be queried or downloaded.4 The following arguments are allowed:

  • --enablerepo=<repoid>
  • --disablerepo=<repoid>
  • --repoid=<repoid>
  • --releasever=<release>
  • --refresh

where the usage is identical to that of DNF.

The second part contains the repository configurations in /etc/yum.repos.d/ format.

Output

qubes.TemplateSearch outputs the packages in %{name}|%{epoch}|%{version}|%{release}|%{reponame}|%{downloadsize}|%{buildtime}|%{license}|%{url}|%{summary}|%{description}| format to standard output, separated by newlines. Note that there is a | at the end of the line. This is because %{description} may contain newlines, and doing so allows us to split the entries by |\n.5

On the other hand, qubes.TemplateDownload directly outputs the downloaded content to standard output.

Progress

Note that some changes may not have been fully reviewed. (Unfortunately, GFM does not seem to provide tri-state checkboxes.)

  • GitHub Project
  • Installation of downloaded RPM
    • Ability to set kernel & other qvm-feature flags mentioned above
    • Appmenu whitelists
    • qvm-template install
    • qvm-template reinstall
    • qvm-template downgrade
    • qvm-template upgrade
  • Repo interaction
  • Manage local templates
    • qvm-template remove
    • qvm-template purge
    • qvm-template clean
  • QubesOS/qubes-issues#5946
    • Document the new admin API call
  • Machine-readable output
    • Simple format
    • JSON format
    • Documentation
  • GUI

Footnotes

  1. Similar to qtm purge mentioned here. ↩

  2. This is due to the cpio format not supporting files over 4 GiB. ↩

  3. ↩

  4. Refer to DNF documentation for the definition of <package-file-spec>. ↩

  5. As we are using dnf repoquery --qf, we are unable to escape the newlines in advance. ↩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment