Skip to content

Instantly share code, notes, and snippets.

@atweiden
Created March 14, 2016 19:26
Show Gist options
  • Star 8 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save atweiden/c60d27357c182f0913d2 to your computer and use it in GitHub Desktop.
Save atweiden/c60d27357c182f0913d2 to your computer and use it in GitHub Desktop.
Guix notes

guix notes

overview

  • guix is pronounced geeks
  • guix wants linux-libre kernel for containerization
  • changing any input to a guix package definition results in a different hash on disk (different directory)

configuring/building guix

important configure script opts

--localstatedir

In a multi-user setup, user profiles are stored in a place registered as a “garbage-collector root”, which ‘$HOME/.guix-profile’ points to (*note Invoking guix gc::). That directory is normally ‘LOCALSTATEDIR/profiles/per-user/USER’, where LOCALSTATEDIR is the value passed to ‘configure’ as ‘--localstatedir’, and USER is the user name. The ‘per-user’ directory is created when ‘guix-daemon’ is started, and the USER sub-directory is created by ‘guix package’.

guix-daemon setup

need to add unpriviledged system users for building guixpkgs:

groupadd --system guixbuild
for i in $(seq -w 1 10); do
  useradd -g guixbuild -G guixbuild         \
          -d /var/empty -s $(which nologin) \
          -c "Guix build user $i" --system  \
          guixbuilder$i
done

The number of build users determines how many build jobs may run in parallel, as specified by the ‘--max-jobs’ option (*note ‘--max-jobs’: Invoking guix-daemon.). The ‘guix-daemon’ program may then be run as ‘root’ with the following command(1):

guix-daemon --build-users-group=guixbuild

this^ command is run by the included systemd service file.

there are many options to consider for guix-daemon:

guix-daemon \
  # distribute build jobs across users in this group
  --build-users-group=GROUP \
  #
  # disallow binary downloads from hydra.gnu.org
  --no-substitutes \
  #
  # alternatively, download binaries from ws-separated urls
  --substitute-urls=URLS \
  #
  # disallow offloading builds to remote machines
  --no-build-hook \
  #
  # cache build failures for quick inspection
  #
  #    When this option is used, `guix gc --list-failures` can be
  #    used to query the set of store items marked as failed;
  #    `guix gc --clear-failures` removes store items from the set of
  #    cached failures.
  #
  --cache-failures \
  #
  # limit builds to N CPU cores (default = max cores)
  --cores=N \
  #
  # limit to N parallel build jobs (default = 1)
  --max-jobs=N \
  #
  # disable automatic file dedups (hard linking)
  --disable-deduplication \
  #
  # keep all build artifacts even when unused
  #
  #    Note that when both ‘--gc-keep-derivations’ and
  #    ‘--gc-keep-outputs’ are used, the effect is to keep all the
  #    build prerequisites (the sources, compiler, libraries, and other
  #    build-time tools) of live objects in the store, regardless of
  #    whether these prerequisites are live. This is convenient for
  #    developers since it saves rebuilds or downloads.
  #
  --gc-keep-derivations=yes --gc-keep-outputs=yes

Generating a signing key for archive exports.

Generate a new key pair for the daemons. This is a prerequisite before archives can be exported with ‘--export’. Note that this operation usually takes time, because it needs to gather enough entropy to generate the key pair.

The generated key pair is typically stored under ‘/etc/guix’, in ‘signing-key.pub’ (public key) and ‘signing-key.sec’ (private key, which must be kept secret.) When PARAMETERS is omitted, an ECDSA key using the Ed25519 curve is generated, or, for Libgcrypt versions before 1.6.0, it is a 4096-bit RSA key. Alternately, PARAMETERS can specify ‘genkey’ parameters suitable for Libgcrypt (*note ‘gcry_pk_genkey’: (gcrypt)General public-key related Functions.).

guix archive --generate-key[=PARAMETERS]

Authorize imports signed by the public key passed on standard input. The public key must be in “s-expression advanced format”—i.e., the same format as the ‘signing-key.pub’ file.

The list of authorized keys is kept in the human-editable file ‘/etc/guix/acl’. The file contains “advanced-format s-expressions” (http://people.csail.mit.edu/rivest/Sexp.txt) and is structured as an access-control list in the Simple Public-Key Infrastructure (SPKI) (http://theworld.com/~cme/spki.txt).

guix archive --authorize

guix client setup

Locale

Packages installed via Guix will not use the host system’s locale data. Instead, you must first install one of the locale packages available with Guix and then define the ‘GUIX_LOCPATH’ environment variable:

$ guix package -i glibc-locales
$ export GUIX_LOCPATH=$HOME/.guix-profile/lib/locale

Note that the ‘glibc-locales’ package contains data for all the locales supported by the GNU libc and weighs in at around 110 MiB. Alternately, the ‘glibc-utf8-locales’ is smaller but limited to a few UTF-8 locales.

Fonts

The majority of graphical applications use Fontconfig to locate and load fonts and perform X11-client-side rendering. Guix’s ‘fontconfig’ package looks for fonts in ‘$HOME/.guix-profile’ by default. Thus, to allow graphical applications installed with Guix to display fonts, you will have to install fonts with Guix as well. Essential font packages include ‘gs-fonts’, ‘font-dejavu’, and ‘font-gnu-freefont-ttf’.

Profile

Add to ~/.bash_profile:

export GUIX_PROFILE="$HOME/.guix-profile"
source "$HOME/.guix-profile/etc/profile"

Recommend guix pkg environment variables

Report environment variable definitions, in Bash syntax, that may be needed in order to use the set of installed packages. These environment variables are used to specify “search paths” for files used by some of the installed packages.

For example, GCC needs the ‘CPATH’ and ‘LIBRARY_PATH’ environment variables to be defined so it can look for headers and libraries in the user’s profile (*note (gcc)Environment Variables::). If GCC and, say, the C library are installed in the profile, then ‘--search-paths’ will suggest setting these variables to ‘PROFILE/include’ and ‘PROFILE/lib’, respectively.

The typical use case is to define these environment variables in the shell:

eval $(guix package --search-paths)

KIND may be one of ‘exact’, ‘prefix’, or ‘suffix’, meaning that the returned environment variable definitions will either be exact settings, or prefixes or suffixes of the current value of these variables. When omitted, KIND defaults to ‘exact’.

guix package --search-paths=[KIND]

[cmd] guix package

Overview

  • runs as non-root user
    • affects ~/.guix-profile
  • binaries install to ~/.guix-profile/bin

Synopsis

guix package OPTIONS

Remove ‘lua’ and install ‘guile’ and ‘guile-cairo’ in a single transaction:

guix package -r lua -i guile guile-cairo

guix profile history traversal

Roll back to the previous “generation” of the profile—i.e., undo the last transaction.

When combined with options such as ‘--install’, roll back occurs before any other actions.

When rolling back from the first generation that actually contains installed packages, the profile is made to point to the “zeroth generation”, which contains no files apart from its own meta-data.

Installing, removing, or upgrading packages from a generation that has been rolled back to overwrites previous future generations. Thus, the history of a profile’s generations is always linear.

guix package --roll-back

Switch to a particular generation defined by PATTERN.

PATTERN may be either a generation number or a number prefixed with “+” or “-”. The latter means: move forward/backward by a specified number of generations. For example, if you want to return to the latest generation after ‘--roll-back’, use ‘--switch-generation=+1’.

The difference between ‘--roll-back’ and ‘--switch-generation=-1’ is that ‘--switch-generation’ will not make a zeroth generation, so if a specified generation does not exist, the current generation will not be changed.

guix package --switch-generation=PATTERN

querying guix pkgs (uses GNU recutils format)

see also: guix size, guix graph

guix package --search=REGEXP
guix package --show=PACKAGE

List the available packages whose name, synopsis, or description matches REGEXP. Print all the meta-data of matching packages in ‘recutils’ format.

This allows specific fields to be extracted using the ‘recsel’ command, for instance:

guix package -s malloc | recsel -p name,version
name: glibc
version: 2.17

name: libgc
version: 7.2alpha6

Similarly, to show the name of all the packages available under the terms of the GNU LGPL version 3:

guix package -s "" | recsel -p name -e 'license ~ "LGPL 3"'
name: elfutils

name: gmp
…

Show details about PACKAGE, taken from the list of available packages, in ‘recutils’ format.

guix package --show=python | recsel -p name,version
name: python
version: 2.7.6

name: python
version: 3.3.5

You may also specify the full name of a package to only get details about a specific version of it:

guix package --show=python-3.3.5 | recsel -p name,version
name: python
version: 3.3.5

List the currently installed packages in the specified profile, with the most recently installed packages shown last. When REGEXP is specified, list only installed packages whose name matches REGEXP.

For each installed package, print the following items, separated by tabs: the package name, its version string, the part of the package that is installed (for instance, ‘out’ for the default output, ‘include’ for its headers, etc.), and the path of this package in the store.

guix package --list-installed[=REGEXP]

List packages currently available in the distribution for this system (*note GNU Distribution::). When REGEXP is specified, list only installed packages whose name matches REGEXP.

For each package, print the following items separated by tabs: its name, its version string, the parts of the package (*note Packages with Multiple Outputs::), and the source location of its definition.

guix package --list-available[=REGEXP]

Return a list of generations along with their creation dates; for each generation, show the installed packages, with the most recently installed packages shown last. Note that the zeroth generation is never shown.

For each installed package, print the following items, separated by tabs: the name of a package, its version string, the part of the package that is installed (*note Packages with Multiple Outputs::), and the location of this package in the store.

When PATTERN is used, the command returns only matching generations. Valid patterns include:

  • Integers and comma-separated integers. Both patterns denote generation numbers. For instance, ‘--list-generations=1’ returns the first one. And ‘--list-generations=1,8,2’ outputs three generations in the specified order. Neither spaces nor trailing commas are allowed.

  • Ranges. ‘--list-generations=2..9’ prints the specified generations and everything in between. Note that the start of a range must be lesser than its end. It is also possible to omit the endpoint. For example, ‘--list-generations=2..’, returns all generations starting from the second one.

  • Durations. You can also get the last N days, weeks, or months by passing an integer along with the first letter of the duration. For example, ‘--list-generations=20d’ lists generations that are up to 20 days old.

guix package --list-generations[=PATTERN]

Delete package generations.

When PATTERN is omitted, delete all generations except the current one.

This command accepts the same patterns as ‘--list-generations’. When PATTERN is specified, delete the matching generations. When PATTERN specifies a duration, generations older than the specified duration match. For instance, ‘--delete-generations=1m’ deletes generations that are more than one month old.

If the current generation matches, it is not deleted. Also, the zeroth generation is never deleted.

Note that deleting generations prevents roll-back to them. Consequently, this command must be used with care.

guix package --delete-generations[=PATTERN]

Finally, since ‘guix package’ may actually start build processes, it supports all the common build options that ‘guix build’ supports.

guix package [BUILDOPTS]

installing binary guix pkgs (substitutes)

Guix supports transparent source/binary deployment, which means that it can either build things locally, or download pre-built items from a server. We call these pre-built items “substitutes”—they are substitutes for local build results. In many cases, downloading a substitute is much faster than building things locally.

Substitutes can be anything resulting from a derivation build (*note Derivations::). Of course, in the common case, they are pre-built package binaries, but source tarballs, for instance, which also result from derivation builds, can be available as substitutes.

The ‘hydra.gnu.org’ server is a front-end to a build farm that builds packages from the GNU distribution continuously for some architectures, and makes them available as substitutes. This is the default source of substitutes; it can be overridden by passing the ‘--substitute-urls’ option either to ‘guix-daemon’ (*note ‘guix-daemon --substitute-urls’: daemon-substitute-urls.) or to client tools such as ‘guix package’ (*note client ‘--substitute-urls’ option: client-substitute-urls.).

To allow Guix to download substitutes from ‘hydra.gnu.org’, you must add its public key to the access control list (ACL) of archive imports, using the ‘guix archive’ command (*note Invoking guix archive::). Doing so implies that you trust ‘hydra.gnu.org’ to not be compromised and to serve genuine substitutes.

This public key is installed along with Guix, in ‘PREFIX/share/guix/hydra.gnu.org.pub’, where PREFIX is the installation prefix of Guix. If you installed Guix from source, make sure you checked the GPG signature of ‘guix-0.9.0.tar.gz’, which contains this public key file. Then, you can run something like this:

guix archive --authorize < hydra.gnu.org.pub

(arch)

guix archive --authorize < /usr/share/guix/hydra.gnu.org.pub

Once this is in place, the output of a command like ‘guix build’ should change from something like:

$ guix build emacs --dry-run
The following derivations would be built:
   /gnu/store/yr7bnx8xwcayd6j95r2clmkdl1qh688w-emacs-24.3.drv
   /gnu/store/x8qsh1hlhgjx6cwsjyvybnfv2i37z23w-dbus-1.6.4.tar.gz.drv
   /gnu/store/1ixwp12fl950d15h2cj11c73733jay0z-alsa-lib-1.0.27.1.tar.bz2.drv
   /gnu/store/nlma1pw0p603fpfiqy7kn4zm105r5dmw-util-linux-2.21.drv
…

to something like:

$ guix build emacs --dry-run
The following files would be downloaded:
   /gnu/store/pk3n22lbq6ydamyymqkkz7i69wiwjiwi-emacs-24.3
   /gnu/store/2ygn4ncnhrpr61rssa6z0d9x22si0va3-libjpeg-8d
   /gnu/store/71yz6lgx4dazma9dwn2mcjxaah9w77jq-cairo-1.12.16
   /gnu/store/7zdhgp0n1518lvfn8mb96sxqfmvqrl7v-libxrender-0.9.7
…

This indicates that substitutes from ‘hydra.gnu.org’ are usable and will be downloaded, when possible, for future builds.

Guix ignores substitutes that are not signed, or that are not signed by one of the keys listed in the ACL. It also detects and raises an error when attempting to use a substitute that has been tampered with.

The substitute mechanism can be disabled globally by running ‘guix-daemon’ with ‘--no-substitutes’ (*note Invoking guix-daemon::). It can also be disabled temporarily by passing the ‘--no-substitutes’ option to ‘guix package’, ‘guix build’, and other command-line tools.

[cmd] guix gc

Packages that are installed but not used may be “garbage-collected”. The ‘guix gc’ command allows users to explicitly run the garbage collector to reclaim space from the ‘/gnu/store’ directory. It is the only way to remove files from ‘/gnu/store’—removing files or directories manually may break it beyond repair!

Prior to running ‘guix gc --collect-garbage’ to make space, it is often useful to remove old generations from user profiles; that way, old package builds referenced by those generations can be reclaimed. This is achieved by running ‘guix package --delete-generations’ (*note Invoking guix package::).

[cmd] guix archive

The ‘guix archive’ command allows users to “export” files from the store into a single archive, and to later “import” them. In particular, it allows store files to be transferred from one machine to another machine’s store. For example, to transfer the ‘emacs’ package to a machine connected over SSH, one would run:

guix archive --export -r emacs | ssh the-machine guix archive --import

Similarly, a complete user profile may be transferred from one machine to another like this:

guix archive --export -r $(readlink -f ~/.guix-profile) \
  | ssh the-machine guix-archive --import

However, note that, in both examples, all of ‘emacs’ and the profile as well as all of their dependencies are transferred (due to ‘-r’), regardless of what is already available in the target machine’s store. The ‘--missing’ option can help figure out which items are missing from the target’s store.

Archives are stored in the “Nix archive” or “Nar” format, which is comparable in spirit to ‘tar’, but with a few noteworthy differences that make it more appropriate for our purposes. First, rather than recording all Unix meta-data for each file, the Nar format only mentions the file type (regular, directory, or symbolic link); Unix permissions and owner/group are dismissed. Second, the order in which directory entries are stored always follows the order of file names according to the C locale collation order. This makes archive production fully deterministic.

When exporting, the daemon digitally signs the contents of the archive, and that digital signature is appended. When importing, the daemon verifies the signature and rejects the import in case of an invalid signature or if the signing key is not authorized.

inspecting guix pkg archives

Read a single-item archive as served by substitute servers (*note Substitutes::) and extract it to DIRECTORY. This is a low-level operation needed in only very narrow use cases; see below.

The primary purpose of this operation is to facilitate inspection of archive contents coming from possibly untrusted substitute servers.

For example, the following command extracts the substitute for Emacs served by ‘hydra.gnu.org’ to ‘/tmp/emacs’:

$ wget -O - \
  http://hydra.gnu.org/nar/…-emacs-24.5 \
  | bunzip2 | guix archive -x /tmp/emacs
guix archive --extract=DIRECTORY

To export store files as an archive to the standard output, run:

guix archive --export OPTIONS SPECIFICATIONS...

SPECIFICATIONS may be either store file names or package specifications, as for ‘guix package’ (*note Invoking guix package::). For instance, the following command creates an archive containing the ‘gui’ output of the ‘git’ package and the main output of ‘emacs’:

guix archive --export git:gui /gnu/store/...-emacs-24.3 > great.nar

If the specified packages are not built yet, ‘guix archive’ automatically builds them. The build process may be controlled with the same options that can be passed to the ‘guix build’ command (*note common build options: Invoking guix build.).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment