Skip to content

Instantly share code, notes, and snippets.

@toraritte

toraritte/nix_pin_version_of_individual_packages.md

Last active March 22, 2024 15:30
Show Gist options
  • Star 9 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save toraritte/62e53be9e6d88d8b6b97391eb3c6558b to your computer and use it in GitHub Desktop.
Save toraritte/62e53be9e6d88d8b6b97391eb3c6558b to your computer and use it in GitHub Desktop.
Download ZIP
(work in progress) State of affairs on how to pin versions of individual Nix packages (or how to refer to a specific package version with Nix)
Raw
nix_pin_version_of_individual_packages.md

Yes, there are several ways to do this, but none of them are as direct and simple as git v2.1.2; htop v1.2.3 and come with a lot of caveats.

aside
Specifying versions for programming language packages are possible too, but that topic seems to be even messier. The most promising standardization effort to date is dream2nix.

0. Methods

That is, available at the time of this writing:

0.1 "versioned" attribute paths (if available)

Single-version policy

We keep multiple versions in nixpkgs only when there's a good reason to. Nix is able to handle any number of versions/configurations, but on the other hand it's much more convenient when all (or most) use just a single one. It leads to better sharing of the effort in many respects: simplified maintenance, testing, sharing of the binaries, etc. It's what most distros do. (Only gentoo diverges from the big ones I know, and they pay a price for it.)

When we do create more variants, we just name them (attribute paths), e.g. gcc48 and gcc49 or ffmpeg and ffmpeg-full.

-- vcunat commented on Sep 6, 2015 on Nixpkgs issue #9682

Citing jtojnar's discourse answer for a more specific example:

$ nix-env -qP --available nodejs
nixos.nodejs       nodejs-10.18.1
nixos.nodejs-10_x  nodejs-10.18.1
nixos.nodejs-12_x  nodejs-12.14.1
nixos.nodejs-13_x  nodejs-13.6.0
$ nix-env -iA nixos.nodejs-13_x

0.2 Pinning

Either explicitly or implicitly with

  • Nix derivation expressions (see Example 2.2)
  • Nix command line tools (see Example 2.1)
  • overlays
  • overrides (see Example 2.3)
  • flakes (pinning is the default)
  • third party, "unofficial", and/or proprietary methods (e.g., niv, niv + Home Manager, flox)

1. Terms

1.1 Pinning

The best description I could find is in this comment by CMCDragonkai; to paraphrase it:

Pinning in a Nix expression means to use a Nix package set (usually Nixpkgs or a fork of it) at a specific point (i.e., in a certain state) of its history.

For example, using a Git (content-addressed) commit hash in the Nixpkgs repo is similar to being in "detached HEAD state": from a Nix expression's perspective, every package definition is the latest version at the time the commit was issued.

When referring to the Nixpkgs package set pinned to 39cd40f (Sep 29, 2017), the "latest" versions will be:

WARNING
The Nixpkgs repo tracks both the official Nix package set and the convenience tools used in Nix expressions! See Example 2.1 below on how this can break assumptions.

As far as I can tell, the term "pinning" is still not defined in any of the official documentation. (The Nixpkgs manual mentions it once with an example but without explanation.)

1.2 Nix expression

A piece of code written using the Nix expression language.

1.3 Nix derivation expression

A Nix derivation expression is a Nix expression that will evaluate to a store derivation (usually by eventually calling the derivation (source) primop, most commonly via mkShell (source) or mkDerivation (source)).

note

See section "1.6 Store derivation" for the difference between a store derivation and a Nix derivation expression.

1.4 Primop

Primitive operations, or primops, are functions that are built into the language. They need not be defined or imported; they are in scope by default. But since they are normal identifiers, they can be overridden by local definitions.

-- Eelco Dolstra, The Purely Functional Software Deployment Model (PhD thesis) (January 18, 2006)

These are implemented in the NixOS/nix repo.

1.5 Nix store (or simply, store)

According to the Nix manual's "Glossary":

[The store is] The location in the file system where store objects live. Typically /nix/store.

Nix operations work through the Nix store directory: inputs are taken from there and outputs get put there. (TODO: Verify this statement.)

1.6 Store objects

[A store object is] A file that is an immediate child of the Nix store directory. These can be regular files, but also entire directory trees.

Examples of store objects:

  • build result of a store derivation
  • a "fetched" source code of an application to be built
  • a shell script
  • temporary storage for a Nix expression

1.6 Store derivation

Paraphrasing Gabriella439's comment:

A store derivation is a set of language-agnostic instructions describing how to build a "software-constructible project" (i.e., something that can be built using software).

Building a store derivation (i.e., executing the build instructions) is analogous to actions such as:

1.6.1 Difference between store derivations and Nix derivation expressions

NOTE "function" and "function application" here feel wrong. Something like "Nix derivation expression = template", "store derivation = template application" ... but then what is the build result ( which will be a "store object")? That is definitely a function application. This feels similar to the distinction between Haskell's kinds and types, but can't expression so this may be wrong. ... Or, maybe: (explain param vs arg)

alias buildResult = storeObject
nix-expression :: param1 -> ... -> paramN -> buildResult

function
     sayStuff () { echo "${1}, ${2}"; }
-> invocable / evaluable function string
     constructed_function="sayStuff 'lofa' 'miez'"
-> function invocation / application
     eval $constructed_function

*------------------------------------------------------*
|                                                      |
|       STORE DERIVATION =/= NIX EXPRESSION            |
|                                                      |
*------------------------------------------------------*
|                                                      |
| NIX EXPRESSION == function                           |
|                                                      |
| ( Describes how to build a component. That is, how ) |
| ( to  compose  its  input parameters, which can be ) |
| ( other components as well.                        ) |
|                                                      |
| STORE DERIVATION == function application             |
|                                                      |
| ( Call a  Nix  expression with concrete arguments. ) |
| ( Corollary: a single Nix  expression  can produce ) |
| ( different derivations depending on the inputs.   ) |
|                                                      |
*------------------------------------------------------*

The purpose of Nix expressions is to produce a store derivation that can be built into a component (executable, library, etc.).

For context:

Two-stage building of Nix expressions. nix-instantiate translates Nix expressions into store derivations, and nix-store --realize builds the derivation into a software component.

Image taken from Eelco Dolstra's PhD thesis, section "2.4 Store derivations".

Extra

Normal form of a Nix expression

According to section "5.4 Translating Nix expressions to store derivations" in Eelco Dolstra's PhD thesis:

The normal form [of a Nix expression] should be

  • a call to derivation, or

  • a nested structure of lists and attribute sets that contain calls to derivation.

In any case, these derivation Nix expressions are subsequently translated to store derivations.

What is a software component?

A package, application, development environment, software library, etc.

More formally from "3.1 What is a component?" in Eelco Dolstra's PhD thesis:

A software component is                      
                                          
    *-------------------------------------*
1.  | a software artifact that is subject |
    | to automatic composition            |
    *-------------------------------------*
                                          
    It can require, and be required by,   
    other components.                     
                                          
    *----------------------*              
2.  | a unit of deployment |              
    *----------------------*

(That entire section is worth reading.)

1.7 Package set

See What are package sets? on the NixOS Discourse, but to paraphrase ryantm's answer:

Package set is a collection of Nix expressions that evaluate to an attribute set (also called "attrset") where the names are the names of the packages (or names of recursive package sets) and the values are Nix derivations.

Examples:

2. Examples

2.1 Pinning Nixpkgs on nix-shell invocation

Compressing a usual Nix-shell expression,

{ pkgs ? import <nixpkgs> {} }:

pkgs.mkShell {

  buildInputs = [
    pkgs.git
    pkgs.htop
  ];

  shellHook = ''
    echo "git:  $(git  --version)"
    echo "htop: $(htop --version)"
  '';
}

into a one-liner, then

nix-shell \
-I nixpkgs=https://github.com/NixOS/nixpkgs/archive/39cd40f7bea40116ecb756d46a687bfd0d2e550e.tar.gz \
-E '{ pkgs ? import <nixpkgs> {} }: pkgs.mkShell { buildInputs = [ pkgs.git pkgs.htop ]; shellHook = "echo \"git: $(git --version)\"; echo \"htop: $(htop --version)\";"; }'

won't work because mkShell has not been implemented yet, thus:

nix-shell \
-I nixpkgs=https://github.com/NixOS/nixpkgs/archive/39cd40f7bea40116ecb756d46a687bfd0d2e550e.tar.gz \
-E '{ pkgs ? import <nixpkgs> {} }: pkgs.stdenv.mkDerivation { name = "shell"; buildInputs = [ pkgs.git pkgs.htop ]; shellHook = "echo \"git: $(git --version)\"; echo \"htop: $(htop --version)\";"; }'

Notes:

  • The only difference between the 2 snippet is that the part pkgs.mkShell { buildInputs has been replaced with pkgs.stdenv.mkDerivation { name = "shell"; buildInputs.

  • Ran both commands on Ubuntu 22.04 (aarch64); also, the 2nd won't run on aarch64-darwin.

2.2 Pin Nixpkgs in a Nix expression

Using jeff-hykin's comment as template:

Step 0. Find the Version you Want

Find (almost) all versions of a package using lazamar's absolutely amazing online tool.

Step 1: Install that version using its Hash

Click on the hash in the row for the version you need (e.g., git 2.23.0 and htop 2.2.0), and either use the nix-env/nix-shell commands listed there one-by-one, or copy the Nix expression snippets into a file (and rename to variables to avoid clashes):

# git_htop.nixshell

let

  pkgs_for_git = import (builtins.fetchTarball {
    url = "https://github.com/NixOS/nixpkgs/archive/bca9437d1eae9519b61a58f2593f25f65494f8e9.tar.gz";
  }) {};

  pkgs_for_htop = import (builtins.fetchTarball {
    url = "https://github.com/NixOS/nixpkgs/archive/b5e903cedb331f9ee268ceebffb58069f1dae9fb.tar.gz";
  }) {};

  # Copied lines above from lazamar's tool,
  # but the shortened form works too:
  #
  #   pkgs = import (builtins.fetchTarball "<url>") {};

in
  
  # Chose `pkgs_for_htop` arbitrarily; what matters
  # is that the  derivation function  exists in the
  # pinned Nixpkgs.

  pkgs_for_htop.mkShell {

    buildInputs = [
      pkgs_for_htop.htop
      pkgs_for_git.git
    ];

    shellHook = ''
      echo "git:  $(git  --version)"
      echo "htop: $(htop --version)"
    '';
  }

and then call it:

nix-shell -v git_htop.nixshell

2.3 Specify packages with overlays (stub)

Simply linking a file from jwiegley's nix-config repo; I don't understand it yet, but wanted to include it anyway.

3. Further reading

Online discussions to follow this still evolving topic:

@toraritte
Copy link
Author

This is a "private" version of this Unix & Linux stackexchange answer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment