nix at the command line is a package manager, but it also a programming language (dpl). the foundation of the nix language provides a minimal toolset for building software reproducibly.
while many have complained that the documentation is lack-luster at best, it is rather resourceful once you've been introduced to the ideas in the nix ecosystem.
in this article, we'll be examining the nix language primitives.
you may follow along with nix repl.
nix is based on the following ideas. the language starts by providing these primitives.
-
it is a pure programming language, in the sense of functional programming, this simply means that state of an expression, program, or function, is expressed entirely in its result/return.
-
lazily evaluated.
-
nix's execution flow is of an directed acyclical graph, meaning that the order of operations is dependent upon its inputs.
- the nix was written in c++ by eelco, 2003.
- the nixos project was started in 2004.
- nix has been used at the following institutions: lhc (europe), google (idx), repl.it.
you may follow along with nix repl.
we will start with talking about the nix language without the package manager.
the start.
nix-repl> a = 1
nix-repl> b = 2
nix-repl>
nix-repl> a + b
3
if you don't understand this part, you may now abandon this document. no prerequisites will be presented here, as their is no apparent demand for such.
attribute sets (attrset) are the primary data structure inside of the nix language, it allows us to use nested values such as below.
nix-repl> foo = { a = 1; b = 2; }
nix-repl>
nix-repl> foo.a + foo.b
3
the core backbone of the nix programming language, is its functions. there is syntacally two kinds of functions inside of nix. the first of which, we'll call short-form function declarations. as seen below.
nix-repl> add-5 = a: a + 5
nix-repl>
nix-repl> x = add-5 10;
15
the second of which, we'll call full form.
nix-repl> add-5 = {a}: a + 5
nix-repl> add-5 { a = 3; }
8
the terms short/full form are used as instruments here to describe the syntical difference between the two declarations, and is not an official term.
functions in nix are provided by builtins namespace. in the following example, we'll sum the list of numbers.
nix-repl> builtins.foldl' (sum: x: sum + x) 0 [ 1 2 3 4 5 ]
15
in functional programming, there is no concepts of loops, instead, they are function calls, which is called on each iteration of the list.
functions can be simplified by using the "currying" technique.
currying occurs when a function is partially applied, or otherwise missing arguments to complete execution.
when a function is curried, you receive a new function object, which can be called with the remaining arguments.
nix-repl> sum = builtins.foldl' (sum: x: sum + x) 0
nix-repl> sum [ 1 2 3 4 5 ]
15
currying is a common practice in functional programming, and is used to help simplify a set of operation.
we've introduced our first variable provided by nix, builtins. earlier, i referred to this as a "namespace", but in reality, its much more like an object. in the nix language, it is an attribute set.
nix-repl> builtins
{ foldl' = <primeop>; ... }
math often uses variables suffixed with quotes, which is often referred to as a "prime". nix makes no special meaning out of the usage of primes, but it should be noted in lagrange's notation, f'(x) and f''(x) are the first and second derivatives of f(x) with respect to x.
it is often, we have some complicated idea that needs to expressed.
the let bindings provide variables which are accessibly inside of the target after the in operator.
foo = list:
let
total = sum list;
first = builtins.head list;
in first + total
its worthy to also note now of the ... operator in full form functions.
attrsets may contain extra keys.
nix-repl> baz = {var, ...}: {}
nix-repl> baz { a = 1; b = 2; var = 3; }
{}
while the @ symbol stores all the parameters this function was called with to an identifier.
nix-repl> baz = args@{var, ...}: { a = args.a; var = args.var; }
nix-repl> baz { a = 1; b = 2; var = 3; }
{ a = 1; var = 3; }
nix-repl> baz = { a = { b = { c = 1; }; }; }
is the same as...
nix-repl> baz = { a.b.c = 1; };
deeply nested attributes can in short-hand form
inherit can be used as short hand for a = foo.a
nix-repl> { inherit (foo) a b; };
{ a = 1; b = 2; }
while
nix-repl> { inherit foo; }
{ foo = { a=1; b=2; } };
attribute sets being the primary backbone of nix, we'll start focusing on it. nix is an immutable language. meaning that once a variable is created, it not be modified.
right side has precedence, and are combined together.
nix-repl> { x = 1; y = 2; } // { x = 2; }
{ x = 2; y = 2; }
however, given the situation of trying to update the ips field on user10.
we try to update a nested field in an attrset by merging the pre-existing set with
the new data.
nix-repl> peers = {
user10.will_be_removed=1;
user10.ips = ["::1"];
}
nix-repl> peers.user10.will_be_removed
1
# right hand side has precedence.
nix-repl> builtins.attrnames (peers // { user10.ips = ["::1" "::3"]; }).user10
[ "ips" ]
so we updates the value user10.ips but, we also lost the user10.will_be_removed attribute. the quick answer to this, is to provide the full set of attributes on the right hand side of the //. this becomes infeasible when dealing with some larger systems. we'll talk about how to over come this later.
first, lets analyze why this the expected behavior.
nix-repl> var = peers // { user10 = { ips = ["::1" "::3"]; will_be_removed=0; }; }
nix-repl> var.user10
{ ips = [ ... ]; will_be_removed = 0; }
when providing the // operator, it overrides the attribute with the right hand side. so when we define user10, it is replacing the existing user10 on peers entirely, and not updating the variable.
we'll talk about overcoming this problem later.
practically speaking, a derivation is simply anything written inside of the nix-store.
the /nix/store contains all the system's installed programs.
this is where the package management side of nix comes into play, first i will show you a how to make derivations without nixpkgs. but it is likely you will never need to know this outside of creating your own core-environment.
first create our builder.sh. this would normally act as the build script for a particular unit of software.
in our case, it will just print all the variables declared in runtime when this is ran.
bash# cat> builder.sh <<eof
# builder.sh
declare -xp
echo foo > $out
eof
notice we don't use #! shebang headers, with nix this property is preserved within the nix system. more on that later.
so first, we'll load in nixpkgs. this is to grab the pkgs.bash.
we could make our own derivation of bash and stem entirely off of nixpkgs.
but for this example, we'll use the existing derivations.
nix-repl> :l <nixpkgs>
xxxxxx variables loaded.
nix-repl> drv = derivation {
name = "foo";
builder = "${pkgs.bash}/bin/bash";
args = [ ./builder.sh ];
system = builtins.currentsystem;
}
nix-repl> :b drv
this derivation produced the following outputs:
out -> /nix/store/gczb4qrag22harvv693wwnflqy7lx5pb-foo
so here we can find our build artifact.
bash# cat /nix/store/gczb4qrag22harvv693wwnflqy7lx5pb-foo
foo
and here we find the printed output of the build.
bash# nix log /nix/store/gczb4qrag22harvv693wwnflqy7lx5pb-foo
...
declare -x home="/homeless-shelter"
declare -x nix_build_cores="4"
declare -x nix_build_top="/tmp/nix-build-foo.drv-0"
declare -x nix_log_fd="2"
declare -x nix_store="/nix/store"
declare -x oldpwd
declare -x path="/path-not-set"
declare -x pwd="/tmp/nix-build-foo.drv-0"
declare -x shlvl="1"
declare -x temp="/tmp/nix-build-foo.drv-0"
declare -x tempdir="/tmp/nix-build-foo.drv-0"
declare -x tmp="/tmp/nix-build-foo.drv-0"
declare -x tmpdir="/tmp/nix-build-foo.drv-0"
declare -x builder="/nix/store/q1g0rl8zfmz7r371fp5p42p4acmv297d-bash-4.4-p19/bin/bash"
declare -x name="foo"
declare -x out="/nix/store/gczb4qrag22harvv693wwnflqy7lx5pb-foo"
declare -x system="x86_64-linux"
...
amazing, we've start building useless stuff. perhaps a step forward to real stuff, finally?
well, this where nixpkgs actually comes in. nixpkgs has an advertised 80,000 derivations inside of it,
it also includes tooling for cross compiling, trivial builders, and stdenv.
practically speaking, you should use stdenv for all your nix from here-on out, since it provides some base functions for composing derivations.
so earlier when we ran :l <nixpkgs>, provided us a few variables, lib pkgs config.
libprovides functionspkgsprovides derivationsconfigis hownixpkgsis configured to work.
nix-repl> :b pkgs.hello
this derivation produced the following outputs:
out -> /nix/store/gczb4qrag22harvv693wwnflqy7lx5pb-hello
stdenv provides a standard environment for nix to build packages with.
it employs the use of "phases", which are effectively ordered bash hooks.
it automatically assumes the builder is bash.
except in the case of cmakelist.txt, makefile being present.
drv = pkgs.stdenv.mkderivation {
name = "foo";
phases = "installphase";
buildinputs = [ pkgs.cowsay ];
installphase = ''
declare -xp
cowsay foo > $out
'';
}
phases="installphase"means that only the install phase will be run, and nosrcvariable is expected.
nix-repl> drv = pkgs.stdenv.mkderivation {
name = "foo-c";
src = ./.;
who="you";
buildinputs = [ pkgs.gcc ];
buildphase = ''
gcc -dwho="''${who}" $src/main.c -o $out
'';
}
./.is the current directory in nix-lang.
// main.c
#include <stdio.h>
#define who "nobody"
int main {
printf("hello %s\n", who);
return 0;
}
its worth noting that we're not making our derivations very tunable.
imagine if someone wanted to use a different version of gcc than we included, or wanted to override the who variable.
nix-repl> drv = pkgs.callpackage ({ stdenv, gcc, who ? "you", ... }:
stdenv.mkderivation {
inherit who;
src = ./.;
buildinputs = [ gcc ];
buildphase = ''
gcc -dwho="''${who}" $src/main.c -o $out
'';
}) {}
the reason its important to ensure our derivations are tunable, is so that we can share them with others, and they can also modify them.
the use of pkgs.callpackage includes 2 special functions, override and overrideattrs.
nix-repl> ndrv = drv.override { who = "mr.bean"; }
nix-repl> :b ndrv
overrideoverwrites the calling parameters.overrideattrsoverwrites the inner structure returned tomkderivation
callpackages is a very interesting function. for every name in the parameters of the inner function, those parameters are filled with pkgs.<parameter>.
in such case as gcc, it references pkgs.gcc. unmatch variables are placed in the second attrset {}. who is provided a default "you",
so there's no need to place it in the second {}.
a few notable environments you can build.
nix-repl> pkgs.python3.withpackages(ps: with ps; [ requests numpy ] )
nix-repl> pkgs.lua5_2.withpackages (ps: with ps; [ busted luafilesystem ]);
nix-repl> pkgs.postgresql.withpackages(ps: with ps; [ h3-pg postgis ])
nix-repl> pkgs.perl.withpackages (p: with p; [ configinifiles fileslurp ])
nix-repl> ruby.withpackages (pkgs: with pkgs; [ slop nokogiri ]
nix-repl> pkgs.haskellpackages.ghcwithpackages (ps: with ps; [ turtle ])
nix-repl> blender.withpackages (ps: [ps.bpycv])
alright, its time to step out of our comfort zone, and see modern nix!.
there is a ton to unfold here, but, it is all things we've seen before. we will break it down piece by piece so that you have a clear understanding of whats going on here.
# flake.nix
# it has to be named this
{
+ inputs.nixpkgs.url = "github:nixos/nixpkgs";
+ inputs.flake-parts.url = "github:herculesci/flake-parts"
+ outputs = inputs@{ self, flake-parts, ... }:
+ flake-parts.lib.mkflake { inherit inputs; } {
+ systems = ["x86_64-linux" "aarch64-linux"];
+ persystem = { config, lib, pkgs, ... }:
+ {
packages.foo = pkgs.callpackage ({bash, cowsay, stdenv, ...}:
stdenv.mkderivation {
name = "foo-cowsay";
phases = "installphase";
buildinputs = [ cowsay ];
installphase = ''
declare -xp
cowsay foo > $out
'';
}
) {};
+ };
+ };
}
we're using flakes here, you might have heard of them. flakes have two top-level attributes.
inputs = {}are repositories that are needed,outputs = _ : {}are exported for other flakes to use.
the section referring to flake-parts.lib.mkflake { inherit inputs; } { ... }
provides framework for composing flakes files.
in this case, we're composing cross compiling of foo-cowsay for x86_64-linux and aarch64-linux.
now its time to build this piece of software.
nix build /path/to/flake#foo-cowsay
if you upload this flake.nix into a github repository, you can also do
nix build github:user/repo#foo-cowsay
we can also run the software
nix run .#foo-cowsay
.is the current directory.
more so, we can enter into development environment for derivation with
nix develop .#foo-cowsay
bash# cowsay "hello"
it can also be attached to our current shell session.
nix shell .#foo-cowsay
bash# foo-cowsay
likewise can also check the build log with the identifier
nix log .#foo-cowsay
why not docker?
packages.foo-cowsay-docker = pkgs.callpackage({ stdenv, dockertools, foo-cowsay, ... }:
dockertools.buildimage {
name = "foo-cowsay";
tag = self.rev;
runasroot = ''
#!${stdenv.shell}
${dockertools.shadowsetup}
'';
config = {
cmd = [ "${foo-cowsay}" ];
workingdir = "/root";
};
});