To use Nix with an ordinary install on the cluster requires access to
/nix and also sharing /nix across all compute nodes. This is not
feasible without a great deal of work by the sys admin. There already
exist shared directories that users have access to (/users and
/working). /working is used mostly for building environments and
writes during job runs. There are two ways to install a non-root
Nix. One is to try to install a version of Nix with a built-in
alternative store location and the other is to use chroot and trick
Nix into thinking it's writing to /nix, but is actually writing to a
local location.
The edited version of the script used to try this is
https://github.com/wd15/nix-no-root/tree/ruth. This is a bash script
that installs Nix and uses a specified location for the Nix store.
This script is
changed
from the original and has an updated version of Nix. Currently, this
method is not working correctly. The install completes, but it makes
some attempts to obtain derivations from /nix/store instead of the
specified Nix store location. One frustration during the install is
trying to reconcile the version of the Nix manager with the version of
Nixpkgs. Using --store=/path/to/new/store as an argument to
nix-shell and nix-env allows it to start installing, but it
results with an error message claiming that it can't find something in
the /nix/store. The tool is designed for older version of Nix
(2.0.4) so things have changed, obviously.
As mentioned on the Nix
wiki,
nix-user-chroot is
the preferred method for installing Nix in an alternative place to
/nix. Here are the steps required.
-
Install rust-dev on ruth (and cluster nodes) as nix-user-chroot is a Rust package.
-
Enable the unshare command on ruth and cluster nodes. Test with,
$ unshare --user --pid echo YES. -
Install Nix using the suggested installation.
-
Remove all local Nix config files such as
~/.nix-profile~/.nix-defexpr~/.nix-channels~/.config/nix
That's it. It basically works after these steps.
Here is a small script to make things easier when running Nix using nix-user-chroot.
#!/bin/bash
# This script runs nix-user-chroot as `nix-root`, for example to enter an nix-shell use
#
# $ nix-root nix-shell --command "python -c 'import pymks; pymks.test()'"
#
# for example.
args=$(printf "%q " "${@}")
~/git/nix-user-chroot/target/release/nix-user-chroot /working/wd15/.nix bash -l -c "$args"
It has the path to the nix-user-chroot executable and the directory
used to mount to /nix (/working/wd15/.nix in this case). Thus,
running nix-shell just requires nix-root nix-shell for example and
any other arguments normally added to nix-shell.