Shebang Scripts in Nix Derivations #nix

shebang_scripts_nix_derivations.md

Shebang Scripts in Nix Derivations

When you create a derivation, and you later run nix-build on the Nix derivation. Nix will transport the source to a chrooted temporary build directory (this actually can be configured in NixOS configuration.nix). The reason to do this is to ensure deterministic builds in a clean environment.

However the environment is so clean that no dependencies that you don't explicitly declare will be available in that environment. Also things that you take for granted like PATH is something that needs to be explicitly built. This is however alleviated by many setupHooks that manipulate the environment usually be creating various standardised environment variables like the PATH variable. While everything will workin nix-shell, when you build the package with nix-build, you will need to wrap any executables with the relevant PATH with wrapProgram.

In some cases, program source code uses hardcoded paths. For binary compilation, Nix knows usually how to deal with this automatically. However for textual scripts that refer to executables in the environment, you can usually solve this by explicitly making sure any required dependency is in the buildInputs or in the checkInputs.

However in some cases, the textual scripts refer to shebang paths like /usr/bin/env bash. They may in fact refer to these not only at the head of a script, but also within the textual body. Nix generally cannot find these automatically and something needs to "patch" these paths such that they point to the proper store paths.

There are many convenience utilities in Nixpkgs such as buildPythonApplication and buildPythonPackage that already deals with these shebang paths in designated locations. In the case of Python, that would be executables specified in the setup.py scripts property.

However not all paths will be found. In those cases you need to know how to use patchShebangs and substituteInPlace.

Here's an example. Let's say you have a Python application. And some of the tests you are running invoke other scripts by fork and exec. This means you may something like:

buildPythonApplication {
  checkPhase = ''
    bin/run-something
  '';
}

Where bin/run-something is:

#!/usr/bin/env bash

echo 'hello world'

After using nix-build --keep-failed, you may get an error saying something like bin/run-something doesn't exist. Yet if you go to the temporary failed build directory, you can see that bin/run-something definitely does exist.

So to solve this you need to explicitly patch the shebang with:

buildPythonApplication {
  checkPhase = ''
    patchShebangs bin
    bin/run-something
  '';
}

In another case, if that same shebang was used somewhere in the textual body of the script, you would instead need to run:

buildPythonApplication {
  checkPhase = ''
    substituteInPlace bin/run-something \
      --replace '/usr/bin/env bash' ${bash}/bin/bash
    bin/run-something
  '';
}

Finally once you build and install that package into the /nix/store. Then you also may need to use wrapProgram to explicitly fix the PATH when the user executes that variable:

buildPythonApplication {
  postFixup = ''
    wrapProgram $out/bin/run-something \
      --set PATH ${lib.makeBinPath [
        coreutils
      ]}
  '';
}