Nix Flakes and Direnv on Mac OSX (Catalina)

flake-direnv.md

Development environment with Nix Flakes and Direnv

This document is targeted at those who seek to build reproducible dev environment across machines, OS, and time.

It maybe easier for remote teams to work together and not spending hours each person setting up asdf/pyenv/rbenv, LSP servers, linters, runtime/libs. Nix is probably the closest thing to Docker in terms of development environment.

Flake is used here because it provides hermetic build, with absolutely no reliance on system environment (be it Arch/Catalina/Mojave). Also it freezes dependencies in flake.lock so builds are reproducible.

This gist provides the setup to develop Java/Clojure/Python applications on Nix. But it can be easily adapted to ruby, nodejs, haskell.

Nix

https://nixos.org/guides/install-nix.html

No assumption is made whether you are using home-manager, configuration.nix. I don't use those because I am not pure enough :p

You just need to know:

1. What is a nix derivation (read here)
2. The main nix commands (read here)
3. What is a nix channel

In summary:

• nix-env is like homebrew, eg. nix-env -iA nixpkgs.nixfmt
• nix-build turns derivation into binary. (think of it as make)
• nix-shell or nix develop allows you go enter a shell with the build environent

Nix packages can be found at https://search.nixos.org/packages

Initially, I have mistaken the Nix channel (eg. nix-channel --add https://nixos.org/channels/nixpkgs-unstable) as the same as the Nix package (nix-env -f '<nixpkgs> -q nix'). They are not the same, the former determines the version/recency of packages, while the latter is a CLI.

Flake

Install bleeding-edge Nix with flake support

# At the time of writing, they are equivalent, just run either 1 of them:
nix-env -iA nixpkgs.nixFlakes # do this,
nix-env -iA nixpkgs.nixUnstable # or this

Enable flake's experimental flag

Modify ~/.config/nix/nix.conf with You might see ca-references in other people's nix.conf, and that is for running nix profile list|add|remove, which is out of the scope of this guide.

experimental-features = nix-command flakes
keep-derivations = true
keep-outputs = true

keep-* is used by nix-direnv in the next section

Usage

To use flake, simply create flake.nix in the project root.

Flakes are self-contained units that have inputs (dependencies) and outputs (packages, deployment instructions, Nix functions for use in other flakes)

taken from https://serokell.io/blog/practical-nix-flakes

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs";
  };

  outputs = { self, nixpkgs }: {
    packages.x86_64-linux.hello = /* something here */;
  };
}

You can use various input format in flake (and that is a power I like)

{
  # github example, also supported gitlab:
  inputs.nixpkgs.url = "github:Mic92/nixpkgs/master";
  # git urls
  inputs.git-example.url = "git+https://git.somehost.tld/user/path";
  # local directories (for absolute paths you can omit 'path:')
  inputs.directory-example.url = "path:/path/to/repo";
  # Use this for non-flakes
  inputs.bar.url = "github:foo/bar/branch";
  inputs.bar.flake = false;
  # Overwrite inputs in a flake
  # This is useful to use the same nixpkgs version in both flakes
  inputs.sops-nix.url = "github:Mic92/sops-nix";
  inputs.sops-nix.inputs.nixpkgs.follows = "nixpkgs";
  # Pin flakes to a specific revision
  inputs.nix-doom-emacs.url = "github:vlaci/nix-doom-emacs?rev=238b18d7b2c8239f676358634bfb32693d3706f3";
  inputs.nix-doom-emacs.flake = false;
}

Direnv

Direnv does not depend on Nix at all, install it from official website https://direnv.net.

Even without Nix, Direnv is pretty useful on its own. I use it with Doom emacs (https://github.com/hlissner/doom-emacs/tree/develop/modules/tools/direnv)

Nix Direnv

Do this for the first time:

git clone https://github.com/nix-community/nix-direnv $HOME/nix-direnv

# then, put this in ~/.direnvrc 
#   to source nix-direnv into .direnv
echo "source $HOME/nix-direnv/direnvrc" >> ~/.direnvrc

Where to go from here

1. Learn more about Nix language, stdenv and the API
2. Read up on Flake here and here
3. Write your own overlay

Java/Clojure

Disclaimer

This is not necessary and overkill. Only shown here for demonstration purposes.

.envrc

use flake

flake.nix

{
  description = "Flake to manage clojure workspace";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/master";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let pkgs = nixpkgs.legacyPackages.${system};
      in {
        devShell = pkgs.mkShell {
          buildInputs = with pkgs; [
            jdk11
            clojure
            leiningen
            clj-kondo
            # clojure-lsp <-- https://github.com/NixOS/nixpkgs/issues/122557 
          ];
          shellHook = ''
            export JAVA_HOME=${pkgs.jdk11}
            PATH="${pkgs.jdk11}/bin:$PATH"
          '';
        };
      });
}

Now all you have to do is to cd into the directory again :)

Reference:

clojure and clj came from https://github.com/clojure/brew-install/blob/1.10.3/src/main/resources/clojure/install/linux-install.sh

Python

In your project root:

.envrc

use flake

flake.nix

Specify the latest version of https://github.com/DavHau/pypi-deps-db, identified by revision (branch/tag) and SHA hash:

nix-env -f "<nixpkgs>" -iA nix-prefetch-git
nix-prefetch-url --unpack https://github.com/DavHau/pypi-deps-db/archive/master.tar.gz

 {
   description = "Flake to manage python workspace";

   inputs = {
     nixpkgs.url = "github:NixOS/nixpkgs/master";
     flake-utils.url = "github:numtide/flake-utils";
     mach-nix.url = "github:DavHau/mach-nix?ref=3.3.0";
   };

   outputs = { self, nixpkgs, flake-utils, mach-nix }:
     let
       # Customize starts
       python = "python39";
       pypiDataRev = "master";
       pypiDataSha256 = "1n78m1si8y20nrh6apif0q2qgp1qg2zhk03w0c2df0ci4bafjcap";
       devShell = pkgs:
         pkgs.mkShell {
           buildInputs = [
             (pkgs.${python}.withPackages
               (ps: with ps; [ pip black pyflakes isort ]))
             pkgs.nodePackages.pyright
             pkgs.nodePackages.prettier
             pkgs.docker
             pkgs.glpk
           ];
         };
       # Customize ends
     in flake-utils.lib.eachDefaultSystem (system:
       let
         pkgs = nixpkgs.legacyPackages.${system};
         # https://github.com/DavHau/mach-nix/issues/153#issuecomment-717690154
         mach-nix-wrapper = import mach-nix { inherit pkgs python pypiDataRev pypiDataSha256; };
         requirements = builtins.readFile ./requirements.txt;
         pythonShell = mach-nix-wrapper.mkPythonShell { inherit requirements; };
         mergeEnvs = envs:
           pkgs.mkShell (builtins.foldl' (a: v: {
             # runtime
             buildInputs = a.buildInputs ++ v.buildInputs;
             # build time
             nativeBuildInputs = a.nativeBuildInputs ++ v.nativeBuildInputs;
           }) (pkgs.mkShell { }) envs);
       in { devShell = mergeEnvs [ (devShell pkgs) pythonShell ]; });
 }

Example requirements.txt

fastapi==0.63.0
loguru==0.5.3
pydantic==1.8.1
starlette==0.13.6
uvicorn==0.13.4
pyvpsolver

Back-link to reddit

https://www.reddit.com/r/Python/comments/npu66t/reproducible_python_environment_with_nix_flake/ha4udvs/?context=3