trying to get systemd-run and nixos working well together

nixos-systemd-run-confusion.md

Learning to use systemd-creds properly:

❯ pass show backups/notes | sudo systemd-creds encrypt --name=borg-notes - /tmp/borg-notes.creds

This encrypts with /var/lib/systemd/credential.secret, you can load using LoadCredentialEncrypted=:

• https://systemd.io/CREDENTIALS/

In a systemd unit, if i provide the property LoadCredentialEncrypted=borg-notes:/tmp/borg-notes.creds, then I can run systemd-creds cat borg-notes to get the password:

❯ sudo systemd-run -P --wait -p LoadCredentialEncrypted=borg-notes:/tmp/borg-notes.creds systemd-creds cat borg-notes
Running as unit: run-u3664.service
<REDACTED PASSWORD>
Finished with result: success

I'm trying to use systemd-run for a backup, so i would like to bring in borg! I'm using the BORG_PASSCOMMAND functionality, basically running borg create -v --stats "$HOME/backups/notes::$(date +%F-%T)" ~/org-roam:

❯ sudo systemd-run -P \
  --wait \
  -p LoadCredentialEncrypted=borg-notes:$(pwd)/borg-notes.creds \
  sh -c 'BORG_PASSCOMMAND="systemd-creds cat borg-notes" nix shell nixpkgs#borgbackup -c borg create -v --stats "$HOME/backups/notes::$(date +%F-%T)" ~/org-roam'

(I know I might need to fix those paths later! Just drafting.)

When executing that systemd-run command, I get an error that date and nix are not found?:

Running as unit: run-u3653.service
/run/current-system/sw/bin/sh: line 1: date: command not found
/run/current-system/sw/bin/sh: line 1: nix: command not found
Finished with result: exit-code
Main processes terminated with: code=exited/status=127

I understand that in derived units, generated from your NixOS configuration, the PATH variable in Environment is set directly, through the systemd.services.<name>.path option:

systemd = {
    services = {
      fetch-followers = {
        path = with pkgs; [ bash fetch-followers ];
        serviceConfig = {
          Type = "oneshot";
          WorkingDirectory = "/var/db/fetch-followers";
          EnvironmentFile = "/var/db/fetch-followers/secrets";
      };
    };
};

We can look up the derived path with nix eval:

❯ nix eval --impure --raw github:djanatyn/flake#nixosConfigurations.desktop.config.systemd.services.fetch-followers.path --apply builtins.toString
/nix/store/2ispfz80kmwrsvwndxkxs56irn86h43p-bash-5.1-p16 /nix/store/96hy69hbqzhg7fmr0kfj34mfz53gk688-fetch-followers-1.0 /nix/store/1xqr1166ji7xy80dp8d2nn1a6x8m6m5w-coreutils-9.1 /nix/store/f2f6n51xg66aq9mdcl9650y4rwyqh1jl-findutils-4.9.0 /nix/store/3bb3jppylvpzx2ky1p3w3cg029hkxgz1-gnugrep-3.7 /nix/store/2gdl6bljzi7a2cw2qsvv843p59p2pxng-gnused-4.8 /nix/store/wv5321690mvbf1da065dg53h7drcdl9z-systemd-251.4

And we can also inspect the generated unit with systemctl cat:

❯ systemctl cat fetch-followers | grep PATH | tr ':' '\n' | head -n 5
Environment="PATH=/nix/store/2ispfz80kmwrsvwndxkxs56irn86h43p-bash-5.1-p16/bin
/nix/store/9vwiazdvnjr2gq1hca4vs78ybgssdx3j-fetch-followers-1.0/bin
/nix/store/1xqr1166ji7xy80dp8d2nn1a6x8m6m5w-coreutils-9.1/bin
/nix/store/f2f6n51xg66aq9mdcl9650y4rwyqh1jl-findutils-4.9.0/bin
/nix/store/3bb3jppylvpzx2ky1p3w3cg029hkxgz1-gnugrep-3.7/bin

So, do I need to set PATH exhaustively in my systemd-run command? Are there any convenient wrappers for setting this up, perhaps hidden deep in nixpkgs? 👀

Maybe something like lib.makeLibraryPath:

❯ nix repl '<nixpkgs>' <<< 'pkgs.lib.makeLibraryPath [pkgs.stdenv.cc.cc  pkgs.glibc]' 2>/dev/null
"/nix/store/c10296m7xgm3ksibcklb2xf48jr635x3-gcc-9.3.0-lib/lib:/nix/store/0c7c96gikmzv87i7lv3vq5s1cmfjd6zf-glibc-2.31-74/lib"

successful-config.md

This works in my NixOS config, at least:

systemd.services = {
  backup-notes = {
    path = with pkgs; [ bash borgbackup coreutils ];
    serviceConfig = {
      Type = "oneshot";
      WorkingDirectory = "/var/lib/backups";
      LoadCredentialEncrypted = "borg-notes:/var/lib/backups/borg-notes.creds";
    };

    script = ''
      BORG_PASSCOMMAND="systemd-creds cat borg-notes" borg create -v --stats \
        "/home/djanatyn/backups/notes::$(date +%F-%T)" \
        /home/djanatyn/org-roam
    '';
  };
};

First, I have to make sure the password is present:

sudo mkdir -p /var/lib/backups
pass show backups/notes | sudo systemd-creds encrypt --name=borg-notes - /var/lib/backups/borg-notes.creds

Then I run sudo nixos-rebuild switch to get the unit derived and added to my running system. Finally, I can run backups: sudo systemctl start --full --wait --no-pager backup-notes

❯ systemctl status  backup-notes
○ backup-notes.service
     Loaded: loaded (/etc/systemd/system/backup-notes.service; linked; preset: enabled)
     Active: inactive (dead)

Nov 05 17:07:26 voidheart backup-notes-start[510449]: Utilization of max. archive size: 0%
Nov 05 17:07:26 voidheart backup-notes-start[510449]: ------------------------------------------------------------------------------
Nov 05 17:07:26 voidheart backup-notes-start[510449]:                        Original size      Compressed size    Deduplicated size
Nov 05 17:07:26 voidheart backup-notes-start[510449]: This archive:                8.07 MB              7.02 MB              3.60 kB
Nov 05 17:07:26 voidheart backup-notes-start[510449]: All archives:               40.22 MB             35.01 MB              7.49 MB
Nov 05 17:07:26 voidheart backup-notes-start[510449]:                        Unique chunks         Total chunks
Nov 05 17:07:26 voidheart backup-notes-start[510449]: Chunk index:                    1382                 6713
Nov 05 17:07:26 voidheart backup-notes-start[510449]: ------------------------------------------------------------------------------
Nov 05 17:07:26 voidheart systemd[1]: backup-notes.service: Deactivated successfully.
Nov 05 17:07:26 voidheart systemd[1]: Finished backup-notes.service.