systemd-run exploration

systemd-run.md

I've been learning about systemd-run:

systemd-run may be used to create and start a transient .service or .scope unit and run the specified COMMAND in it. It may also be used to create and start a transient .path, .socket, or .timer unit, that activates a .service unit when elapsing. If a command is run as transient service unit, it will be started and managed by the service manager like any other service, and thus shows up in the output of systemctl list-units like any other unit. It will run in a clean and detached execution environment, with the service manager as its parent process. In this mode, systemd-run will start the service asynchronously in the background and return after the command has begun execution (unless --no-block or --wait are specified, see below).

I am interested in using systemd-run as a way to start up background tasks, without needing to run a program in tmux. I ran this command to download a YouTube playlist:

❯ systemd-run --user \
  --remain-after-exit \
  --unit=download-tileman \
  --working-directory=/jellyfin-share/youtube/tileman \
  -p IPAccounting=yes \
  nix shell nixpkgs#yt-dlp -c yt-dlp 'https://www.youtube.com/playlist?list=PLWiMc19-qaA3uAQXQUcn8c1VR5ZwqVOXl' --write-info-json

That starts a new service:

Running as unit: download-tileman.service

I get a lot of information on the execution of the task with systemctl status download-tileman:

❯ systemctl --user status download-tileman
● download-tileman.service - /run/current-system/sw/bin/nix shell nixpkgs#yt-dlp -c yt-dlp https://www.youtube.com/playlist?list=>
     Loaded: loaded (/run/user/1000/systemd/transient/download-tileman.service; transient)
  Transient: yes
     Active: active (running) since Sun 2022-10-16 12:56:11 EDT; 5s ago
   Main PID: 1358143 (yt-dlp)
      Tasks: 1 (limit: 38447)
     Memory: 42.4M
        CPU: 3.068s
     CGroup: /user.slice/user-1000.slice/user@1000.service/app.slice/download-tileman.service
             └─1358143 /nix/store/9srs642k875z3qdk8glapjycncf2pa51-python3-3.10.7/bin/python3.10 /nix/store/r30s4a48b3n5icgmwgm4d>

And I can "tail" the execution with journalctl (remember to use the --user flag):

❯ journalctl --user --follow -u download-tileman
Oct 16 12:56:34 voidheart nix[1358143]: [info] Writing video metadata as JSON to: the 6,000 tile cape ｜ tileman #13 [X-E3fL4LHbM].info.json
Oct 16 12:56:34 voidheart nix[1358143]: [download] the 6,000 tile cape ｜ tileman #13 [X-E3fL4LHbM].webm has already been downloaded
Oct 16 12:56:34 voidheart nix[1358143]: [download] Downloading video 14 of 18
Oct 16 12:56:34 voidheart nix[1358143]: [youtube] f-QcbCGhLNM: Downloading webpage
Oct 16 12:56:35 voidheart nix[1358143]: [youtube] f-QcbCGhLNM: Downloading android player API JSON
Oct 16 12:56:36 voidheart nix[1358143]: [info] f-QcbCGhLNM: Downloading 1 format(s): 303+251
Oct 16 12:56:36 voidheart nix[1358143]: [info] Writing video metadata as JSON to: the city of potions ｜ tileman #14 [f-QcbCGhLNM].info.json
Oct 16 12:56:36 voidheart nix[1358143]: [download] the city of potions ｜ tileman #14 [f-QcbCGhLNM].webm has already been downloaded
...

Once the task is done, I can see how long it took to run:

❯ systemctl --user status download-tileman | grep -P 'Active|CPU'
     Active: active (exited) since Sun 2022-10-16 12:56:11 EDT; 11min ago
        CPU: 16.970s

(It didn't take that long because I had already downloaded the videos! 😅)

I'm going to start using systemd-run as a way to queue up long-running processes (big file downloads with aria2c, file transfers with rsync, testing nix derivation builds for nixpkgs, nixos-rebuild build, ad-hoc backups). I was using pueue to do this kind of thing previously, but systemd is everywhere.

There's a lot of interesting features. For example, it's possible to restrict network traffic in your systemd-run transient unit:

• -p IPAddressDeny=any -p IPAddressAllow=8.8.8.8 -p IPAddressAllow=127.0.0.0/8

Lennart Poettering writes:

The IP access list logic can in many ways provide a more modern replacement for the venerable TCP Wrapper, but unlike it it applies to all IP sockets of a service unconditionally, and requires no explicit support in any way in the service's code: no patching required. It can also replace (or augment) some facets of IP firewalling, i.e. Linux NetFilter/iptables. It's a simple way to lock down distribution/vendor supplied system services by default. For example, if you ship a service that you know never needs to access the network, then simply set IPAddressDeny=any (possibly combined with IPAddressAllow=localhost)