My main project, that I named "NixOS a la carte", revolves around refactoring of the NixOS modules, such that NixOS evaluation becomes more efficient and new use cases are enabled, such as NixOS-based docker images. More on that later. Such topics have been brought up in RFC 22 (minimal modules list) and RFC 78 (system agnostic config files). RFC 78 was in limbo because it lacked an example and RFC 22 was closed for bad reasons in my opinion. So my goal was to show a way forward.
In the early design of the module system, the modules were supposed to use imports to declare all their dependencies on other modules. In NixOS this idea was dropped in favor of a list that imports all modules, because this was a simpler solution that did not require any maintenance on the imports. This has served the project well, but it had two important consequences: performance worsening with every module added and very little attention for module design and, as a consequence, little to no support for alternate use cases.
After all, if all modules are imported always, there is no incentive to think about the patterns in module code. So for this "NixOS a la carte" project, I've started refactoring the core of NixOS such that everything doesn't depend on everything else anymore. This leads to a usable graph of imports which means we can import only what we need instead of everything.
A lot of this comes down to splitting modules and adding dependency inversions. A good driver for these changes is to write new tests, which are more like unit tests, which NixOS didn't use to have.
The effect of splitting modules is that some configurations will only have to depend on one, not the other, which means that a number of unused modules can be dropped from the evaluation.
Dependency inversion allows more modules to be split up. For instance, it used to be that the system.build.toplevel derivation had a hardcoded build script that always included an initrd (stage1), an activation script, etc. While technically nothing is wrong with that, it does lead to hard dependencies on the modules that provide these components; some of which are unused. Instead of top-level depending on the initrd through a monolithic script, the initrd module can write to the top-level systemBuilderCommands, inverting the dependency and making the use of the initrd modules optional.
This kind of refactoring usually involves deciding which component is more fundamental than the other. While it is possible to completely decouple the two modules in the example, this can be counterproductive. systemBuilderCommands would have to be in its own module, which is unnecessarily complicated and there would be uncertainty as to whether that option value even gets used anywhere.
Well-engineered modules are easier to test. My first goal was to isolate the module that provides /etc. By refactoring, this module can now run in isolation and it's easy to write a test for it with fakeroot or vmTools.runInLinuxVM. Did you know that runInLinuxVM has only just over a second of startup overhead? I was positively surprised.
Besides an etc test, I've added a test that builds a container image with the etc and users-groups modules. Another test runs a custom PostgreSQL container based on some NixOS modules, reusing the configuration file generation, similar to what would be desired in RFC 78, but also reusing the user setup. Interestingly, the activation script can run during the image build, leading to a lean image that starts the service immediately.
A proof of concept is in nixpkgs#148456.
While the refactoring is entirely compatible with current NixOS, the efficiency can only be gained by using a new entrypoint to NixOS which doesn't import all modules. Deciding exactly where and how to add this has been surprisingly more difficult than the refactoring. Some innovation is possible around the function that invokes the module system for NixOS. I've done some experimentation in this area, but extra complexity seems unwarranted at this time, so I want to add only the simplest function for this purpose. A little bit of rebasing will be necessary, but I see no real challenges for incorporating the refactorings into NixOS master.
In retrospect, I wish I'd chosen a smaller project for myself, so I could finish it entirely during the sprint. I did make a lot of time for discussions, helping others and some learning, so which are more important than finishing any one thing in my perspective.
While I don't think it makes sense to write about everyone else's project here, one notable other project where I helped a bit was the makeBinaryWrapper that Jacek/tfc/jonge worked on. The combination of the two projects could lead to very minimal images and systems, potentially without or with fewer interpreters, which is good for security.
Besides a few conversations, I did not spend much time on Hercules CI during the sprint. It's good to be reminded that that's possible, but I'm also excited to focus on its product development again. I can certainly recommend it for testing, building and deploying any Nixified project.