This document describes some of the issues that exist with stdenv.mkDerivation
and that could or should be considered when redesigning it.
stdenv.mkDerivation comes with several fixed phases. It is possible to
override these phases or add other phases. It is however difficult to make
phases depend on one another as that requires describing their relative order
and that is unfortunately not possible.
As an example, there are multiple Python-related hooks that add to
postFixupHooks. Certain hooks depend on other hooks, yet all get added to
postFixupHooks, and it is not known in which order they get executed.
A possible solution is to topologically sort the phases and have phases define
before and after phases.
There is no support for logging. Currently anything can write to stdout or
stderr. There are no abstractions for providing structured output. For
example, it would be useful a certain message would be printed if a phase
started and finished.
The default phases expect make. While a lot of core software uses make, this
is irrelevant for most non-C/C++ software.
Support for make should be a setup hook providing phases as is done also for
other build systems.
Many hooks providing custom phases listen to specific environment variables that were defined for the hook in question. There is no structure for specifying these variables.
Use structured attributes and let each hook/phase have an attribute set in the
e.g. a phases set. Each hook could provide a schema, e.g. using jsonschema
or using the NixOS module system.
It is difficult to determine what is causing something to run during a build. It is difficult to find out which variables affect the build.
Structured logging that shows the decisions that were made and which variables were used for this.
The meta attribute set provides useful metadata about packages. Having this
possibility to store metadata is useful for packages inside Nixpkgs but also
outside.
Generally speaking the build, installation and testing of a package is done in one derivation. This is not always desirable since e.g. testing may require additional dependencies increasing the total work that needs to be done when rebuilds are needed.
An example here are Python packages, specifically those containing extension modules (compiled code). Tests are typically enabled and can take a long time. If a test fails, the whole builds fail, requiring not just rerunning the test-suite but also the entire build. Furthermore, there are typically a lot of test-time dependencies that are now added to the build-time closure.
Offer first-class support for installation and/or testing in separate derivations. This can flatten the dependency tree and reduce the amount of work that needs to be done. This becomes especially interesting in connection with content-addressed derivations.