I worked on splitting packages into multiple outputs. Nix does support derivations with multiple outputs for a few years (I estimate), but it is little used. I focused on using it widely in nixpkgs. The main aim is to reduce runtime closure sizes by separating files not needed at runtime.
Most of the general ideas and parts of the current code were done about one and two years ago, mainly by Eelco. I updated the code wrt. stdenv changes since then and started tweaking and fixing the builds from the most deeply depended packages.
The largest savings are currently due to not having headers and documentation in runtime closures. Probably much more could be saved by turning locale data into plugins, but it's unclear how to do it, so that was not attempted (yet).
Code sharing in nix expressions is attempted as much as possible. That is, for typical package it should be enough just to define the list of outputs with standard names, e.g.
outputs = [ "dev" "out" "bin" "man" "doc" ];
and the stdenv code ensures passing flags to configure/install phases, and it also moves some kinds of files if present in a wrong output. Most our base packages are built by autotools where this approach already works very well.
The splitting still needs lots of work. The progress done around the sprint can be currently found only in my repo https://github.com/vcunat/nixpkgs/compare/v/modular and it isn't very tidy, as inspecting more packages sometimes uncovered a need for changes in the general stdenv code. I plan to re-work the changes soon into commits that are better described and reviewable.
Before having such features in master, we will also have to solve some issues related to using the split packages. That doesn't seem very realistic for the planned 14.10 release. Up to now all files for a package were on one path. In the nix language one can specify a particular output in style package.bin, and not doing so means the first output, which by convention contains or propagates everything needed to build against that package. That migrates well the package input specifications, but it seems very unsuitable e.g. for systemPackages on NixOS where one often does not e.g. want headers and wants to add man pages instead. Currently, nix-env behaves differently and installs the "out" output by default, which may be missing many files that people were used to until now. Also the names now get like "xz-5.0.5-dev", which plays badly with nix-env, as we've seen, and it seems better to e.g. create names like "xz-dev-5.0.5" (that is nix issue).(Is partially wrong, as nix-env is cleverer about these than I assumed, at least with nix-1.8.)
Unchanged text in readable format: 😁
I worked on splitting packages into multiple outputs. Nix does support derivations with multiple outputs for a few years (I estimate), but it is little used. I focused on using it widely in nixpkgs. The main aim is to reduce runtime closure sizes by separating files not needed at runtime.
Most of the general ideas and parts of the current code were done about one and two years ago, mainly by Eelco. I updated the code wrt. stdenv changes since then and started tweaking and fixing the builds from the most deeply depended packages.
The largest savings are currently due to not having headers and documentation in runtime closures. Probably much more could be saved by turning locale data into plugins, but it's unclear how to do it, so that was not attempted (yet).
Code sharing in nix expressions is attempted as much as possible. That is, for typical package it should be enough just to define the list of outputs with standard names, e.g.
and the stdenv code ensures passing flags to configure/install phases, and it also moves some kinds of files if present in a wrong output. Most our base packages are built by autotools where this approach already works very well.
The splitting still needs lots of work. The progress done around the sprint can be currently found only in my repo https://github.com/vcunat/nixpkgs/compare/v/modular and it isn't very tidy, as inspecting more packages sometimes uncovered a need for changes in the general stdenv code. I plan to re-work the changes soon into commits that are better described and reviewable.
Before having such features in master, we will also have to solve some issues related to using the split packages. That doesn't seem very realistic for the planned 14.10 release. Up to now all files for a package were on one path. In the nix language one can specify a particular output in style package.bin, and not doing so means the first output, which by convention contains or propagates everything needed to build against that package. That migrates well the package input specifications, but it seems very unsuitable e.g. for systemPackages on NixOS where one often does not e.g. want headers and wants to add man pages instead. Currently, nix-env behaves differently and installs the "out" output by default, which may be missing many files that people were used to until now. Also the names now get like
xz-5.0.5-dev, which plays badly with nix-env, as we've seen, and it seems better to e.g. create names likexz-dev-5.0.5(that is nix issue).