Skip to content

Instantly share code, notes, and snippets.

@LnL7

LnL7/lnl-overlay.nix

Last active October 13, 2024 16:17
Show Gist options
  • Save LnL7/570349866bb69467d0caf5cb175faa74 to your computer and use it in GitHub Desktop.
Save LnL7/570349866bb69467d0caf5cb175faa74 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
lnl-overlay.nix
self: super:
{
# Install overlay:
# $ mkdir -p ~/.config/nixpkgs/overlays
# $ curl https://gist.githubusercontent.com/LnL7/570349866bb69467d0caf5cb175faa74/raw/3f3d53fe8e8713ee321ee894ecf76edbcb0b3711/lnl-overlay.nix -o ~/.config/nixpkgs/overlays/lnl.nix
userPackages = super.userPackages or {} // {
# Example:
hello = self.hello;
# add more packages here...
# Default packages:
# cacert = self.cacert;
# nix = self.nix; # don't enable this on multi-user.
nix-rebuild = super.writeScriptBin "nix-rebuild" ''
#!${super.stdenv.shell}
if ! command -v nix-env &>/dev/null; then
echo "warning: nix-env was not found in PATH, add nix to userPackages" >&2
PATH=${self.nix}/bin:$PATH
fi
exec nix-env -f '<nixpkgs>' -r -iA userPackages "$@"
'';
};
}
Copy link

ghost commented Oct 13, 2017

on macOS, if you haven't installed overlays before, you need to create the directory:

$ mkdir -p ~/.config/nixpkgs/overlays

@grahamc
Copy link

grahamc commented May 23, 2018

Aftr adding this overlay, run:

nix-env -f '<nixpkgs>' -r -iA userPackages

and from then on you edit this file and run nix-rebuild.

@jcrben
Copy link

jcrben commented Jul 23, 2018 •
edited
Loading

UPDATE: looks like userPackages isn't showing up in a search but other overlay packages are - I tried giving it a name but that didn't fix it. Guess it's missing some attributes.

Is it expected that this will show up in a search? Been fiddling around for a while trying to get it (mentioned more at NixOS/nixpkgs#43266 (comment)).

@LnL7
Copy link
Author

LnL7 commented Aug 11, 2018

Use super.userPackages or {} // super.recurseIntoAttrs { ... } to expose these when searching packages, etc.

@jabranham
Copy link

This doesn't seem to make info manuals of those programs available.

@lilyball
Copy link

lilyball commented May 31, 2019 •
edited
Loading

I tweaked this to print out the difference in packages on every rebuild:

self: super:

{
  userPackages = super.userPackages or {} // {
    # My packages
    inherit (self) hello;
    # add more packages here...

    # Default packages
    inherit (self)
      cacert
      nix; # don't enable nix on multi-user
    
    nix-rebuild = super.writeScriptBin "nix-rebuild" ''
      #!${super.stdenv.shell}
      set -e
      if ! command -v nix-env &>/dev/null; then
        echo "warning: nix-env was not found in PATH, add nix to userPackages" >&2
        PATH=${self.nix}/bin:$PATH
      fi
      IFS=- read -r _ oldGen _ <<<"$(readlink "$(readlink ~/.nix-profile)")"
      oldVersions=$(readlink ~/.nix-profile/package_versions || echo "/dev/null")
      nix-env -f '<nixpkgs>' -r -iA userPackages "$@"
      IFS=- read -r _ newGen _ <<<"$(readlink "$(readlink ~/.nix-profile)")"
      ${self.diffutils}/bin/diff --color -u --label "generation $oldGen" $oldVersions \
        --label "generation $newGen" ~/.nix-profile/package_versions \
        || true
    '';

    packageVersions =
      let
        versions = super.lib.attrsets.mapAttrsToList (_: pkg: pkg.name) self.userPackages;
        versionText = super.lib.strings.concatMapStrings (s: s+"\n") versions;
      in
      super.writeTextDir "package_versions" versionText;
  };
}

This dumps all the user packages into a file ~/.nix-profile/package_versions and then diffs this across the install. The output looks like

--- generation 68
+++ generation 69
@@ -33,6 +33,7 @@
 rlwrap-0.43
 ShellCheck-0.6.0
 taskwarrior-2.5.1
+tig-2.4.1
 tmux-2.9a
 unison-2.51.2
 vit-1.3

@steshaw
Copy link

steshaw commented Jun 12, 2019

Nice, @lilyball 😄

@lilyball
Copy link

I just updated my comment because I realized that the diff command was exiting with status code 1 when there's a difference. The update just tacks on || true so the whole rebuild doesn't exit with status code 1.

@steshaw
Copy link

steshaw commented Jun 12, 2019

Thanks again, @lilyball. I should have noticed:

image

@lilyball
Copy link

Here's the version I've been using lately:

self: super:

{
  userPackages = super.userPackages or {}
    // import ./packages.nix self super
    // super.lib.optionalAttrs (builtins.pathExists ./local.nix) (import ./local.nix self super)
    // {
      # Default packages for single-user; don't include this for multi-user
      inherit (self) cacert nix;
    }
    // {
      # Utilities
      nix-rebuild = super.writeShellScriptBin "nix-rebuild" ''
        set -e
        if ! command -v nix-env &>/dev/null; then
          echo "warning: nix-env was not found in PATH, add nix to userPackages" >&2
          PATH=${self.nix}/bin:$PATH
        fi
        IFS=- read -r _ oldGen _ <<<"$(readlink "$(readlink ~/.nix-profile)")"
        oldVersions=$(readlink ~/.nix-profile/package_versions || echo "/dev/null")
        nix-env -f '<nixpkgs>' -r -iA userPackages "$@"
        IFS=- read -r _ newGen _ <<<"$(readlink "$(readlink ~/.nix-profile)")"
        ${self.diffutils}/bin/diff --color -u --label "generation $oldGen" $oldVersions \
          --label "generation $newGen" ~/.nix-profile/package_versions \
          || true
      '';
      nix-what-rebuild = super.writeShellScriptBin "nix-what-rebuild" ''
        set -e
        if ! command -v nix-env &>/dev/null; then
          echo "warning: nix-env was not found in PATH, add nix to userPackages" >&2
          PATH=${self.nix}/bin:$PATH
        fi
        IFS=- read -r _ oldGen _ <<<"$(readlink "$(readlink ~/.nix-profile)")"
        oldVersions=$(readlink ~/.nix-profile/package_versions || echo "/dev/null")
        newVersions=$(nix-build --no-out-link -A userPackages.packageVersions '<nixpkgs>')
        ${self.diffutils}/bin/diff --color -u --label "generation $oldGen" "$oldVersions" \
          --label "after rebuild" "$newVersions" \
          && echo "no changes" \
          || true
      '';

      packageVersions =
        let
          collect = attrs:
            let recurse = x:
              if super.lib.isDerivation x then [x]
              else if x.recurseForDerivations or false then collect x
              else [];
            in super.lib.concatMap recurse (super.lib.attrValues attrs);
          versions = map (pkg: pkg.name) (collect self.userPackages);
          versionText = super.lib.strings.concatMapStrings (s: s+"\n") versions;
        in
        super.writeTextDir "package_versions" versionText;
  };
}

With this version my packages are stored in a sibling file called packages.nix, which looks like

self: super:
# Packages exposed as part of nixpkgs.userPackages
{
  inherit (self)
    fish tmux
    # …
    ;
}

and I can optionally add a separate local.nix that looks the same. The reason for this is I sync this overlay between machines, and this way I can exclude local.nix from the syncing in order to get per-machine customization.

Besides that, the differences between this and my previous version are the nix-what-rebuild command that prints the diff that nix-rebuild will print if I run it, and it fixes the package version tracking to handle children of userPackages being package sets (which installs the whole set if it's marked with recurseForDerivations).

@AndersonTorres
Copy link

@lilyball how can I use it with flakes?

@lilyball
Copy link

lilyball commented Oct 2, 2020

@AndersonTorres What, like, pulling in packages from flakes? I honestly don’t know, I haven’t used flakes, and what I can find with some quick internet searching does not mention any way to access flakes from outside of other flakes. Which is to say, I don’t know how flakes are actually used outside of NixOS. Surely there must be some way to do this though.

@ppenguin
Copy link

How can I use packages which have a . (period) in the name, e.g. gnome3.dconf-editor?

Now I'm getting

 nix-env -f '<nixpkgs>' -r -iA userPackages
error: syntax error, unexpected '.', at /home/ppenguin/.config/nixpkgs/packages.nix:13:8

@lilyball
Copy link

@ppenguin You’ll have to show your packages.nix (or at least line 13), because the period there isn’t in the package name, it’s just part of the attribute path.

If the period was part of the package name (like in my case I have a package named apollo-2.30.2) you have to quote the name, like nixpkgs."apollo-2.30.2"), but that’s not the case here. What you have here is just a syntax error and I can’t diagnose it without seeing the code.

@ppenguin
Copy link

ppenguin commented Jan 24, 2021 •
edited
Loading

@lilyball thanks for the quick reply, my packages.nix:

self: super:
# Packages exposed as part of nixpkgs.userPackages
{
  inherit (self)
        aspell
        bc
        coreutils
        gdb
        # ... 
        # gnome3.dconf-editor
        # xorg.xdpyinfo
        # xorg.xev
        # ...
    ;
}

If I don't comment out the above lines with the ., I get the mentioned error, but if I don't use the full attribute path (i.e. when I just have e.g. xev in the file), the package is not found:

$ nix-rebuild
at: (3:2) in file: /home/ppenguin/.config/nixpkgs/packages.nix

     2| # Packages exposed as part of nixpkgs.userPackages
     3| {
      |  ^
     4|   inherit (self)

attribute 'xev' missing

$  nix-env -qaP '.*xev.*' | cat
nixos.xorg.xev  xev-1.2.3

If I quote like this "xorg.xev" or like this "nixos.xorg.xev", it also gives me the attribute missing error.

I'm using the latest script you posted.

@lilyball
Copy link

@ppenguin You can’t use dotted paths in inherit like that. You’ll have to write it like

{
  inherit (self)
        aspell
        bc
        coreutils
        gdb
    ;
  inherit (self.gnome3)
        dconf-editor
    ;
  inherit (self.xorg)
        xdpyinfo
        xev
    ;
}

@ppenguin
Copy link

@lilyball Thanks a lot, that's something I'd have figured out by myself about a few decades later 😁

Copy link

ghost commented Mar 15, 2021

does this need to be setup for every channel?

@Amir-Ahmad
Copy link

@lilyball thanks for sharing that, looks great. I'm getting an infinite recursion error when trying to use your 2020 version (The older one works fine).

nix-env -f '' -r -iA userPackages
error: infinite recursion encountered, at /home/user/.config/nixpkgs/overlays/packages.nix:4:17

Any idea what could be causing this?

@lilyball
Copy link

@Amir-Ahmad What does your packages.nix look like?

@Amir-Ahmad
Copy link

Amir-Ahmad commented Nov 11, 2021 •
edited
Loading

@lilyball I was trying with this, copied from your comment https://gist.github.com/LnL7/570349866bb69467d0caf5cb175faa74#gistcomment-3372828.

self: super:
# Packages exposed as part of nixpkgs.userPackages
{
  inherit (self)
    fish tmux
    # …
    ;
}

@lilyball
Copy link

lilyball commented Nov 11, 2021 •
edited
Loading

@Amir-Ahmad Oh I think you probably put this in ~/.config/nixpkgs/overlays/ directly, yes? I suppose I should have elaborated. Your directory structure should look like

~/.config/nixpkgs/overlays/
    userPackages/
        default.nix
        packages.nix

The nested folder name (userPackages above) doesn't matter, but the two files do need to be nested in a folder, and the main file is default.nix in this folder.

What this does is it makes userPackages/default.nix to be the actual overlay, and that in turn imports ./packages.nix to construct the userPackages set. By putting both files in the top level overlays folder you instead make them two separate overlays and the packages.nix file is then trying to overwrite each package with itself, which causes infinite recursion.

@Amir-Ahmad
Copy link

@lilyball It's working now! thanks for the explanation 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment