Installing gccEmacs (native-comp) with Nix

gccEmacs.md

WARNING: THIS GIST IS OUT OF DATE AND NO LONGER RELEVANT

• Native-comp was enabled by default in nixpgks
• Pgtk is not enabled by default, for that you can either override the derivation or use emacsPgtk from the nix-community emacs overlay if you don't want to build it yourself

Nix

Adding the overlay and configuring cachix

Option 1: Adding the overlay to configuration.nix or home.nix

Add the nix-community overlay to your nixpkgs. You can add the following to your configuration.nix (NixOS, non-user specific) or to your home-manager's home.nix. Note in the case of the latter, this will not be available for installation via nix-env.

{
  nixpkgs.overlays = [
    (import (builtins.fetchTarball {
      url = https://github.com/nix-community/emacs-overlay/archive/master.tar.gz;
    }))
  ];
}

See nixos or home-manager options for more information on adding overlays.

Option 2: Add to the overlays directory

echo "import (builtins.fetchTarball {
      url = https://github.com/nix-community/emacs-overlay/archive/master.tar.gz;
    })" >> $HOME/.config/nixpkgs/overlays/emacs.nix

Optional: Add cachix support to avoid compilation

As this package is installed via an overlay, it is not built by the Hydra CI/CD pipeline that underlies nixpkgs. Instead, the binary is built by the nix-community hydra and pushed to Cachix. Follow the instructions here to add the nix-community cachix.

nix-env -iA cachix -f https://cachix.org/api/v1/install
cachix use nix-community

You will need to ensure that you are are installing the emacs overlay from the nixos-unstable unstable channel, not the nixpkgs-unstable channel. If you are using home-manager, you will need to set your default channel to nixos-unstable (you can do this on non-NixOS systems, the difference is that nixos-unstable lags behind nixpkgs-unstable due to requiring more comprehensive tests before deployment). Alternatively, you can separately import nixos-unstable import <nixos-unstable> { overlays = [ overlay1 overlay2 ]; } separately, and use a separate channel specifically for emacs.

Installing emacs

If you followed option 2, the following will work.

nix-env -iA nixpkgs.emacsGcc

More likely, you will want to declaratively install in your configuration.nix or home.nix.

Home-manager provides a handy module, you can also add extra packages such as vterm.

{
  programs.emacs = {
    enable = true;
    package = pkgs.emacsGcc;
    extraPackages = (epkgs: [ epkgs.vterm ] );
  };
}

Alternatively you can add something like the following to you configuration.nix system packages

{
emacsWithPackages = (pkgs.emacsPackagesGen pkgs.emacsGcc).emacsWithPackages (epkgs: ([epkgs.vterm]));
}

Notes

As of 2020-08-03 the above works on fedora/ubuntu/debian and under NixOS. If you get an immediate segfault upon starting emacs, please ensure that your font is appropriately configured.

@collares Thanks! I tried that, but got an error message saying that

error: file 'nixos-unstable' was not found in the Nix search path (add it using $NIX_PATH or -I)

From what I gather, that's because user channels aren't included in the nix search path by default (according to this issue). I'm trying to see if I can figure out how to add them, but in the meantime:

Instead, I tried doing importing the unstable channel instead via a url (inspired by this post on functor.tokyo:

let

  unstable = import (fetchTarball
    https://github.com/NixOS/nixpkgs-channels/archive/nixos-unstable.tar.gz) {
    overlays = [
      (import (builtins.fetchTarball {
        url =
          "https://github.com/nix-community/emacs-overlay/archive/master.tar.gz";
      }))
    ];
};

  my-emacs = unstable.emacsWithPackagesFromUsePackage {
    config = ~/.emacs.d/init.el;
    package = unstable.emacsGcc;
    alwaysTangle = true;
    extraEmacsPackages = epkgs: [
      epkgs.exwm
      epkgs.emacsql-sqlite
      epkgs.vterm
      epkgs.pdf-tools
    ];
  };

in
{
 # ...
}

Now it builds, but it still has compiles Emacs from the ground up. Should this work? I wonder whether there's something wrong with my Cachix setup. There's nothing else that needs to be done than install it and run the cachix use nix-community command, right?

Update

Wait, I managed to figure something out: I had to add the unstable channel as the root user and then run sudo nix-channel --update nixos-unstable. Is that the right approach?

At this point the caching looks like it works. I still have to manually compile EXWM it seems, but at least that's only about a minute as opposed to fifteen, so that's better.

Still, if it's possible to use the cache and the unstable channel without needing to have a stable channel set up (such as by using fetchTarball to get the channel, for instance), I'd be very interested in hearing about that. And if it isn't, why isn't it?

@thomasheartman The nixpkgs-channels repo is deprecated. Try using https://github.com/NixOS/nixpkgs/archive/nixos-unstable.tar.gz instead.

@collares Oh, yes; there we go! Thank you! That seems to have done the job.

For anyone else wondering, this is what I ended up doing:

let

  unstable = import (fetchTarball
    "https://github.com/NixOS/nixpkgs/archive/nixos-unstable.tar.gz") {
      overlays = [
        (import (builtins.fetchTarball {
          url =
            "https://github.com/nix-community/emacs-overlay/archive/master.tar.gz";
        }))
      ];
    };

  my-emacs = (pkgs.emacsPackagesGen unstable.emacsGcc).emacsWithPackages
    (epkgs: [ epkgs.exwm epkgs.emacsql-sqlite epkgs.vterm epkgs.pdf-tools ]);

in {

}

Just switching the URL was enough to make it work. Moving away from emacsWithPackagesFromUsePackage is mosty because I use Straight, and I have the impression that those two don't play well together, if at all.

In my home manager I have the following:

{ config, pkgs, ... }:

let

  unstable = import (fetchTarball
    "https://github.com/NixOS/nixpkgs/archive/nixos-unstable.tar.gz") {
      overlays = [
        (import (builtins.fetchTarball {
          url = https://github.com/nix-community/emacs-overlay/archive/master.tar.gz;
        }))
      ];
    };

in {
# <other config above>
    programs.emacs = {
      enable = true;
      package = unstable.emacsGcc;
      extraPackages = epkgs: with epkgs; [
          vterm
      ];
  };
# <other config after>
}

I have my channels updated (not that I think it would matter since I'm importing it directly) and have my cachix set to nix-community and your's/mjlbach.

Still, though, nix refuses to pull from the cache and tries to build emacs itself. Is there anything I'm missing here?

I don't provide a cache for emacsGcc on linux, only on macOS. I provide a cache for emacsPgtkGcc on Linux, for issues with the nix-community cache you should report there.

Ah my mistake about that. However, I just switched it and nix is still trying to build emacs when using emacsPgtkGcc instead of emacsGcc in my above config.

Well, you're using the nix-community overlay in that snippet, so like I said you can file an issue with the nix-community repo, or switch to using my overlay

Sorry, I am a bit confused, I thought I had it the way your readme shows:

Would you be able to tell me what I have to change in my config to have it use your's?

@jacksonludwig This may be unrelated, but could also be what you're seeing. I'm using the same approach (with the imported unstable channel). For a while this worked as expected and pulled the latest build from the cache. However, at some point last week (I think), this stopped working for me and I'm now back to having to build Emacs manually again.

Of course, it could be that I messed something else up somehow, but I don't think so, and seeing you experience something similar makes me think that there could be something wrong with the overlay/cache. However, I can't say for certain that I didn't mess it up myself without taking a closer look at my git history, so I may very well be in the wrong here (Occam's razor and all).

Just thought I'd chime in. If you manage to solve it, I'd love to hear about it. 😄

@thomasheartman yea, thanks for chiming in. I think you may be correct, as I also use the neovim-nightly community overlay that mjlbach maintains which has been pulled from the cache correctly every time I run home-manager switch. I feel like there is an issue somewhere in the emacs community overlay at the moment.

FWIW I made an issue on the community-overlay repo.

I just wanted to be 100% clear about how to configure this. My ~/.nixpkgs/darwin-configuration.nix was created by following the installation instructions at https://wickedchicken.github.io/post/macos-nix-setup/

I then added the cachix binary cache via:

nix-env -iA cachix -f https://cachix.org/api/v1/install
cachix use nix-community

Here's my darwin-configuration.nix:

{ config, pkgs, ... }:

let

  unstable = import (fetchTarball
    "https://github.com/NixOS/nixpkgs/archive/nixos-unstable.tar.gz") {
      overlays = [
        (import (builtins.fetchTarball {
          url =
            "https://github.com/nix-community/emacs-overlay/archive/master.tar.gz";
        }))
      ];
    };

  my-emacs = (pkgs.emacsPackagesGen unstable.emacsGcc).emacsWithPackages
    (epkgs: [ epkgs.exwm epkgs.emacsql-sqlite epkgs.vterm epkgs.pdf-tools ]);

in {
  # List packages installed in system profile. To search by name, run:
  # $ nix-env -qaP | grep wget
  environment.systemPackages = with pkgs; [
    vim
    curl
    wget
    my-emacs
    ];

  # Use a custom configuration.nix location.
  # $ darwin-rebuild switch -I darwin-config=$HOME/.config/nixpkgs/darwin/configuration.nix
  # environment.darwinConfig = "$HOME/.config/nixpkgs/darwin/configuration.nix";

  # Auto upgrade nix package and the daemon service.
  # services.nix-daemon.enable = true;
  # nix.package = pkgs.nix;

  # Create /etc/bashrc that loads the nix-darwin environment.
  programs.zsh.enable = true;  # default shell on catalina
  # programs.fish.enable = true;

  # Used for backwards compatibility, please read the changelog before changing.
  # $ darwin-rebuild changelog
  system.stateVersion = 4;
}

Right now, my emacs 28 is being built manually. I think I'm referring to the wrong emacs in the my-emacs = block. Does anyone know the correct package to refer to so I can grab the binary rather than rebuild?

Hey, @anthonyclarka2!

It may just be that you've been unlucky with when you've tried to pull the package (as mentioned in this issue on the emacs-overlay repo). If you want to pin it to a specific version, you could follow the advice in this thread.

At least from what I can tell, it looks right. However, I'm definitely not the expert, and may very well be wrong, so let's see what the other folks say.

---

Edit: fixed the link to the advice on pinning the overlay version. It previously pointed back to this thread, but should now point to a thread on the emacs-overlay repo instead.

I ended up using the instructions from this page: https://github.com/twlz0ne/nix-gccemacs-darwin

nix-env -iA cachix -f https://cachix.org/api/v1/install
cachix use gccemacs-darwin
nix-env -iA emacsGccDarwin -f https://github.com/twlz0ne/nix-gccemacs-darwin/archive/master.zip

It may just be that you've been unlucky with when you've tried to pull the package (as mentioned in this issue on the emacs-overlay repo). If you want to pin it to a specific version, you could follow the advice in this thread.

@thomasheartman Is that link to advice on specific-version pinning actually in this thread, or elsewhere? The suggestions here have been very helpful, and I can get an emacs that I like, but I'm moderately annoyed by frequent rebuilds under configuration.nix changes (caching doesn't seem to work for me and/or I'm just switching my config frequently enough that I'm hitting a window prior to cache population) and would like to be able to pin and bump manually.

Other than that outstanding question, this thread has been very useful, so thanks!

@intractable Oh, sorry; I must have pasted the wrong link. I meant to link to this issue on the emacs-overlay repo and not to this thread. I'll update my original comment too.

In short, though, the way you do it is by changing the URL to use a specific commit instead of the latest master commit (by swapping master for the desired commit hash) and by adding a sha256 field to the fetchTarball function. If you don't know the hash for the version you're downloading (which is likely), add 0000000000000000000000000000000000000000000000000000 or some other dummy value and change it to what nix reports.

As an example, my config looks like this now and it seems to work.

        import (
          builtins.fetchTarball {
            url =
              "https://github.com/nix-community/emacs-overlay/archive/8320c615b706f0d459544d7d37a59c5a5ff5e7e0.tar.gz";
            sha256 = "1pajyn4n0yzi8qxlqjlh20zhdifxfvxqdcjmphqmb8b5p2grk2rx";
          }
        )

@thomasheartman I had totally forgotten that one can put the gitrev directly into the URL like that. Thanks so much!

Is it still the case that you have to use the community overlay to get Emacs with both pgtk and nativecomp? If so, we really need to step it up. IMO nixpkgs#emacs should give you pgtk and nativecomp on linux. Fuck that patch that set it to lucid, nobody and nobody wants that.

I really should take this gist down:

• Native-comp was enabled here
• Pgtk is not enabled by default, for that you can either override the derivation or use emacsPgtk from the emacs-overlay if you don't want to build it yourself

Thanks for the update ❤️ . Glad to hear about native-comp being enabled in the default builds.

I just tried adding a pgtk variant package (https://github.com/edrex/nixpkgs/tree/emacs-pgtk) but the result is running under xwayland, not sure what's up with that. Did I miss a flag (seems withGTK3 should default to true when withPgtk = true) or is it broken?

With this, there would be 4 variants (emacs with lucid, emacs-gtk with the old busted gtk/X hybrid, pgtk, and nox. Each of these does its own ELN/ELC precompilation steps, which take awhile (~30mins on my gen8 i7).

Is there anything we could do to share steps among the variants? Are the eln cache outputs the same with the different UI options? If so maybe we could split that into a shared package? That alone would help enormously, since the bulk of the build time is precompilation.

'Twere me, there would just be emacs with pgtk and nox, but I get that it would go against community norms to ship an unofficial patchset as the default. Any recent progress with pgtk upstream, or is it stuck?

I actually switched to helix awhile back, so I don't have too much of a horse in the game, but I still open up emacs to do my laundry ahem use magit.

The pgtk branch was merged awhile ago, I assume the reason that it's not the default in nixpkgs is because Xorg is still the default (for better or worse) and there is no reason to use pgtk on X11.

I don't use nix actively anymore, my understanding is that without content addressed derivations it would be difficult to share the compiled elisp since it's dependent on the emacs input (not sure about this). For your case, maybe consider a remote builder with hercules-ci or GHA.

Not a helix user (without plugins any editor is DOA for me)

no reason to use pgtk on X11

Oh right, that seems valid.

without content addressed derivations it would be difficult to share the compiled elisp

This also makes total sense, if the emacs build is used as an input to the compilation process. Ideally the emacs packages could be split into a minimal (nox) core that's used for compilation and then several gui layers, but IDT that would be easily doable.

without plugins any editor is DOA for me

I get that position. For me at least, the focus on working really well out of box is just.. so good and refreshing. They are taking their time adding features, and polishing and iterating each one carefully before moving on. Also, what people mean by "plugin" is quite variable, in terms of the capabilities provided by the plugin API. Some upcoming features (helix-editor/helix#3393, helix-editor/helix#4709) will satisfy a lot of the user-customization aspect of plugins, while a full plugin host is coming, just needs time to bake. Honestly, I've never been so productive in an editor, and it only took me about a week to become proficient, coming from vim/evil.

without content addressed derivations it would be difficult to share the compiled elisp

Actually hmm, I think we could just pick an emacs variant package to use for the compilation derivation, and have all variants depend on that same expression. Does that make sense?

I would suggest discussing this on matrix with the nix emacs people :)

will do. thanks for engaging :)