Some emacs-related links I pointed people at earlier, dropped here for posterity, with commentary.
12.04+ seems to have Emacs 24.3 in its default repos. Just in case, here is a decent PPA with good instructions:
https://launchpad.net/~cassou/+archive/emacs
You can use homebrew to install Emacs, or get a binary build here:
There's also the Emacs Mac Port, which is a patch set distributed by its author via the emacs-devel mailing list:
http://lists.gnu.org/archive/html/emacs-devel/2014-03/msg01271.html
You have to grab the sources for emacs and build it yourself, but you're rewarded with smooth scrolling and pointless slide animations.
I don't know anything about windows. Apparently these exist:
http://ftp.gnu.org/gnu/emacs/windows/
You need leiningen anyway.
This is the quickest way to get a bulletproof Clojure setup for emacs. It is very opinionated and does not attempt to use ELPA (the emacs packaging system), though nothing stops you from making that change.
https://github.com/overtone/emacs-live
CIDER (Clojure IDE R....?) comes with Emacs Live, but is worth calling out because you will need to memorize at least half a dozen of the key chords:
https://github.com/clojure-emacs/cider
Paredit takes care of keeping your parentheses balanced. There are alternatives (mostly smartparens and nothing) but this is what comes with Emacs Live.
Paredit is fantastic, but early on you'll find yourself fighting with it a lot. Here is a cheat sheet, take some time to look it over, and memorize slurping and barfing key combos, as well as how to override the balancing logic and force a single paren to be added/removed.
One day, despite Paredit's best efforts, you will find yourself with unbalanced parens. On that day, you will need this cheat sheet:
http://www.emacswiki.org/emacs/PareditCheatsheet
Here's the Emacs guide linked to from the Clojure getting started documentation:
I made an error last night. It is Ubuntu 14.04 that has the 24.3 version of emacs.