Setting up disk encryption for Ubuntu is pretty straightforward. You just enter all the neccessary information in the installer, and it magically works. But if you want to use a USB key not to enter your passphrase each and every time you boot your machine, as you would obviously want to do with, say, a server, for some reason things get really complicated really quickly.
So if you want your machine conveniently encrypted with a USB key unlock option, then join me, on this journey of frustration, bitter disappointment, and, as is always the case with linux, new knowledge.
You may think that the main point of this document is to teach you how to set up the USB key unlock. In fact, this is not the case. The most valuable piece of information you will find here lies ahead. The how do you recover from "cryptsetup failed: bad password or options or moon phase or guess what it is" error section.