I am glad it has been helpful for understanding vim.
The layout was a separate challenge from capturing the vim modes and transitions. The layout challenge I gave myself was to have no overlapping edges. I don't think a tool like graphviz can solve the layout of this graph. In my case it was a hybrid of manual effort and various tools. I am a Mechanical Engineer and have always enjoyed eigenvalue/vector problems. Roughly this was my approach: I created an adjacency matrix and calculated eigenvalues (sorted from min to max value) and their corresponding eigenvectors. These values can be used to cluster the graph into a tree of small graphs. Each sub-graph was laid out (from the top level clusters down to the leaves). This is a kind of divide and conquer approach... It is much easier to focus on laying out the simpler sub graphs... and then laying out their sub-graphs and so on.
One additional comment: the node for normal mode is actually two nodes in this rendering. This helps with the layout. It could be merged back into a single node if you consider the 'normal' mode node to be a donut shape with all other nodes in the middle. A donut shaped node is not a traditional shape for a node, but its topology allows the graph to be laid out without overlapping edges. As well, aesthetically, I liked the idea because you can think of 'normal' mode as a kind of home. But rather than be a 'home' at the center, the 'home' is the donut ring encompassing all modes.
Some day, when I have time, I'd like to build an interactive web page that renders 'normal' mode's node as a donut ring... and has features to filter/simplify the graph. As well, I'd like to have 2 views of this graph in tabs. One tab would be the visual rendering of the graph like this. And the other tab would have the adjacency matrix (a table) where the rows/columns are the nodes. And the rows/columns are grouped into a tree structure (dictated by their clustering). The adjacency matrix view with expandable/foldable rows would also allow studying the clustering one level at a time. It's a rough idea... I may have to implement it to really communicate the idea I am thinking of.
Beautiful! I want this on my wall!
Visitor: "What's that? Looks like a fancy techie thing. Is that something from Star Trek?"
Me: laughs "Not quite, actually it is the state diagram for my favorite text editor Vim."
(In my head I imagine all my visitors as astute critiques of technical wonder and are easily impressed by my overwhelming technical prowess)