Rethinking graphviz

gistfile1.txt

NB: This is a stream of consciousness after a minimum of thought.  It is possible that the graphviz community has already been over all of this.  Please don't take this too seriously.

Graphviz is awesome.  It allows you to logically describe a graph and then lay it out.

Graphviz is one of the most frustrating pieces of software I've ever used.  It is easy to get everything you *want* in the diagram but increadibly hard to make it actually look the way you imagine it.

The fundamental problem, in my mind, is that there is an unfortunate incosistency around subgraphs.  Most of the layout algorithms tend to do global layout when complex diagrams often times require different layout algorithms for different parts of the project.

To fix this I'd do the following:
* Allow true subgraph layout of nodes and edges completely contained in the subgraph.  Allow users to easily mix and match layout algorithms in the same diagram.
* Remember the positions of nodes and let users reference nodes into a subgraph.  Those edges would be laid out by the parent graph.
* Introduce a more explicit layout algorithm that is similar to someething like the HTML flexbox model.
* Get some more modern looking defaults for styles.
* Allow users to manually shift the layout around after it is done -- allow a bit of imperative programing so that users can impact the graph after it is laid out.
* I'd think about leaving a bunch of the old renders behind and just do SVG.  
* Allow users to use javascript to define new shapes.  
* Have the whole thing run in the browser.
* Concentrate on translating the SVG into the other necessary formats (PNG and PDF/PS are probably good enough at this point).  There are probably other SVG renderes that can be used out of the box.
* Perhaps mix/match this with things like d3.js.  Allow users to data bind live and re-layout.

This is a great list. Ambitious. I like the idea of SVG-only output; would it be possible to just stop there and allow other tools to do any onwards rasterization/encapsulation/transformation?

What do you have in mind for "Allow users to manually shift the layout around after it is done"? You can already do this in a ~primitive way by using (say) sfdp to lay out a graph, then adjust positions/colors/etc, then using neato -n to render/rasterize it. Akin to this: https://gist.github.com/hepwori/6051376.

I wonder if it's worth considering the (at least!) two rather separate use cases for graphviz: (1) production of diagrams; versus (2) production of large-scale network visualizations, like this thing https://twitter.com/isaach/status/356640049258983424. It makes little sense imo to consider things like live data binding and in-browser for (2) but a ton of sense for (1), for example.

Is it possible that graphviz is too general, or at least can't realistically be extended further while still being a fully generalized tool spanning (1) and (2)?

That's an excellent list of topics. The collaborators on this project (Emden Gansner, John Ellson, Yifan Hu and I, with assists from many others like Gordon Woodhull and Dwight Perry) have discussed some of them. I'll expand these comments. To start, the main topics are:

1. Features for concrete diagrams, such as shapes defined in Javascript, more reasonable defaults, etc.
2. Layout features, such as fully general, nested layouts, and online dynamic layout (geometric tweaking and full graph editing)
3. Implementation, e.g. execute graphviz in the browser, cut driver bloatware, etc.

These are all good suggestions. It's easiest to talk about the driver complexity, so I'll just point out that we buy into a lot of fontconfig/freetype/cairopango/Quartz etc. complication so it can be both cross-platform and work well with native libraries in particular target environments like OSX. If moving everything to SVG (or Javascript? they both came up?) then the layout code needs a way to find out the precise sizes of text labels and node shapes before generating a layout. BTW probably the situation has improved but for a long time we found SVG to be under-specified; you can ask for a Times-Roman font (or Times? or TimesRoman? or NewTimesRoman?) but it's hard to know what exactly will happen at render time, probably in a different computer than the one generating the layout. CairoPango (which is a huge chunk of code) goes the full route of loading a local font and then writing the strokes for every glyph to be rendered. echo "digraph G { X }" | dot -Tsvg:cairo turns into a 3K SVG file but at least we are really sure how the letter X will be rendered. It appears we are affected by the eternal struggle between nailing everything down at content creation time vs. allowing some flexibility at rendering time. Naturally we want both, depending on what we're doing.

I'll also point out there is some really great code in Tim Dwyer's COLA (Constrained Layout) http://marvl.infotech.monash.edu/webcola/
and https://github.com/tgdwyer/WebCola that handles various dynamic layouts very well, has many nice features, and runs in the browser. Not sure if this might be a better starting point than existing Graphviz core.