Thinking about a markup syntax and system which will be perfect for documentation

perfect_documentation_markup_system.md

Perfect Documentation Markup System

I've spent a lot of time over the past few years thinking about markup markup languages and systems in the context of writing documentation for commercial and open source projects. I've had the pleasure of creating a large Sphinx based document for a personal project, and in a user manual for a commercial project. I've also created a number of decent sized MkDocs based documents, and compared them with Sphinx and Hugo based alternatives. I've grappled with the practicalities of introducing such a system into a company that already has established document production workflows, and coming to grips with the fact that document production involves many more people than the original writers, and a system needs to cater for those people also. Throughout all of this, I've come to the conclusion that the current markdown based systems, as good as they are, still have a number of fundamental flaws, which need to be addressed for them to see more widespread use in industry.

Problems with All

• Translation is an afterthought, tooling is limited

Problems with reStructuredText

• Headings syntax is inconsistent

There are two different ways to define a heading.

*******
Heading
*******

or

Heading
*******

• The order of the introduction of symbols for headings determines their heading level, this leads inconsistencies across documents in the same project.
• lack of implementation, other than docutils and pandoc there aren't really many comprehensive parsers for reStructuredText
• lack of tools, there are not many editors with built in support for rendering reStructuredText and no WISWYG editors.
• for things like figures, they require three spaces and confusing subsequent syntax, it's easy for beginners to get wrong
• tables often have strange limitations due to parsing requirments
• hyperlinks with names have complicated syntax
• toctree is not a standard feature, and it is annoying to use for pdf documents
• currently no preview rendering for a number of source control systems

Problems with Markdown

• a number of different specs, although it appears to be coalescing around Commonmark thankfully
• inconsistent heading specifications, atx and setext, just pick one!
• no specification for inter-document referencing, or heading ids
• syntax has not been designed to be extensible, and there is no spec for extending it. Many systems which extend markdown do so in a way which does not fail/render gracefully