document system.md

Document ALL the things

Inspired by the question if and how devdocs.io could add have a feature to add customs docs (see issue 295). I am beginning to wonder about the best way to create a System which centralizes all documentations you need for your work.

The end product

Should fulfill certain criteria:

• You can add your own documentation of in-house applications
• All documentations look the same
  • That is to say the system is coherent in itself
• Provide an API for others to get DOCs from us

Starting to think

What we probably need is a format of sorts which all other documentation gets parsed into.

Markdown

We could just take say ordered markdown files and display them. With this we need to parse everything to markdown files. We simply put them in a folder structure and create a web-view for that.

This is a very nice solution to display the documentation. We render from markdown into a fixed template which makes the system coherent. On top of that we can simply parse things to be markdown files.

Downsides:
We loose all annotation of documentation. Say you document a module in JavaScript and it throws an error. Now we use something like JSDoc to generate markdown documentation. This approach can display that well. But say a third party want to use our API to display information about a function when a users integrates the documented module. We could not supply anything for this kind of case because we know nothing about the structure of our data.

Conclusion:
This is a good approach to render things out in the end. However we probably need an intermediary step of documentation with more semantics.

See also:
A system which does something similar is documentUp it takes markdown that is to say the main readme of a public repo and creates nicely readable websites from that. This relies on the readme being structured with headings.

API

Ideally we wish to provide an API for others to use the documentation which we got into a single format. This could have a great benefit in writing extensions for editors and the likes as you only have to handle a single format.

Challenge:
The challenge comparing to just displaying documentations on our website in a single design is that this needs more semantics. This needs information about the presented data. While getting everything into markdown would be perfectly acceptable to display it, it does not work for an API. Here we need information about something being a function and having certain parameters for example.

Idea 1:
We could just add some form of output to the API. So say something is documented in JSDoc, we could provide markdown or whatever to display the documentation on our site and then also provide the JSDoc as JSON.

Document Anything Format

A format in which every documentation can be in would be best. Then write parsers that can translate between this and other formats.

Considerations

JSDoc:
Would be one such format that we could translate to and from.

HTML:
Some programs or command line tools only need documentation on how to use them. They only have a website. We could parse that into our format but not the other way around.

Personal remarks

What I would care most about is:

• Docs about frameworks I use
• Docs about JS
• HTML
• CSS
• Document my JS (JSDoc)

An easy way

Would be to create a system that you can parse every format you have into. At least HTML pages because according to wikipedia all major documentation generators output HTML.

This would not really come with an API. Or at least only with one that returns HTML or markdown files.

A few thoughts:

• you need many data sources. Web developers for instance will typically need the documentation for HTML, a Javascript library, CSS, SQL and some other tools. In particular, the documentation of the Javascript library as well as the SQL documentation (because you will use some specific RDBMS, the general SQL specification is not sufficient) will not be freely available. You won't be given access to existing documentation unless you buy some licenses - if they allow it at all.
  You can see that devdocs.io does only have PostgreSQL - neither MySQL nor Microsoft SQL Server. Not that somebody misses the latter one.
• The bulk of work will be to interpret the specific documentation and parse it into the common documentation format. In this step, you have to take all the specifics of the language or specification at hand into account. You cannot blindly import something from existing data sources, because you need to consider its structure in order to generate a consistent, meaningful universal documentation.

To preserve the semantic structure XML could be used. The process could be as follows:
original documentation => parser => XML intermediary format => Markdown (presentation layer) OR API access OR arbitary programs can read it
XML can be parsed directly into XHTML by means of XLST.

I still have a question: Do you want everything to look the same, or to be consistent with respect to structure?
Stuff like CSS and SQL are inherently different, therefore structural consistency will be hard to achieve. So I suppose you are going for a consistent interface with a consistent look and feel, different data sources but no uniform structure for the documentation?