Comprehensive Documentation

comprehensive_documentation.md

Software is layered.

Documentation is not. If your documentation states

Run npm install foo to install this module

It is really saying

• Learn how to use Terminal
• Install XCode tools
• Install homebrew
• Install node
• Learn npm
• npm install foo
• Install Sublime Text or some editor
• Learn how to use that editor
• Learn JavaScript
• Learn how to use foo in a library using npm
• Learn how foo works

This the curse of knowledge and a boatload of implicit requirements.

With computers, you cannot learn anything in isolation.

This means that a lot of tools that are accessible to programmers, like Jekyll or anything that requires a CLI, are inaccessible to everyone else.

Despite being generally a documentation maximalist - I believe that it's good to spend more time on documentation than code - I don't think that the answer is for every software project to document every step required to use it. That's unreasonable, and the task of writing documentation is hard. When you write documentation, you should have your heart in it, your spellcheck on, and be prepared to spend much more time editing than typing, and after that, maintaining. It's not something that should be done to tick a box.

---

So what's the answer? Well, a random guess would be:

Take the list of documentation needed for npm install foo, which would probably be 20k words total, and split it into things that are actually specific to foo

• npm install foo
• Learn how foo works

And those that aren't.

• Learn how to use Terminal
• Install XCode tools
• Install homebrew
• Install node
• Learn npm
• Install Sublime Text or some editor
• Learn how to use that editor
• Learn JavaScript
• Learn how to use foo in a library using npm

And then take each kind of typical install instruction:

• Install some homebrew cli utility
• Install some python module
• Install some ruby gem
• Install some server application written in node

And automatically expand modular documentation into a pipeline of tasks:

type	requirements
python module	terminal + homebrew + xcode + editor + python
node module	terminal + homebrew + xcode + editor + npm
ruby gem	terminal + homebrew + xcode + editor + rbenv + gems

It gets harder, of course: after you're done with this very OSX-centric guide, you would write swap-out parts for Windows & Linux and hope that they don't trickle down the stream for every other step.

And it gets worse the further away someone is from "a scratch install of everything": how do you install jekyll with system ruby, or rbenv, or rvm, or jruby, or in a vm, or within a build system.

---

Anyway, this is how I feel about comprehensive documentation.

How would you propose solving this issue? Maybe create a separate reference for showing how to npm install things and get set up quickly?

I liked @iandees thought on a kind of documentation dependency manager. Would just need to ensure the referenced docs aren't a similar dead-end or your problem isn't really addressed.

Maybe it would be helpful to look at some examples of where there are known gaps and think about how they might be addressed? You've mentioned Jekyll and I second it. I will add a few that I've recently failed to grok:

• Geotype because I just don't know enough about Node to even get started.
• Leaflet Ajax because I don't understand ajax very well and, it turns out, don't understand Leaflet's methods very well. The examples also use what I would consider more advanced approaches to putting a map with some vector data onto a page, obscuring the parts specific to Leaflet Ajax. Mapbox's Guide on Mapbox.js is actually a great intro to Leaflet's methods.
• Making QGIS and ArcMap work at the same time on Windows without breaking Python (I actually have a long list of user forum links because I have to fix everything any time one or the other updates). I know how to make it work but I don't know why it works. I also imagine this is not an uncommon situation for QGIS users and am surprised the docs aren't better.

Don't know if that's helpful or not, but these are all top of mind for me. Happy to be a beta tester if you want to try any of your ideas out as I have very little curse of knowledge.

Additional thought - would it be useful to gather docs that do a good job of spanning the requirements spectrum?

@mikolalysenko

Online education things like Codecademy already tend to have the right flow - there are classes that you basically have done or haven't done and they'll show up in different colors.

How this might work out would be like

• We'd have some central website like turtles.io where we'd have a core team of people who write docs. The docs are heavily based off of existing examples but attempt to follow a similar style and depth.
• The homepage of turtles.io would let you choose your "type of thing" and you can either
  • Link to a landing page that lists / shows relevant docs
  • Make badges for your repo to link directly to individual parts of the docs

Another view. As a user, say I come across a relatively "advanced" module. Let's take https://www.npmjs.com/package/ndarray-fft

If someone comes across the module with zero knowledge of programming, this is not a realistic place for them to start. I actually stumbled across the module yesterday with almost no knowledge of ndarray and zero knowledge of what and fft was.

1. I clicked the link and read about ndarray. Google revealed a nice presentation summary of the project as well.
2. I googled fast fourrier transforms and found a nice wikipedia page.
3. done. I now know what the module is for, and the readme describes the interface.

If every module like this contained a collection of links about npm, node, javascript, CS array concepts, and x, and y.. I think I would quickly be overloaded to the point where following the steps I followed would be more difficult.

The most comprehensive documentation systems I have seen in the wild always tend to devolve into overwhelming monoliths that encourage glossing over important information. If you start to automate this, you run the risk of hitting the same eye-glazing problems caused by systems like doxygen.

If you need a super comprehensive zero-to-hero style document, I think examples like browserify-handbook or stream-handbook provide a better solution. A comprehensive guide outside the main documentation flow allows beginners a place to learn from scratch, without hindering the users who spend 99% of the time using the docs. Maybe a 50/50 solution is creating these guides, and linking to them prominently in the terse readme?

I'd also add that now that's it's easier to write and publish small modules, I've found lists like these to be incredibly useful. Competency in one realm of programming doesn't mean that you're comfortable elsewhere, and seeing what others are using to solve problems you're new to makes it a hell of a lot easier to get started.