Skip to content

Instantly share code, notes, and snippets.

@KevinAst

KevinAst/article.md

Last active March 5, 2023 13:24
Show Gist options
  • Star 52 You must be signed in to star a gist
  • Fork 5 You must be signed in to fork a gist
  • Save KevinAst/7e12648245ff2a8e9c1557135014b933 to your computer and use it in GitHub Desktop.
Save KevinAst/7e12648245ff2a8e9c1557135014b933 to your computer and use it in GitHub Desktop.
Download ZIP
Integrating GitBook with JSDoc to Document Your Open Source Project
Raw
article.md

Integrating GitBook with JSDoc to Document Your Open Source Project

Introduction

Good documentation should include two distinct elements - a Guide and an API:

  1. The Guide builds concepts, providing examples, etc.

    GitBook is well suited to generate the Guide, supporting a variety of options, using standard Markdown.

  2. The API provides low-level specifics, with function signatures, details, etc.

    JSDoc (a descendant of JavaDoc) is ideal to define the low-level API. Using this technique, the API is defined from embedded comments in the code itself.

Previously, I documented my projects with JSDoc exclusively (both the Guide and API). JSDoc has a @tutorial feature that provides a rudimentary ability to create some semblance of a Guide. This technique, however, had several limitations, related to how the Guide was structured. I really wanted to drive my documentation with GitBook, but I didn't want to give up my JSDoc generation of the API.

The question then became: how does one integrate these two technologies (GitBook and JSDoc)? They both generate their own html resources, with their own navigation structure that do NOT integrate (out of the box).

The secret in accomplishing this integration is to use the jsdoc-to-markdown package to generate the low-level API as Markdown (from the JSDoc comments in your code). Once this is accomplished, it can be easily integrated into the overall GitBook structure.

Here is an example of the documentation in one of my projects:

  • before: using JSDoc only
  • and after: using GitBook/JSDoc

At a Glance

This article is a bit lengthy, however I think it provides some valuable insight. You are welcome to "pick and choose" the features promoted here.

The article assumes you have a working knowledge of both GitBook and JSDoc. It is not a comprehensive discussion on either topic. Rather our focus is on the tweaks and adjustments required to make GitBook/JSDoc integration possible. Should you need it, you will find some Useful Links (below) regarding GitBook and JSDoc.

At a glance, we discuss the following topics:

Please Note that the jsdoc-to-markdown package uses the jsdoc2md command. You will see these terms used interchangeably within this article.

Basics

Here is my project structure (focusing on its documentation):

{project-root}/

  package.json ... various npm scripts, INCLUDING docs generation

  book.json   ... GitBook configuration

  src/        ... project source code (jsdoc2md scan root)
    *.js      ... embedded JSDoc comments defining my API documentation

  docs/       ... master source of my GitBook project documentation
    toc.md     .  the summary Table of Contents (defining the left panel navigation)
    intro.md   .  the Guide Introduction
    api.md     .  generated API docs (output from jsdoc2md scanning our src/*.js JSDoc comments)
    *.md       .  various Markdown section/chapter files making up our docs
    sectionN/ ... optional docs dirs (providing more organization in our docs)
      *.md

  tooling/       ... various project build utils (scripts/config/etc.)
    docs/        ... docs related
      jsdoc.json  .  jsdoc2md configuration (for ES6 features)
      *.hbs       .  jsdoc2md template overrides (handlebars)

  _book/      ... machine generated documentation (output of GitBook)
    *.html
    *.js
    *.css

This highlights how the documentation is generated and maintained. The basics of the documentation process is:

  • The docs/api.md Markdown file is machine generated (via jsdoc2md), which scans the JSDoc comments in our source code.

  • Once that is accomplished, we are free to organize our docs however we want - using GitBook.

The key aspect that is critical to understand is you do NOT install/use JSDoc directly. Rather, you install jsdoc-to-markdown and use it to generate the docs/api.md Markdown file that is included in the overall GitBook documentation.

Docs Related Dev Dependencies

In regards to tooling, the documentation process needs two packages:

Install these as dev dependencies:

$ npm install --save-dev gitbook-cli
$ npm install --save-dev jsdoc-to-markdown

If you are using ES6 features in your code, perform the following additional setup (per the jsdoc2md wiki):

  • Install additional dev dependencies (providing you don't have them already):

    $ npm install --save-dev jsdoc-babel
$ npm install --save-dev babel-preset-es2015

  • Setup the following jsdoc config file:

    tooling/
  docs/
    jsdoc.json
    ==========
      {
        "plugins": ["node_modules/jsdoc-babel"],
        "babel": {
          "presets": [ "es2015" ]
        }
      }

  • include the --configure tooling/docs/jsdoc.json option in your jsdoc2md command

Basic GitBook Template

Define the following doc-related files in your project:

book.json (GitBook configuration)
=========
  {
    "gitbook":   "3.2.2",
    "root":      "./docs",
    "structure": { 
      "readme":  "intro.md",
      "summary": "toc.md" 
    }
  }

docs/

  toc.md (defines the left-nav)
  ======
    # Table of content 
    * [Getting Started](start.md)
    * [API Reference](api.md)

  intro.md (docs introduction)
  ========
    # astx-redux-util

    The is our **introduction** to astx-redux-util.

    Here are some references to the [Getting Started](start.md) and [API](api.md).


  start.md (our getting started)
  ========
    # Getting Started

    This is the **Getting Started** section.

    Here are some references to the [Introduction](intro.md) and [API](api.md).

Docs Related NPM Scripts

Setup the following docs-related npm scripts (in your package.json):

  • generate docs/api.md from embedded JSDoc comments in src/*.js (run as needed)

    "docs:api": "jsdoc2md --configure tooling/docs/jsdoc.json --files src/**/*.js > docs/api.md"

  • install gitbook/plugins/dependencies

    "docs:prepare": "gitbook install"

  • build docs into _book/ directory - integrating Dev Guide (GitBook in docs/*) with API (JSDoc comments in src/*.js)

    "docs:build": "npm run docs:prepare && npm run docs:api && gitbook build"

  • build/serve docs (http://localhost:4000/), continuously watching for changes

    "docs:serve": "npm run docs:prepare && npm run docs:api && gitbook serve"

  • clean the machine-generated docs (_book/)

    "docs:clean":   "rimraf _book"

Visualize Your Docs

In order to visually see your GitBook docs, you must promote them through a server (due to GitBook's usage of client-side JavaScript communications and the browser cross-site restrictions of local file systems).

As it turns out, that is not a big deal, because GitBook provides a "serve" utility which launches a test service promoting your docs from the _book/ directory. As a side benefit it also watches for changes in your Markdown files, and re-builds the _book/ destination (life is good).

Simply run the following npm script:

$ npm run docs:prepare # run once (to initialize gitbook)
$ npm run docs:serve

And browse to: http://localhost:4000/

This represents the basic structure of my documentation.

  • You immediately have all the features of GitBook, including a left-nav, search ability, style changer, social media shares, etc.

    gbStart1

  • You can navigate to one of your sections (either through the left-nav or one of your links), and change your style:

    gbStart2

  • And of course, you have access to your API - which was machine generated from the JSDoc comments embedded in your code.

    gbStart3

This pretty much covers the basics! However (as always) there is a bit of tweaking to complete the process. That is what the remainder of this doc will cover. The devil is in the details!

Customizing jsdoc2md

I have applied several customizations to the jsdoc2md utility to make it both functional, and make it really stand out.

Overriding jsdoc2md Output

Several enhancements that follow, require changes to the jsdoc2md output. jsdoc2md uses handlebars as it's template engine. You can easily override any of the templates by using the jsdoc2md custom partial feature.

All jsdoc2md templates are found in the dependent dmd package (within your node_modules). The following command will list all the jsdoc2md handlebars templates:

$ cd node_modules/dmd

$ find . | grep hbs

  partials/all-docs/all-docs.hbs
  partials/all-docs/docs/body/access.hbs
  ... snip snip
  partials/main.hbs ...  THE STARTING POINT
  ... snip snip
  partials/shared/value/link.hbs
  partials/shared/value/linked-type-list.hbs

The starting template is main.hbs. To find a template of interest, either work your way through the various templates (starting with main.hbs), or simply perform a recursive grep for some well-known phrase (from your current output).

To override the hbs template, simply make a modified copy in tooling/docs/, and reference it with the jsdoc2md --partial option (in your npm docs:api script found in package.json). NOTE: It is important to use the same file name, because filenames are significant in handlebars.

$ jsdoc2md --partial tooling/docs/<file>.hbs ...other-ops

Indexing the API from GitBook

We are going to want the GitBook left-nav to index various parts of the API (i.e. the machine generated docs/api.md). Before we can accomplish this, we need to apply some adjustments:

  1. Eliminate Internal API Index

    First of all, by default, jsdoc2md will embed it's own index in the generated api.md, when it accumulates multiple identifiers. This is not needed, because we will be using our own GitBook indexing scheme. This is easily disabled by using the --global-index-format none jsdoc2md command-line option. Simply add this to your npm docs:api script in package.json.

  2. Fix API Anchors

    Out of the box, jsdoc2md creates anchors using name <a name="ref">, instead of id <a id="ref".

    Unfortunately, the GitBook markdown parser requires id to make a section "linkable". The reason jsdoc2md operates this way is it's primary focus is for Github-based markdown - which requires name (see Issue122).

    This is easily fixed by overriding the header.hbs handlebar partial (node_modules/dmd/partials/all-docs/docs/header.hbs).

    • make a copy in tooling/docs/header.hbs
    • modify, changing name= to id=
    • add --partial tooling/docs/header.hbs option to your npm docs:api script

    You can see this resource in my project's tooling/docs/header.hbs.

  3. Add GitBook Index to API

    Now we can index the individual API components by simply adding the appropriate entries to the GitBook docs/toc.md (your entries will obviously vary):

    * [API Reference](api.md)
  * [conditionalReducer](api.md#conditionalReducer)
  * [joinReducers](api.md#joinReducers)
  * [reducerHash](api.md#reducerHash)

    Now the left-nav promotes each API component of interest:

    gbApiIndex

Improve API Headers

While we are at it, we can improve on the look of the function headers within the API.

Once again, this is a change to the tooling/docs/header.hbs handlebar partial. Modify it to:

  • to use an h5 html tag styled with a border
  • add more spacing so that when indexed, the prior entry is not visible

You can see this resource in my project's tooling/docs/header.hbs.

Now the header is more ascetically pleasing:

gpApiHeader

Make API Types Linkable

Out of the box, all API code references (i.e. function return types and parameter types) are NOT linkable. Here is the generated markdown output:

**Returns**: <code>[reducerFn](#reducerFn)</code> - snip snip

After a bit of research, I discovered that Gitbook-flavored markdown will NOT honor hyperlinks embedded in a code block (i.e. the <code> element).

As previously mentioned, jsdoc2md is attempting to accommodate the "least common denominator" from the scores of markdown parsers (see Issue123).

This was easily fixed by overriding the link.hbs handlebar partial (node_modules/dmd/partials/shared/value/link.hbs).

  • make a copy in tooling/docs/link.hbs
  • modify, removing the outer <code> tags, replacing them with in-line code backticks (within the link).
  • add --partial tooling/docs/link.hbs option to your npm docs:api script

You can see this resource in my project's tooling/docs/link.hbs.

Now the references are coded as follows AND my return types, and parameter types are linkable!

**Returns**: [`reducerFn`](#reducerFn) - snip snip

Minor API Adjustments

Each component of the machine-generated API:

  • has a rather unconventional Kind section, which typically emits something like "global function" -or- "global typedef"
  • outputs the function Returns section before the parameter list

This is fixed by overriding the body.hbs handlebar partial (node_modules/dmd/partials/all-docs/docs/body/body.hbs).

  • make a copy in tooling/docs/body.hbs
  • modify:
    • removing scope (which emits Kind:),
    • and moving returns after params
  • add --partial tooling/docs/body.hbs option to your npm docs:api script

You can see this resource in my project's tooling/docs/body.hbs.

Customizing GitBook

For the most part, no functional GitBook customization is required (like what we saw in jsdoc2md). With that said, there are a couple of things that are worth highlighting:

Book Styling

GitBook allows you to provide styling adjustments to your book through css overrides. Simply create a docs/styles/website.css resource. This is automatically referenced by GitBook (the convention of finding a styles/website.css in the root of your book).

I would suggest minimizing this to non-vulnerable areas. For example, I like to condense unused space wherever I find it. Here is my css overrides (docs/styles/website.css):

/* slim left-nav width (was: 300px) */
.book-summary {
  width: 250px;
}
.book.with-summary .book-body {
  left: 250px;
}

/* condense left-nav spacing (was: 10px 15px) */
.book-summary ul.summary li a, .book-summary ul.summary li span {
  padding: 5px 15px;
}

/* condense line-spacing */
.markdown-section {
  line-height: 1.4;     /* overall line-spacing (was: 1.7) */
}
.markdown-section pre {
  line-height: 1.2;     /* code-snippet line-spacing (was: 1.7) */
  padding:    .5em;     /* also reduce padding around code block (was: .85em 1em) */
}

Toolbar Adjustments

The top part of every GitBook page contains a header (referred to as the "GitBook Website Toolbar"). This is where you find various controls, such as left-nav toggle, style changer, social media shares, etc. This toolbar is customizable through GitBook plugins.

  1. Social Media Icons

    By default, GitBook uses a "sharing" plugin that promotes various social media sharing options.

    You can disable this plugin by adding the "-sharing" plugin item (in your book.json file):

    plugins": [ "-sharing"],

    If you decide to keep this plugin, you can configure it through the "pluginsConfig.sharing" book.json section:

    "pluginsConfig": {
  "sharing": {
      "facebook":   false,
      "twitter":    false,
      "google":     false,
      "weibo":      false,
      "instapaper": false,
      "vk":         false,
      "all": ["twitter", "facebook", "google"]
  }
}

  2. Toolbar Icons

    To embellish the toolbar with various icons, I use the "toolbar" GitBook plugin. Modify your book.json to include the following (changing the URL appropriately):

      "plugins": ["toolbar"],
  "pluginsConfig": {
    "toolbar": {
        "buttons": [
          {
            "label": "GitHub",
            "icon": "fa fa-github",
            "url": "https://github.com/KevinAst/astx-redux-util"
          },
          {
            "label": "NPM",
            "icon": "fa fa-bullseye",
            "url": "https://www.npmjs.com/package/astx-redux-util"
          }
        ]
    }
  }

Template Links

All links in GitBook use the standard Markdown link link syntax, for example:

[My Section](/section1/mySection.mk#sub-section)

One of the things I liked about using JSDoc (for both Guide and API) was the ability to seamlessly link from one part of the doc to another. Using JSDoc you can use inline tags to link to another part of the documentation:

The nice thing about this is it encapsulates BOTH:

  • the anchor tag (no matter where it lives in your doc)
  • and it's visible name (so you don't have to repeat this text over-and-over).

As it turns out, we can accomplish the same thing using GitBook templates. We simply register each desired link as a GitBook variable, and then reference that variable through GitBook templates.

As an example, given the following variable definition (in book.json):

  "variables": { 
    "guide": {
      "intro": "[Introduction](/intro.md)",
      "start": "[Getting Started](/start.md)"
    },
    "api": {
      "ref":                  "[API Reference](/api.md)",
      "conditionalReducer":   "[`conditionalReducer()`](/api.md#conditionalReducer)",
      "joinReducers":         "[`joinReducers()`](/api.md#joinReducers)",
      "reducerHash":          "[`reducerHash()`](/api.md#reducerHash)"
    }
  }

You can reference them within your doc through a GitBook template:

  THIS: 
  Here are some references to the {{book.guide.start}} and {{book.api.reducerHash}}.

  IS EQUIVALENT TO THIS:
  Here are some references to the [Getting Started](/start.md) and [`reducerHash()`](/api.md#reducerHash).

BE AWARE: If you reference a non-existent GitBook variable (in your template), is simply expands to nothing.

BEST PRACTICE: Use Absolute Link References! Notice that in the example above, all links are absolute (i.e. they begin with /). This allows the link to be used anywhere within your doc, even when you employ a rich docs directory structure (i.e. organizing files in various directories). Please note that absolute is defined from your documentation root. Even if you deploy your docs in a versioned server directory, it still works!

SideBar: If you define an invalid link address, it simply silently does NOT work. If you activate a browser console (F12), you will see a VERY LAME indicator that the anchor is NOT found ... something like: "Uncaught TypeError: Cannot read property 'top' of undefined". This is the result of a GitBook JavaScript module NOT properly recognizing this error condition. SideBar: If you wish to track this, I submitted GitBook Issue58.

Project Name and Version

It is a good practice to prominently promote the project name and current version. The easiest way to accomplish this is to inject it at the top of the left-nav (via the toc.md) making it visible on every page.

Currently, I am just hard-coding this. Simply add the following to the top of your docs/toc.md file (your info will vary):

### astx-redux-util (1.0.0)

Please note It would be nice to pull this from the package.json file (especially because the version is so volatile). I played with the packageinfo GitBook plugin, but the required template usage is currently NOT honored in the toc.md expansion (a GitBook limitation).

Now the doc appears as follows:

gpVersion

Disable Live Reloading

By default GitBook uses the livereload plugin which implements a live-reloading scheme that watches for changes in your book. Information on this plugin is very scarce (as you can see from the link above), but here are some point of interest:

  • In general it seems to slow everything down.

  • When deployed in production (on GitHub Pages), the browser exhibits a long-running spinner and eventually times out trying to connect to:

    https://astx-redux-util.js.org:35729/livereload.js

  • Because of this, I would HIGHLY recommend disabling this plugin!

    Simply add the "-livereload" plugin item (in your book.json file):

    plugins": [ "-livereload"],

  • When this plugin is disabled, life is good ... fast loading, and no timeouts in the production deployment!

Deployment

I use Github Project Pages to deploy my documentation. There are several options here, but I use the gh-pages branch of my project to auto deploy it's documentation. SideBar: You can also register your documentation site as a sub-domain of js.org.

The basic process of deploying my docs is to push the content of the _book/ directory to the gh-pages branch of my project (using git).

I currently do this by hand (via git), because I maintain a top-level version directory in my gh-pages branch - allowing multiple versions of my documentation to co-exist.

If versioned documentation is not a requirement, I would recommend using the popular gh-pages npm package, which allows you to deploy your docs with one simple npm script:

  "scripts": {
    "docs:publish": "gh-pages --dist _book"
  }

Final Result

My astx-redux-util Github project employs all of these documentation techniques. Use it as a complete and comprehensive example.

The documentation for this project can be found at https://astx-redux-util.js.org/.

gpVersion

Useful Links

GitBook:

jsdoc-to-markdown:

JSDoc:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment