Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Thoughts after 6 months of literate programming

Thoughts after 6 months of literate programming

I wrote my first literate program in November of 2015. Since then, I've been writing literate programs on an almost daily basis. It has been an experience with a sort of enlightenment that I haven't had in a long time. Not only is it a lot of fun to write literate programs, I feel like I have gained a new "super power" of sorts.

Below are my thoughts on my experiences with literate programming so far.

Literate programs I've written to date

Project name Lines of code Words in literate document Ratio of words to lines
Token Cleaner 46 496 10.782609
Okta Sign-In Widget 120 3862 32.183333
Okta SCIM Beta 360 7966 22.127778
Dial-a-cat 370 3620 9.7837838
Okta OIDC Beta 752 7731 10.280585

Explanation:

  • Lines of code is the number of lines of "code" generated by the literate program. "code" is in quotes because the code is anything in an Org Babel "source" block, it could be Python, Shell, HTML, etc. This number comes from running this command on an Org Babel file: awk '/BEGIN_SRC/,/END_SRC/' $filename
  • Words in document per lines of code is the number of words in a file, note that in includes the source code as well. This number comes from running this command on an Org Babel file: wc -w
  • Ratio of words to lines is: Words in literate document divided by Lines of code

What I like about literate programming

The expressiveness of literate programming

I feel hobbled by in-line code comments now. One of the clearest benefits of literate programming is being able to weave my code into large sections of prose, instead of interleaving small sections of prose into code.

Now I find myself spending time to add in detailed references an explanations with my code. One example of this is including a curl command, links to RFCs, and a note about "1-indexed" versus "0-indexed" differences in the "Resource Paging" section of my Okta SCIM Beta document. Another example is being able to cover a function line-by-line, as well as include unit tests in-line in the "Validating an OIDC id_token from Okta" section of my Okta OIDC Beta document.

Ability to generate data from tables

Take a look at the table listed in the "Dependencies" section of the Okta SCIM Beta document. Then take a look at the requirements.txt file in the same repository. The dependencies table and the requirements.txt file are both generated from a single table in Org Babel!

This is all made possible by the ability to write in-line code to generate the requirements.txt file and code to generate the table markdown for GitHub Pages.

Being able to include a Copyright notice across multiple files

In the Okta SCIM Beta document, I am able define a copyright notice in one place and have that copyright notice written to a LICENSE.txt file and have the same notice included into the scim-server.py file.

Having shell commands, and output, all in one place

When I was writing the Okta OIDC Beta document, I spent a lot of time finding the arcane openssl incantations needed to generate x509 certificates that I needed for a unit test. Because I was writing a literate document, I was able to include all of the commands that I used, along with a detailed explanation of what each command did.

Being able to execute these commands and see their output without needing to switch to a terminal made it really easy to interactively explore the commands that I needed to generate the certificates, and to keep track of those commands for the next time that I need to generate similar certificates or keys.

Tracking time and future work in the same document

Since Org Babel is part of Org Mode, it is trivial to make use of Org's time tracking and project planning features.

I usually keep track of my time in my usual Org setup, but when appropriate it's nice to be able to keep track of my time inside the same literate document that I'm working on.

You can see an example of time tracking and project planning in the dial-a-cat document. (Look for the lines with CLOCK, DONE, or TODO.

Easily keep code and prose consistent

Being able to keep code and prose synchronized is a problem that I first encountered when I was writing blog posts for Twilio, searching for this "holy grail" is what first got me interested in literate programming. What I wanted was a tool that could generate Markdown and code from the same file, which is exactly what Org Babel can do.

I was sold on Org Babel after I converted my favorite blog post on the Twilio blog to a literate document, and found an inconsistency in the process.

Now, because I know that my code and prose will always be consistent, I find myself making quick changes to code that I wouldn't have made otherwise.

What is the best way to describe code?

Now that I have a tool which makes it easy to write an "essay" about my code, I'm faced with new questions that I never had the luxury of considering before.

I imagine that satisfactorily answering the question "how can I describe this code best?" is a lifelong endeavor.

Here are the specific questions that I find myself facing at this moment:

Literate documents are harder to refactor

Once a section of code is embedded in prose, it becomes a lot harder to change that code.

Because of this, I find that the best way to write a literate program is to start with the code first and wait to turn the code into a literate document after extensive refactoring.

The structure of a document wants to change as a project grows

Describing a short program can be as easy as describing the program from "top to bottom", as I do in my token cleaner project.

However, as a project grows, I'm finding that the structure I used to describe a small project will change when it becomes a medium sized project, and change again when it becomes a large project.

Should I describe every line of every program?

Another thing that I've been struggling with is figuring out the right balance between describing a program line-by-line, or skipping large sections of code entirely. I feel like the ideal method will lean more towards the "line-by-line" approach, but I think I need more hands on experience in this area.

Closing

Literate programming has been a very worthwhile discipline to learn, I still have a lot of things to learn, and I look forward to getting better at writing literate programs in the decades ahead.

I'd love to hear and feedback that you have on this document. Please reach out to me on Twitter!

  • Joël Franusic, May 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.