Skip to content

Instantly share code, notes, and snippets.

@xificurC

xificurC/println_driven_development_with_a_twist.html

Created March 24, 2023 15:32
Show Gist options
  • Save xificurC/21e211fdb5e8845d4d46137f2748d6bf to your computer and use it in GitHub Desktop.
Save xificurC/21e211fdb5e8845d4d46137f2748d6bf to your computer and use it in GitHub Desktop.
Download ZIP
Raw
println_driven_development_with_a_twist.html
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<!-- 2023-03-24 Pi 16:05 -->
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Println Driven Development, with a twist</title>
<meta name="author" content="Peter Nagy" />
<meta name="generator" content="Org Mode" />
<style>
#content { max-width: 60em; margin: auto; }
.title { text-align: center;
margin-bottom: .2em; }
.subtitle { text-align: center;
font-size: medium;
font-weight: bold;
margin-top:0; }
.todo { font-family: monospace; color: red; }
.done { font-family: monospace; color: green; }
.priority { font-family: monospace; color: orange; }
.tag { background-color: #eee; font-family: monospace;
padding: 2px; font-size: 80%; font-weight: normal; }
.timestamp { color: #bebebe; }
.timestamp-kwd { color: #5f9ea0; }
.org-right { margin-left: auto; margin-right: 0px; text-align: right; }
.org-left { margin-left: 0px; margin-right: auto; text-align: left; }
.org-center { margin-left: auto; margin-right: auto; text-align: center; }
.underline { text-decoration: underline; }
#postamble p, #preamble p { font-size: 90%; margin: .2em; }
p.verse { margin-left: 3%; }
pre {
border: 1px solid #e6e6e6;
border-radius: 3px;
background-color: #f2f2f2;
padding: 8pt;
font-family: monospace;
overflow: auto;
margin: 1.2em;
}
pre.src {
position: relative;
overflow: auto;
}
pre.src:before {
display: none;
position: absolute;
top: -8px;
right: 12px;
padding: 3px;
color: #555;
background-color: #f2f2f299;
}
pre.src:hover:before { display: inline; margin-top: 14px;}
/* Languages per Org manual */
pre.src-asymptote:before { content: 'Asymptote'; }
pre.src-awk:before { content: 'Awk'; }
pre.src-authinfo::before { content: 'Authinfo'; }
pre.src-C:before { content: 'C'; }
/* pre.src-C++ doesn't work in CSS */
pre.src-clojure:before { content: 'Clojure'; }
pre.src-css:before { content: 'CSS'; }
pre.src-D:before { content: 'D'; }
pre.src-ditaa:before { content: 'ditaa'; }
pre.src-dot:before { content: 'Graphviz'; }
pre.src-calc:before { content: 'Emacs Calc'; }
pre.src-emacs-lisp:before { content: 'Emacs Lisp'; }
pre.src-fortran:before { content: 'Fortran'; }
pre.src-gnuplot:before { content: 'gnuplot'; }
pre.src-haskell:before { content: 'Haskell'; }
pre.src-hledger:before { content: 'hledger'; }
pre.src-java:before { content: 'Java'; }
pre.src-js:before { content: 'Javascript'; }
pre.src-latex:before { content: 'LaTeX'; }
pre.src-ledger:before { content: 'Ledger'; }
pre.src-lisp:before { content: 'Lisp'; }
pre.src-lilypond:before { content: 'Lilypond'; }
pre.src-lua:before { content: 'Lua'; }
pre.src-matlab:before { content: 'MATLAB'; }
pre.src-mscgen:before { content: 'Mscgen'; }
pre.src-ocaml:before { content: 'Objective Caml'; }
pre.src-octave:before { content: 'Octave'; }
pre.src-org:before { content: 'Org mode'; }
pre.src-oz:before { content: 'OZ'; }
pre.src-plantuml:before { content: 'Plantuml'; }
pre.src-processing:before { content: 'Processing.js'; }
pre.src-python:before { content: 'Python'; }
pre.src-R:before { content: 'R'; }
pre.src-ruby:before { content: 'Ruby'; }
pre.src-sass:before { content: 'Sass'; }
pre.src-scheme:before { content: 'Scheme'; }
pre.src-screen:before { content: 'Gnu Screen'; }
pre.src-sed:before { content: 'Sed'; }
pre.src-sh:before { content: 'shell'; }
pre.src-sql:before { content: 'SQL'; }
pre.src-sqlite:before { content: 'SQLite'; }
/* additional languages in org.el's org-babel-load-languages alist */
pre.src-forth:before { content: 'Forth'; }
pre.src-io:before { content: 'IO'; }
pre.src-J:before { content: 'J'; }
pre.src-makefile:before { content: 'Makefile'; }
pre.src-maxima:before { content: 'Maxima'; }
pre.src-perl:before { content: 'Perl'; }
pre.src-picolisp:before { content: 'Pico Lisp'; }
pre.src-scala:before { content: 'Scala'; }
pre.src-shell:before { content: 'Shell Script'; }
pre.src-ebnf2ps:before { content: 'ebfn2ps'; }
/* additional language identifiers per "defun org-babel-execute"
in ob-*.el */
pre.src-cpp:before { content: 'C++'; }
pre.src-abc:before { content: 'ABC'; }
pre.src-coq:before { content: 'Coq'; }
pre.src-groovy:before { content: 'Groovy'; }
/* additional language identifiers from org-babel-shell-names in
ob-shell.el: ob-shell is the only babel language using a lambda to put
the execution function name together. */
pre.src-bash:before { content: 'bash'; }
pre.src-csh:before { content: 'csh'; }
pre.src-ash:before { content: 'ash'; }
pre.src-dash:before { content: 'dash'; }
pre.src-ksh:before { content: 'ksh'; }
pre.src-mksh:before { content: 'mksh'; }
pre.src-posh:before { content: 'posh'; }
/* Additional Emacs modes also supported by the LaTeX listings package */
pre.src-ada:before { content: 'Ada'; }
pre.src-asm:before { content: 'Assembler'; }
pre.src-caml:before { content: 'Caml'; }
pre.src-delphi:before { content: 'Delphi'; }
pre.src-html:before { content: 'HTML'; }
pre.src-idl:before { content: 'IDL'; }
pre.src-mercury:before { content: 'Mercury'; }
pre.src-metapost:before { content: 'MetaPost'; }
pre.src-modula-2:before { content: 'Modula-2'; }
pre.src-pascal:before { content: 'Pascal'; }
pre.src-ps:before { content: 'PostScript'; }
pre.src-prolog:before { content: 'Prolog'; }
pre.src-simula:before { content: 'Simula'; }
pre.src-tcl:before { content: 'tcl'; }
pre.src-tex:before { content: 'TeX'; }
pre.src-plain-tex:before { content: 'Plain TeX'; }
pre.src-verilog:before { content: 'Verilog'; }
pre.src-vhdl:before { content: 'VHDL'; }
pre.src-xml:before { content: 'XML'; }
pre.src-nxml:before { content: 'XML'; }
/* add a generic configuration mode; LaTeX export needs an additional
(add-to-list 'org-latex-listings-langs '(conf " ")) in .emacs */
pre.src-conf:before { content: 'Configuration File'; }
table { border-collapse:collapse; }
caption.t-above { caption-side: top; }
caption.t-bottom { caption-side: bottom; }
td, th { vertical-align:top; }
th.org-right { text-align: center; }
th.org-left { text-align: center; }
th.org-center { text-align: center; }
td.org-right { text-align: right; }
td.org-left { text-align: left; }
td.org-center { text-align: center; }
dt { font-weight: bold; }
.footpara { display: inline; }
.footdef { margin-bottom: 1em; }
.figure { padding: 1em; }
.figure p { text-align: center; }
.equation-container {
display: table;
text-align: center;
width: 100%;
}
.equation {
vertical-align: middle;
}
.equation-label {
display: table-cell;
text-align: right;
vertical-align: middle;
}
.inlinetask {
padding: 10px;
border: 2px solid gray;
margin: 10px;
background: #ffffcc;
}
#org-div-home-and-up
{ text-align: right; font-size: 70%; white-space: nowrap; }
textarea { overflow-x: auto; }
.linenr { font-size: smaller }
.code-highlighted { background-color: #ffff00; }
.org-info-js_info-navigation { border-style: none; }
#org-info-js_console-label
{ font-size: 10px; font-weight: bold; white-space: nowrap; }
.org-info-js_search-highlight
{ background-color: #ffff00; color: #000000; font-weight: bold; }
.org-svg { }
</style>
</head>
<body>
<div id="content" class="content">
<h1 class="title">Println Driven Development, with a twist</h1>
<div id="table-of-contents" role="doc-toc">
<h2>Table of Contents</h2>
<div id="text-table-of-contents" role="doc-toc">
<ul>
<li><a href="#org679350d">1. Tracing</a></li>
<li><a href="#orgf08504a">2. Electric tracing</a></li>
<li><a href="#orgedf6277">3. This was hard to build, right?</a></li>
<li><a href="#orga3bc4ba">4. What&rsquo;s next?</a></li>
</ul>
</div>
</div>
<p>
There are 2 main approaches to developing and debugging code:
</p>
<ul class="org-ul">
<li>the IDE developers. They use all the fancy tools their editor provides to gain
leverage.</li>
<li>the <code>println</code> developers. They sprinkle a few of them here and there to
surgically introspect the running system where their mental model of the code
has gaps.</li>
</ul>
<p>
Both of these have their pros and cons. Many developers have a strong opinion
which is better. Ultimately these options are very black and white though and
have a number of hard to solve drawbacks:
</p>
<ul class="org-ul">
<li>an IDE will overwhelm you with data not important for your task at hand</li>
<li>if an IDE doesn&rsquo;t provide you with the right information with the right
representation you&rsquo;ll either build it yourself, which costs a lof of time, or
not have it at all</li>
<li>printlns don&rsquo;t scale to more than 2 or 3</li>
<li>printlns in loops or core libraries will spam</li>
<li>printlns lose the big picture view</li>
</ul>
<p>
Can we find another approach that solves these issues?
</p>
<div id="outline-container-org679350d" class="outline-2">
<h2 id="org679350d"><span class="section-number-2">1.</span> Tracing</h2>
<div class="outline-text-2" id="text-1">
<p>
You might have seen OpenTracing, OpenTelemetry and their implementations like
Jaeger, Zipkin or Grafana Tempo. These distributed tracing solutions mix the
println and IDE approach:
</p>
<ul class="org-ul">
<li>you decide what data to show; no clutter</li>
<li>the results are correlated; don&rsquo;t lose the big picture</li>
<li>you get a nice UI; see the tree, not a log</li>
</ul>
<p>
The distributed part is essential to correlate across multiple services, which
is more and more common. But let&rsquo;s not forget the local part either! Tracing
allows you to quickly build a personalized view of the problem you&rsquo;re solving.
It&rsquo;s the fastest way to get the developer UI you need.
</p>
</div>
</div>
<div id="outline-container-orgf08504a" class="outline-2">
<h2 id="orgf08504a"><span class="section-number-2">2.</span> Electric tracing</h2>
<div class="outline-text-2" id="text-2">
<p>
During our hack week at Hyperfiddle we focused on the local part - build a
client-side trace view to introspect your system. Using it is just as easy as
sprinkling <code>println</code>&rsquo;s but the leverage is amplified.
</p>
<p>
Here&rsquo;s a simple trace output of a fizzbuzz implementation:
</p>
<div id="org26f7e9a" class="figure">
<p><img src="https://i.imgur.com/NWFoDap.png" alt="fizzbuzz.png" />
</p>
<p><span class="figure-number">Figure 1: </span>FizzBuzz trace</p>
</div>
<p>
At a quick glance we can see the output, the input number, the strings we
generate. But we can also see Electric work skipping (the duplicate <code>nil</code> didn&rsquo;t
emit) and measure the time it took to get from one trace to another (not visible
on the screenshot). A lof of information for little work.
</p>
<p>
Here is another screenshot of a bug we will be looking at shortly:
</p>
<div id="org953e2f8" class="figure">
<p><img src="https://i.imgur.com/VEC8stA.png" alt="xtdb_todo_list_infinite_loop_bug.png" />
</p>
<p><span class="figure-number">Figure 2: </span>XTDB infinite loop on checkbox mark</p>
</div>
<p>
The callback never finishes and the <code>:status</code> field is messed up. But the
checkbox returns new values, which is unexpected! The bug isn&rsquo;t fixed by looking
at the trace but it gives quick and valuable insights.
</p>
</div>
</div>
<div id="outline-container-orgedf6277" class="outline-2">
<h2 id="orgedf6277"><span class="section-number-2">3.</span> This was hard to build, right?</h2>
<div class="outline-text-2" id="text-3">
<p>
Not at all! The <code>trace</code> macro is ~12LOC! The UI is a bit bigger but clocks in at
~200LOC with tests, comments and an inlined sha256 implementation that will be
factored out. Since Electric Clojure is fully reactive it&rsquo;s trivial to build
dynamic, reactive, realtime UIs. The DX is great, too, the feedback loops are
quick and the abstractions allow concentrating on the essential complexity. The
whole project took only ~5 man-days including design, implementation, testing,
demos, and this blog post, too.
</p>
</div>
</div>
<div id="outline-container-orga3bc4ba" class="outline-2">
<h2 id="orga3bc4ba"><span class="section-number-2">4.</span> What&rsquo;s next?</h2>
<div class="outline-text-2" id="text-4">
<p>
We&rsquo;ll be dogfooding this tool and thereby further improve it. We encourage our
users to try it out, too! And in the future we might push this a layer deeper so
we can trace on the server as well and get distributed traces!
</p>
</div>
</div>
</div>
<div id="postamble" class="status">
<p class="author">Author: Peter Nagy</p>
<p class="date">Created: 2023-03-24 Pi 16:05</p>
</div>
</body>
</html>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment