Skip to content

Instantly share code, notes, and snippets.

View mikedeboer's full-sized avatar

Mike de Boer mikedeboer

View GitHub Profile
@mikedeboer
mikedeboer / entities.js
Last active July 28, 2020 23:28
encode/ decode XML entities
const xmlEntityMap = {
"quot": "34", "amp": "38", "apos": "39", "lt": "60", "gt": "62",
"nbsp": "160", "iexcl": "161", "cent": "162", "pound": "163", "curren": "164",
"yen": "165", "brvbar": "166", "sect": "167", "uml": "168", "copy": "169",
"ordf": "170", "laquo": "171", "not": "172", "shy": "173", "reg": "174",
"macr": "175", "deg": "176", "plusmn": "177", "sup2": "178", "sup3": "179",
"acute": "180", "micro": "181", "para": "182", "middot": "183", "cedil": "184",
"sup1": "185", "ordm": "186", "raquo": "187", "frac14": "188", "frac12": "189",
"frac34": "190", "iquest": "191", "agrave": ["192", "224"], "aacute": ["193", "225"],
"acirc": ["194", "226"], "atilde": ["195", "227"], "auml": ["196", "228"],

Perf: a new streaming API, based on msgpack

=============================================

We use a number of persistence mechanisms in our codebase, one of which is storing stringified JSON to disk. The most notable example of this is session persistence (as used by session restore). We use JSON here, because it's a very developer-friendly format to deal with; it's just JavaScript. Being able to persist 'JS' data structures makes it all the more practical to use. However, there are a few downsides to using JSON-to-disk as a full blown persistence mechanism.

  1. JSON is not streamable; there is (of course) no limit as to how large a blob of JSON may be before it's persisted. When you store large datasets (think: metrics, browser session data), reading it back into memory requires a JSON.parse(blob), which will block the main thread O(size-of-blob)[1] time. In other words: there is no off-main-thread, chunked deserialization mechanism for JSON.
  2. JSON is not compact; there no built-in method to compress s
@mikedeboer
mikedeboer / async_jsm.md
Last active December 16, 2015 20:49
Simplify async code-flows and unify unit tests

Simplify async code-flows and unify unit tests

The thing I always do when I encounter a large code base, is roam though it and make a mental map of the structure; the good parts and ones that would benefit from some TLC. A part I noticed did not yet receive the amount of attention it deserves are unit tests and asynchronous code handling. I'll be blunt about it, here's what I did: I ported the async NodeJS library to a Mozilla-style module and on top of that I built a unit testing library, inspired by Mocha (among others). I can imagine that the fellas I just mentioned don't make you recall anything, so here are two links:

First of all, the Github numbers will show you that they are crazy popular NodeJS libraries. And well-documented. And well-written. And awesome.

@mikedeboer
mikedeboer / reasons_to_NOT_use_promise_libraries.md
Last active December 15, 2015 18:39
10 reasons to NOT use a Promise library

10 reasons to NOT use a Promise library

  1. Promises don't solve the async code flow any better than async helper methods (like https://github.com/caolan/async)
  2. makes debugging a pain when you actually care about a readable call stack. Most Promise implementations blow up your callstack and make the stack origin disappear. Async helpers (basically wrapper methods) maintains the stack and tries to keep the wrapping to a minimum. No magic.
  3. error handling is obfuscated and usually introduces non-optimal code paths by needing to wrap everything in a try...catch block. Additionally, code inside a try...catch is hard to optimize for a JIT compiler (http://jsperf.com/try-catch-performance-overhead).
  4. error/ failure state handling needs to be expressed explicitly, there is no convention thus enforcement to handle errors properly. This makes using promises likely to introduce issues a developer doesn't expect
  5. promises implementations differ a LOT from each other, most often in the area of semantics. T
@mikedeboer
mikedeboer / qpatches.bash
Last active December 15, 2015 05:19
This script takes your Hg queue, unwinds it and applies each and every queue separately. It then generates a patch to a local directory of the qtip, applies the next queue item until it's back to where it started (all queue items applied).
#!/bin/bash
# Author: Mike de Boer <mdeboer AT mozilla.com>
# This script takes your Hg queue, unwinds it if needed and applies each and
# every queue separately. It then generates a patch to a local directory of the
# qtip, applies the next queue item until it's back to where it started.
_gen_patches() {
local i len
local patches=($(hg qapplied 2>/dev/null))
@mikedeboer
mikedeboer / resume_after_next.md
Created July 4, 2012 12:31
Why to call resume() after next()

When we write Connect middlewares that contain async operations, it is good practice to 'pause' the HTTP request with the help of the Connect utility function. When the async operation has finished, the HTTP request is resumed again. Let's have a look at the documentation for this utility:

Pause `data` and `end` events on the given `obj`.
Middleware performing async tasks _should_ utilize
this utility (or similar), to re-emit data once
the async operation has completed, otherwise these
events may be lost.
var jsBase64 = (function() {
var b64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=";
// public method for encoding
function encode(data) {
var o1, o2, o3, h1, h2, h3, h4, bits, i = 0, ac = 0, enc = "",
tmp_arr = [];
if (!data)
/*
* Copyright (c) 2010 Tobias Schneider
* This script is freely distributable under the terms of the MIT license.
*/
(function(){
var UPC_SET = {
"3211": '0',
"2221": '1',
"2122": '2',