HerbCaudill

Local-first resources

Background

• Automerge: Making servers optional for real-time collaboration A very clear explanation by Martin Kleppmann of how CRDTs work, along with an introduction to Automerge. This is the video that started me down this whole road, as documented in this blog post
• Local-first software: You own your data, in spite of the cloud The manifesto that gave a name to the concept and that still stands as the best introduction to it; by Martin Kleppmann & Peter Van Hardenberg et al.
• localfirst.fm A podcast by Johannes Schickling about local-first software development. The first episode with Peter Van Hardenberg gives a good overview of the current state of the art.

Getting started

HerbCaudill

localfirst/auth provider for Automerge Repo

This is pseudo-documentation for a hypothetical auth provider for Automerge Repo built around @localfirst/auth.

Configuration

A LocalFirstAuthProvider is configured with information about the local user and device.

import { LocalFirstAuthProvider, createUser, createDevice } from 'automerge-repo-auth-localfirstauth'

HerbCaudill

import { exec as _exec } from 'child_process'
import fs from 'fs'
import path from 'path'
import { fileURLToPath } from 'url'
import { promisify } from 'util'

const exec = promisify(_exec)

// ensure outputDir exists
const __dirname = fileURLToPath(new URL('.', import.meta.url))

HerbCaudill

  // Available variables:
  // - Machine
  // - interpret
  // - assign
  // - send
  // - sendParent
  // - spawn
  // - raise
  // - actions

HerbCaudill

Proposed message types

type SyncMessage = {
  type: 'SYNC_MESSAGE'
  senderId: PeerId
  recipientId: PeerId
  documentId: DocumentId
  payload: Uint8Array // Automerge binary sync message
}

HerbCaudill

This is a talk I keep referring back to, and I wanted to have it in text form. I grabbed the raw machine-generated transcript from YouTube and used GPT-3 to help me turn it into well-punctuated sentences and paragraphs. I had to do some additional cleanup, but it got me most of the way there - my first experience getting AI to help me out with a real task!

---

Deconstructing the Database

Rich Hickey, author of Clojure, and designer of Datomic presents a new way to look at database architectures in this talk from JaxConf 2012. https://www.youtube.com/watch?v=Cym4TZwTCNU

The title of this talk is "Deconstructing the Database".

HerbCaudill

Automerge repository API

A replicated key-value store that magically synchronizes with peers in the background.

Why

Automerge is a CRDT that ...

The original Automerge API leaves a number of difficult problems to be solved in userland: Storage, network communication, and synchronization are all left as an exercise for the developer.

HerbCaudill

/* dynalist.css */

/* tags */

.node-tag {
  background: rgba(255, 165, 0, 0.8);
  font-size: 0.8em;
  border-radius: 0.3em;
  font-weight: bold;
  padding: 2px 5px;

HerbCaudill

git branch -m master main
git push -u origin main

git push origin --delete master

@REM will probably get an error on the last line
@REM go to https://github.com/XXXXX/XXXXX/settings/branches and change default branch
@REM then retry the last line

HerbCaudill

import { BloomFilter } from 'bloomfilter' // https://github.com/jasondavies/bloomfilter.js

/**
 * Returns an optimally configured Bloom filter for the given number of elements, using the
 * calculations from https://www.di-mgt.com.au/bloom-filter.html .
 * @param n The number of elements to be added to the Bloom filter
 * @param p The target false positive rate
 */
const optimalBloomFilter = (n: number, p: number = 0.01) => {
  const LOG2 = Math.log(2)