Skip to content

Instantly share code, notes, and snippets.



View GitHub Profile
RangerMauve /
Created January 17, 2023 05:13 — forked from HerbCaudill/
Cleaned-up transcript of Rich Hickey's talk "Deconstructing the Database"

This is a talk I keep referring back to, and I wanted to have it in text form. I grabbed the raw machine-generated transcript from YouTube and used GPT-3 to help me turn it into well-punctuated sentences and paragraphs. I had to do some additional cleanup, but it got me most of the way there - my first experience getting AI to help me out with a real task!

Deconstructing the Database

Rich Hickey, author of Clojure, and designer of Datomic presents a new way to look at database architectures in this talk from JaxConf 2012.

The title of this talk is "Deconstructing the Database".

RangerMauve / merge-post-examples.js
Created March 8, 2022 00:18
Merge post searches for USHIN-db
View merge-post-examples.js
Start searches on each DB
Start pulling values from each iterator in a "puller"
"pullers" keep track of the latest state of an iterator, and enable pulling more data out
While there is still new data:
Find the puller with the latest piece of data
Get it's value
Pull the next item in that iterator
Yield the found value
RangerMauve /
Created July 20, 2021 17:15

Hyperswarm Web 2

  • Actively prioritize WebRTC connections
    • Before looking up on hyperswarm-proxy, do WebRTC search
  • Optimize for fast initial connection
    • Debug performance to get the first peer ASAP
  • Strip out unneccessary dependencies
    • discrovery-swarm-webrtc has a lot of stuff in it, use lower levels instead
    • Use MMST directly to reduce overall connections
  • Agressive deduplication
RangerMauve /
Last active May 16, 2021 00:22
Hyperbee Indexed DB

Hyperbee Indexed DB


Hyperbee is super useful for storing data and searching for data using it's ordered key search. When you set up your keyspace just right, you can query a small subset of a large database and quickly load what you need without having to download the entire dataset or traverse it.

This module seeks to make it easier to set up these indexes for JSON data and to make it easy to query large datasets.

How it works

RangerMauve /
Created April 19, 2021 21:55
Hypercore Access Control

Acess Control In Hypercore

Allow Lists

One way to revoke access to a hypercore is to control who is allowed to replicate a hypercore.

WHen you create hypercore replication stream you can control when feeds can be added or removed.

As well, after a replication stream is created you can see a peer's remote public key which can be used to securely identify a peer in the network.

RangerMauve / filesystem.js
Created March 27, 2021 21:11
Test opening a write stream for the FileSystem API
View filesystem.js
// Execute this in Chrome Dev Tools on an HTTPs web page
// Choose a file on your FS
[fileHandle] = await window.showOpenFilePicker();
// This will throw an error on Electron 12
writable = await fileHandle.createWritable();
RangerMauve /
Created April 24, 2020 22:02
GNOME Palette

GNOME Palette

  • Kinda like web-palette but for Linux desktop
  • Use GJS's AT-SPI bindings
  • Search for every button / heading / hyperlink on the page
  • Generate a palette which lets you fuzzy match
  • Integrate with shell as a search providr? Look at this extension
RangerMauve / DatArchive.dream.js
Created August 27, 2019 19:27
Dream DatArchive API for extensions / peers
View DatArchive.dream.js
const archive = await DatArchive.load('someurl', {
// Gets passed down to hyperdrive constructor
extensions: ['example', 'foo', 'bar']
archive.addEventListener('extension', ({detail}) => {
const {type, message, peer} = detail
if(type === 'foo') {
// Similar to the hypercore API:
RangerMauve /
Last active July 30, 2019 22:04
Corestore API proposal

Corestore API Proposal


  • Storing multiple hypercores in a single "group"
  • Purging storage for a given hypercore key
  • Able to replicate whole group
  • Specifying which key should be used for replication handshake

API (from the SDK)

View gist:62501c98b42c63f2de6ee0644a6852ff
WebRTC, unlike Dat, requires doing a handshake through a third party.
With beaker, that third party can be the existing P2P connections that you have.
To connect between two peers you need to do the following:
- Create a Dat Archive for the call website
- Listen on messages in the [dat-peers API](
- Have the user enter a "room"
- Set up the WebRTC connection, send the SDP handshake and the `room` through datPeers
- When you get messages for your room, pass them to the local PeerConnection as being remote SDP or ICE