RangerMauve

This is a talk I keep referring back to, and I wanted to have it in text form. I grabbed the raw machine-generated transcript from YouTube and used GPT-3 to help me turn it into well-punctuated sentences and paragraphs. I had to do some additional cleanup, but it got me most of the way there - my first experience getting AI to help me out with a real task!

---

Deconstructing the Database

Rich Hickey, author of Clojure, and designer of Datomic presents a new way to look at database architectures in this talk from JaxConf 2012. https://www.youtube.com/watch?v=Cym4TZwTCNU

The title of this talk is "Deconstructing the Database".

RangerMauve

/*
Start searches on each DB
Start pulling values from each iterator in a "puller"
"pullers" keep track of the latest state of an iterator, and enable pulling more data out
While there is still new data:
Find the puller with the latest piece of data
Get it's value
Pull the next item in that iterator
Yield the found value

RangerMauve

Hyperswarm Web 2

• Actively prioritize WebRTC connections
  • Before looking up on hyperswarm-proxy, do WebRTC search
• Optimize for fast initial connection
  • Debug performance to get the first peer ASAP
• Strip out unneccessary dependencies
  • discrovery-swarm-webrtc has a lot of stuff in it, use lower levels instead
  • Use MMST directly to reduce overall connections
• Agressive deduplication

RangerMauve

Hyperbee Indexed DB

Purpose:

Hyperbee is super useful for storing data and searching for data using it's ordered key search. When you set up your keyspace just right, you can query a small subset of a large database and quickly load what you need without having to download the entire dataset or traverse it.

This module seeks to make it easier to set up these indexes for JSON data and to make it easy to query large datasets.

How it works

RangerMauve

Acess Control In Hypercore

Allow Lists

One way to revoke access to a hypercore is to control who is allowed to replicate a hypercore.

WHen you create hypercore replication stream you can control when feeds can be added or removed.

As well, after a replication stream is created you can see a peer's remote public key which can be used to securely identify a peer in the network.

RangerMauve

// Execute this in Chrome Dev Tools on an HTTPs web page

// Choose a file on your FS
[fileHandle] = await window.showOpenFilePicker();

// This will throw an error on Electron 12
writable = await fileHandle.createWritable();

RangerMauve

GNOME Palette

• Kinda like web-palette but for Linux desktop
• Use GJS's AT-SPI bindings
• Search for every button / heading / hyperlink on the page
• Generate a palette which lets you fuzzy match
• Integrate with shell as a search providr? Look at this extension

RangerMauve

const archive = await DatArchive.load('someurl', {
  // Gets passed down to hyperdrive constructor
  extensions: ['example', 'foo', 'bar']
})

archive.addEventListener('extension', ({detail}) => {
 const {type, message, peer} = detail
 
 if(type === 'foo') {
   // Similar to the hypercore API: https://github.com/mafintosh/hypercore#feedonextension-name-message-peer

RangerMauve

Corestore API Proposal

Requirements

• Storing multiple hypercores in a single "group"
• Purging storage for a given hypercore key
• Able to replicate whole group
• Specifying which key should be used for replication handshake

API (from the SDK)

RangerMauve

WebRTC, unlike Dat, requires doing a handshake through a third party.
With beaker, that third party can be the existing P2P connections that you have.

To connect between two peers you need to do the following:

- Create a Dat Archive for the call website
- Listen on messages in the [dat-peers API](https://beakerbrowser.com/docs/apis/experimental-datpeers)
- Have the user enter a "room"
- Set up the WebRTC connection, send the SDP handshake and the `room` through datPeers
- When you get messages for your room, pass them to the local PeerConnection as being remote SDP or ICE