juliangruber

#!/usr/bin/env bash

ipv4ranges=(
  10.0.0.0/8
  100.64.0.0/10
  169.254.0.0/16
  172.16.0.0/12
  192.0.0.0/24
  192.0.2.0/24
  192.168.0.0/16

juliangruber

Current Saturn topology, with multi-peer retrieval (TBD) between L1<>L2:

flowchart TB
HttpClient <--> DNS[L1 DNS]
DNS <--> L1'
subgraph L1s[L1s deployed to data centers]
  L1
  L1'
 L1''

juliangruber

$ npm run build

> libp2p@0.38.0 build
> aegir build

[08:47:41] tsc [started]
src/circuit/auto-relay.ts:54:80 - error TS2345: Argument of type '(evt: CustomEvent<Connection>) => void' is not assignable to parameter of type 'EventHandler<CustomEvent<Connection>> | null'.
  Type '(evt: CustomEvent<Connection>) => void' is not assignable to type 'EventCallback<CustomEvent<Connection>>'.
    Types of parameters 'evt' and 'evt' are incompatible.
      Type 'CustomEvent<import("/Users/julian/dev/libp2p/js-libp2p/node_modules/@libp2p/interface-connection-manager/node_modules/@libp2p/interface-connection/dist/src/index").Connection>' is not assignable to type 'CustomEvent<import("/Users/julian/dev/libp2p/js-libp2p/node_modules/@libp2p/interface-connection/dist/src/index").Connection>'.

juliangruber

L1<>L2 architecture

Conventions

• Naming
  • L1: L1 machine
  • l1: Individual l1 worker process
• The overview graph lists all involved components. In future graphs irrelevant components are omitted for readability, which doesn't mean they have been removed

Problems to solve

juliangruber

const dgram = require('dgram')
const glossy = require('glossy')

const syslogProducer = new glossy.Produce()
const socket = dgram.createSocket('udp4')

const sendToPapertrail = (chunk, { severity }) => {
  const syslog = syslogProducer.produce({
    facility: 'user',
    severity,

juliangruber

const fs = require('fs')
const hyperdrive = require('hyperdrive')
const src = `/tmp/${Math.random()}/`
console.log(src)
const storage = require('dat-storage')(src)

const archive = hyperdrive(storage, { latest: true, indexing: true })
archive.writeFile('/dat.json', 'hi', err => {
  if (err) throw err

juliangruber

const fs = require('fs')
const src = `/tmp/${Math.random()}`

Dat(src, (err, dat) => {
  if (err) throw err
  dat.archive.writeFile('/dat.json', 'hi', err => {
    if (err) throw err

    console.log(fs.readFileSync(`${src}/dat.json`)) // => empty
    dat.archive.readFile('/dat.json', (err, data) => {

juliangruber

$ node bench-01-pull-stream.js
pull3*100000: 1995.798ms
pull_compose*100000: 1968.901ms
pull_chain*100000: 1871.930ms

$ node bench-02-async-stream.js
async3*100000: 3295.666ms
async3*100000: 3145.636ms
async3*100000: 2943.988ms

juliangruber

// https://github.com/juliangruber/async-stream

function heap (name, Stream) {
  var start = Date.now()
  var heap = process.memoryUsage().heapUsed
  var a = [], N = 100000
  for(var i = 0; i < N; i++)
    a.push(Stream())
  console.log(name, (process.memoryUsage().heapUsed - heap)/N, (Date.now()-start))
  a = null

juliangruber

const writeToRemote = new PassThrough()

reconnect(async con => {
  await sendState(con)
  writeToRemote.pipe(con)
}).on('disconnect', () => {
  writeToRemote.unpipe()
})

theRestOfYourProgram.pipe(writeToRemote)