belisarius222

+$  note
  $%
  ::  %ames
      [%sift ships=(list ship)]
      [%spew veb=(list verb)]
  ::  %behn
      [%wait date=@da]
      [%rest date=@da] 
  ::  %clay
      ::  TODO: %warp, %info, %park, %merg

belisarius222

Ames Push-Based Reconnect

As it stands, Ames re-sends packets to an offline peer forever, every two minutes. For a ship with, say, 2400 offline subscribers to five chat channels, this adds up to a hundred packets being sent per second to ships that aren't there. A lot of those packets will also go through a relay before being hurled into the void, forcing the relay to do essentially meaningless work.

This is a fair amount of wasted CPU, bandwidth, and disk usage (every retry is a new Arvo event on the sender, which must be written to disk). To make it worse, the retry rate trades off against reconnect latency. If my ship comes back online after a few minutes of downtime, it will take on average one minute -- half the retry interval -- before I hear retried packets from my peers. Lowering the retry interval to lessen the reconnect time would increase the bunk packet rate, which increases systemic load on the network.

A couple years ago, ~wicdev-wisryt proposed switching Ames to use push-based recon

belisarius222

Remote Scry Protocol Proposal

Overview

Despite Urbit's "scry" namespace being global (every request path contains the host ship), there is no way to query other ships. This proposal adds a second Urbit-to-Urbit network protocol that implements remote scrying. This will allow for ships to field read requests without incurring disk writes, and since the namespace is immutable, caching responses will be simple and worthwhile.

To "scry" in Urbit means to query the Urbit namespace. Conceptually, if a query resolves, it can produce either a piece of marked data (meaning tagged with a system-recognized type) or an empty result indicating that this path will never contain data. Not all requests resolve; some "block", which represents a refusal or inability to answer the question (such as a local query for a file at a future date). The namespace is immutable in the sense that all nonblocking results to the same query must be identical. Whether a query resolves is not specified; a query could succeed, the

belisarius222

A scry packet is either a request packet, sent from requester to host, or
a response packet, sent from host to requester.
Some request packets and all response packets are signed.
Paths are encoded as null-terminated ASCII text, except for the host ship,
which is encoded the same way as in Ames,
so as not to slow down relaying with string parsing.

Scry packets are not responsible for peer discovery and
therefore do not include an "origin" field like Ames packets.

belisarius222

[8.909.898.678.765.747.192.222.496.419.433 [1.547.502.848 122.585.133.865.456.070.768.289.138] [17.826.304 [727.297.206 0 0] 3.636.038.703 [1.471.624.404 [46.901.714 [98.508.064.685.512.714.131.431.889.268.921.467.136 [377.881.856 [38.863.872 0 0] 208.369.818.243.882.581.233.561.692.627.075.007.488 0 460.032 [163.132.979.714.802.956.572.376.767.866.019.709.696 0 0] 207.507.362.372.271.488.339.077.218.035.923.222.784 0 0] 273.039.027 0 0] 271.646.908.320.357.328.305.654.769.512.108.916.992 [2.437.088.256 [63.236.944.151.159.559.491.195.372.633.782.419.712 [2.122.444.651 [12.190.720 0 0] 0] 6.554.368 [2.865.038.758 0 0] 2.642.114.485 0 0] 2.505.935.919 [322.028.769.875.806.775.348.013.658.829.832.061.184 [2.381.819.040 0 2.359.899.348 0 0] 2.099.883.190 0 0] 2.278.978.205 0 0] 9.372.066 [3.178.299.826 [1.742.733.824 [818.291.878 0 0] 851.970.230 0 0] 0] 61.283 [467.890.845 [210.535.763.981.882.411.786.672.100.501.168.259.328 [8.061.184 0 0] 0] 200.055.289.992.422.841.599.103.354.385.164.337.408 0 0] 933.036.847

belisarius222

It's useful to have standard names for types, but it's also useful to have standard names for variables. In Hoon, we strive for unambigious and fast pronounceability. If you read a Hoon program to me over the phone, I should be able to dictate it perfectly without wondering how something is spelled. While this goal is probably not completely achievable, we can get a lot closer than we are right now. I propose a scheme for getting close to this.

In many programming languages, single-letter variable names are common. In Hoon, these are called "ultra-lapidary style". These usually satisfy Hoon's pronunciation requirement, but they do not map naturally onto meaningful concepts except when the number of variables is very low. For a comparator function, taking in 'a' and 'b' is fine. But for arguments whose meaning is more semantic than positional, which is most of the time, a name should have some mnemonic connection to the value it signifies.

The natural way to do this is a full word, which should be c

belisarius222

For those of you at home, I wanted to include some instructions for S3 support.

S3 support lets you store credentials for your S3 object storage buckets on your Urbit ship; once you have done so, you will get additional functionality for uploading your own media within Chat and Groups (for avatars).

It’s for power users at the moment — it requires some technical setup, but we hope to surface some UI for it as we go.

You will need to get an S3 bucket set up. If you’re using AWS, it will have to support signature v2, not v4. We found DigitalOcean worked well. The bucket has to be publicly readable; allow CORS from * origins, and allow * headers. Your provider should have details on setting access permissions and CORS.

Once you have gotten your bucket setup, poke the s3-store on your ship with your details. You can do this in Dojo or in web Dojo, it will work however.

belisarius222

|%
::  $hark: general-purpose notification
::
::    Arguably .link should be a $beam, not a path, but $path is a bit
::    more general and can easily encode a beam.  The idea is that each
::    notification should be able to point to the part of the system
::    that generated it, so that if the user clicks the link, they will
::    be taken to the relevant part of an application.
::
::    The link could refer to just the application that generated the

belisarius222

Ford Fusion

Overview and Rationale

Ford Fusion was an overhaul of Urbit's over-the-air upgrade process and a rewrite of its build system. The new update system corrects a few long-standing bugs with the previous one, and the new build system is simpler, smaller (by around 5,000 lines), and easier to manage.

Since deployment of Ford Fusion to the livenet in late June, over-the-air updates (OTAs) have been much smoother. Before Ford Fusion, it was common for an OTA to take several hours, use too much memory, and leave ships in inconsistent states. After Ford Fusion, multiple OTAs have been pushed out, including kernelspace changes, and most users didn't even notice.

Urbit has always been able to update itself OTA, but this process has often been rocky. Updating an operating system kernel on-the-fly is a difficult problem in general, like performing heart replacement surgery on yourself while running a marathon. Code that allows Linux to update its kernel in this way became a startup called Ksplice, won

belisarius222

|%
::  +mung: 31-bit non-zero murmur3-32 based merkle hash
::
++  mung
  ~/  %mung
  =/  syd  0xcafe.babe
  |=  n=*
  ^-  @
  ?^  n
    $(n (con $(n -.n) (lsh 0 31 $(n +.n))))