IPFS pub/sub proposal using object blocking

gistfile1.txt

# Proposed poor-man's IPFS/IPNS pubsub

A rough proposal to implement the widely desired [pub/sub feature on ipfs](https://github.com/ipfs/notes/issues/64)
without requiring  additional capabilities or message-passing behavior from the network or the clients. It could be
implemented as an IPFS application right now and in fact I've got sample JS that uses the js-http-api to do so
on the publish side. Hopefully building out the subscribe side this week.

I've just started diving into IPFS so this could be a super naïve approach.

The proposal's core concepts:
  * **subscription is modeled as querying a non-existent block** whose future multihash is known, but has not been added to the DAG.
    Once this block exists, its payload will use IPNS to provide a forward linked list to one or more future blocks in the
    publication stream.
  * **publication is modeled as publishing the block** with that desired multihash, and providing a payload that
    links forward to one or more future blocks in the publication stream.

 The proposal's key properties:
   * It requires the IPNS router so it's not the fastest ride in town.
   * It requires that clients patiently wait for `wantlist` blocks that can't currently found on the network
     - I believe this is the current behavior of `go-ipfs`
   * It is loosely inspired by [mutable value chains](https://www.google.com/url?q=https%3A%2F%2Fjoearms.github.io%2F2015%2F06%2F19%2FMutable-Value_Chains.html&sa=D&sntz=1&usg=AFQjCNE_xPLM8fTlJ1Nx79HiyD1K-v2YxQ)
   * It requires no additional capabilities or message-passing behavior from any actor in the network.

 The proposal's minimum necessary flow (or just jump down to the example):
   * PUBLISHER creates a SUBSCRIPTION LINK json object
     - The link contains a reference to an `/ipns/` address under PUBLISHER's control.
     - The reference will contain the future payload for the SUBSCRIPTION LINK.
       - henceforth `<SubscriptionLink.PayloadLocation>`
     - The link contains a sequence ID unique in this stream (e.g. "1")
       - In reality we will want to hash this with the private key to avoid spoofing
   * PUBLISHER calculates the SUBSCRIPTION LINK multihash, but does not add it to the DAG
     - Uses `ipfs add --only-hash`
     - Henceforth `<SubscriptionLink.MultiHash>`
   * PUBLISHER distributes `<SubscriptionLink.MultiHash>` widely
   * SUBSCRIBER adds `<SubscriptionLink.MultiHash>` to his `wantlist`.
   * SUBSCRIBER waits for the object to exist in the DAG.
   * PUBLISHER writes content and adds it to the DAG.
     - It has ID `<Content.MultiHash>`
   * PUBLISHER creates a tree/folder at `<SubscriptionLink.PayloadLocation>` under his IPNS root.
     - Henceforth `PayloadTree`
   * PUBLISHER writes `<Content.Multihash>` into `PayloadTree/data`
   * PUBLISHER calculates the hash for a NEW SUBSCRIPTION LINK using the same process as before
     - This link now has sequence ID 2
     - It should probably also contain an ipfs (not ipns) backlink to `PayloadTree`
     - Henceforth `<NewSubscriptionLink.MultiHash>`
   * PUBLISHER writes `<NewSubscriptionLink.MultiHash` into `PayloadTree/next`
   * PUBLISHER publishes the new version of his IPNS root.
     - The first subscription link's payload is now ready to consume, but nobody knows how to find it.
   * PUBLISHER adds and pins the first SUBSCRIPTION LINK json object to the DAG.
   * *The subscription link (which is really more of a notification) starts percolating through the network as it satisfies
     subscribers wantlists.*
   * SUBSCRIBER, who has patiently waited `<SubscriptionLink.MultiHash>` in his `wantlist` for ages, receives and re-pins the
     subscription link
     - Repinning helps other subscribers.
   * SUBSCRIBER dereferences `<SubscriptionLink.PayloadLocation>`, finding both the content and the next subscription link
     within `PayloadTree/data` and `PayloadTree/next` respectively.
   * SUBSCRIBER acquires the content using `ipfs get`
   * SUBSCRIBER adds `<NewSubscriptionLink.Multihash>` to his `wantlist`, and waits.

Example of basic necessary flow:

## Step 1: Alice distributes a subscription link for her future blog
| Publisher (Alice) |
--- |
| *"Think I'll start a blog. Don't know what I'll write yet, but let me give the world a subscription link."* |

```json
// Alice creates a subscription link: this JSON content.
// She saves it to ~/first-publication.json
{
  "payloadLocation":"/ipns/<Alice.PeerId>/ipfs-pubsub/blog/1",
  "publishedBy":["<Alice.PeerId>"],
  "sequenceId":1 // or Sign(1, PrivateKey)
}
```
```bash
# Alice discovers the hash of this subscription link.
# She does not add it to the DAG.
alice~> ipfs add --only-hash ~/first-publication.json
added <FirstPublication.MultiHash> ~/first-publication.json # Not true! We didn't add it, we just discovered the hash.
```

| Publisher (Alice) | Subscriber (Bob) |
--- | --- |
| *"Hey subscribe to my blog at `<FirstPublication.MultiHash>`!"*| |
| | *"Sure, Alice!"* |
| | `bob~> ipfs object get <FirstPublication.MultiHash>` |
| | *Prepares to wait the long haul. Distributes this object on his wantlist, as does every other future subscriber to Alice.* |

## Step 2: Alice writes her first blog content, publishes it.
| Publisher (Alice) |
--- |
| *"Time for the first post!"*|
```bash
# Alice writes and adds her first post to the DAG.
# Nobody but her knows about the published content.
alice~> echo "I ate shit for breakfast" > post1.txt && ipfs add post1.txt
added <Post1.MultiHash> post1.txt

# Alice prepares the payload for the first publication
# She chose her blog's multihash as the payload.
alice~> PAYLOAD_DIR=$IPNS_ROOT/<Alice.PeerId>/ipfs-pubsub/blog/1
alice~> mkdir -p $PAYLOAD_DIR
alice~> echo "<Post1.MultiHash>" > $PAYLOAD_DIR/data
```
```json
// Before publishing the first publication, she needs to prepare the second subscription link
// so that her followers will know when she posts again!
// She creates the following json and saves it to ~/second-publication.json
{
  "payloadLocation":"/ipns/<Alice.PeerId>/ipfs-pubsub/blog/2",
  "publishedBy":["<Alice.PeerId>"],
  "sequenceId":2 // or Sign(2, PrivateKey)
}
```
```bash
# Alice calculates the new subscription link's hash, and posts that hash as the "next" reference
# in the first subscription's payload
alice~> ipfs add --only-hash ~/second-publication.json
added <SecondPublication.MultiHash> ~/second-publication
alice~> echo "<SecondPublication.MultiHash>" > $PAYLOAD_DIR/next

# The message payload for the first publication is ready to go.
# Alice updates her IPNS in preparation of broadcasting her blog post
alice~> ipfs add --recursive $IPNS_ROOT
added <NewIpnsRoot.MultiHash>
alice~> ipfs name publish <NewIpnsRoot.MultiHash>

# Now she finally publishes to her subscribers, satisfying the wantlist of her many blog followers and notifying
# them that content has updated
alice~> ipfs add --pin ~/first-publication.json
```

## Step 3: Bob grabs the new content, waits for more
| Subscriber (Bob) |
--- |
| *"Oh look, we finally received <FirstPublication.MultiHash>. I guess Alice finally posted!"*|
| *"Well let's look at the content."* |
```bash
# Now that Bob has FirstPublication, he can go get the payload. He does this by dereferencing
# the payloadLocation of publication event he just received, and grabbing the "data" link from that tree.
# You'll recall that the "data" link contained the ID to the actual blog content, and the "next" link
# contained the future ID for the next publication event.

bob~> ipfs cat <FirstPublication.payloadLocation>/data | ipfs cat
I ate shit for breakfast

# "Gross. I wonder what she'll say next. Let's subscribe to the next one."
bob~> ipfs get <FirstPublication.payloadLication>/next | ipfs get
# This will block until Alice writes her next blog post
```