justmoon/ILP Kit 3.0 Release Plan.md Secret

## ILP Kit 3.0 Release Plan.md

      
    Raw
  

              ILP Kit 3.0 Release Plan.md
            
          
    ILP Kit 3.0 Release Plan

Major Changes

(All changes are described below in more detail.)

Make ilp-connector stateless and extract standalone mode
Add support for asymmetric trustlines
Deprecate Five Bells Ledger
Switch to binary ILQP (#390) ✔
Improvements to routing
Quote by liquidity curve
Support for webhooks (via custom receivers) (#179)

Release Goals

ILP Kit 2.0, although short-lived, has been the most stable ILP Kit release to date. We've done crazy demos with it and we love it.
However, we still have a number of issues:

Stability issues with websockets
High overall latency
Single-process architecture (no way to scale)
State is fragmented (ledger tables, ILP Kit tables, plugin store tables)

In order to make progress at addressing these issues, we aim for the following goals for this release:

No more clunky internal websockets
All state is managed by ILP Kit (to be further consolidated in the future)
UI feels more responsive

Details

Stateless connector library

We've compromised on the connector's statelessness to accomodate stateful plugins.
Currently, the connector provides a store, which uses Sequelize to connect to a database. Unfortunately, the store only provides key-value semantics, no transactions. As a result, this mechanism is not safe to use in a multi-process environment. (Multiple ILP Kits using the same database.)
However, it has turned out that plugins build two main primitives on top of the store: Balance and TransferLog
Before

Today, the ILP Kit instantiates plugins like this:
await connector.addPlugin(hostInfo.ledgerName, {
  currency: options.currencyCode, // connectors have this option to contradict the ledgerInfo's currencyCode, but we don't use that.
  plugin: 'ilp-plugin-virtual',
  store: true,
  options
})
After

We propose to remove the store feature from ILP Connector and instead have stateful plugins accept a backend parameter.
await connector.addPlugin(hostInfo.ledgerName, {
  currency: options.currencyCode, // connectors have this option to contradict the ledgerInfo's currencyCode, but we don't use that.
  plugin: 'ilp-plugin-virtual',
  options: {
    backend: createPluginBackend()
  }
})
PluginBackend

The backend parameter is an object of the abstract type PluginBackend:
interface PluginBackend {
  getBalance(name: string, opts: BalanceOptions): Balance
  getTransferLog(): TransferLog
}

interface Balance {
  setMaximum(n: string): void
  getMaximum(): string
  connect(): Promise<void>
  get(): string
  add(n: string): Promise<void>
  sub(n: string): Promise<void>
}

interface BalanceOptions {
  maximum: string | null, // Maximum allowed balance (or null if unlimited)
  minimum: string | null, // Minimum allowed balance (or null if unlimited)
  key: string // used as a prefix for entries in the key/value store (defaults to '')
}

interface TransferLog {
  get(transferId: string): Promise<Transfer> // Asynchronously retrieve a transfer from the persistent store
  getCached(transferId: string): Transfer // Synchronously retrieve transfer from cache
  getFulfillment(transferId: string): Promise<string> // Retrieve a transfer's fulfillment from the persistent store
  dropCached(transferId: string) // Remove transfer from cache
  fulfill(transferId: string, fulfillment: string): Promise<void> // Set transfer state to fulfilled
  cancel(transferId: string): Promise<void> // Set transfer state to canceled
}

interface Transfer {
  // see Ledger Plugin Interface
}
We can imagine several implementations of this interface:

MockPluginBackend - In-memory backend for unit tests or development
SequelizePluginBackend - Simple single-process backend for ILP Kit
KafkaPluginBackend - More sophisticated and scalable future backend for multi-process environments

Backwards compatibility

In order to keep plugins compatible with old versions of ILP Kit, we can implement a createBackendFromStore method in ilp-plugin-shared, which creates a SequelizePluginBackend from a store. When instantiating a plugin with a _store, but without a backend, the plugin would generate a backend using createBackendFromStore.
In order to keep ILP Kit 3.0 compatible with ILP Kit 2.1 databases, we can make it so ILP Kit 3.0 also uses ilp-plugin-shared to generate a SequelizeBackend which stores data in exactly the same way (same key names etc.) that it is being done currently. Future versions of ILP Kit may use more sophisticated backends and we will use migrations to move the data over.
Refactoring standalone support

ILP Connector is currently half written as a library and half as a server. We propose a refactor such that ilp-connector is a pure JavaScript library that needs to be included from JavaScript. The ability to run a connector standalone should be factored out into a scripts/server.js script inside of the ilp-connector repo.
Support for asymmetric trust lines

In order to support asymmetric trust lines, we create two new ledger plugins: Plugin ILP Kit Server and Plugin ILP Kit Client.

When instantiated, Plugin ILP Kit Server creates two objects:

A plugin instance to give to the connector, which looks to the connector more or less like it's just talking to a Plugin Bells.
A plugin instance that ILP Kit can use internally, which looks like a Plugin Bells instance with admin privileges (can listen on all accounts etc.)

Plugin ILP Kit Server also creates a public interface (HTTP + Websocket?) which ILP Kit exposes and which can be consumed by Plugin ILP Kit Client.
Deprecate Five Bells Ledger

We have to decide if we still want to support Five Bells Ledger in the 3.0 release.
If yes, we could add a flag in the Users table for ILP Kit to determine if a user is a five-bells-ledger or asymmetric trust line user.
If no, we need to write a migration that converts all ledger users to asymmetric trustline users.
Binary ILQP

This is the only change proposed in this document that is visible by peers. If any of the other changes are projected to take a long time to implement, arguably we should release binary ILQP first.
Routing changes

@michiel: Can you provide a summary here?
Quoting by liquidity curve

ILQP supports getting a liquidity curve instead of a quote for a specific amount. We should implement and support this functionality in ILP Kit.
The end goal is that ILP Kit will immediately request a quote when the destination identifier has been entered, even before the amount has been chosen. That way, the user can type different amounts and the quote will update in real-time, without a round-trip to the server.
The quote should be periodically refreshed (subject to the quote expiry).
Support for webhooks

Webhooks are the most common way for integrating external shopping cart systems and applications with payment systems. In order to make integrating ILP as easy as possible for merchants, we should support some form of webhooks.
The current PR aims to implement webhooks via SPSP custom receivers. (E.g. user "bob" creates a receiver bob+myshop@red.ilpdemo.org which is linked to a specific webhook endpoint.)