warpfork

This is a working document to snapshot some thoughts about the evolution of Warpforge and its APIs.

Observations (that lead to some desires for changes)

• We should seriously consider moving the Catalogs to an even yet plainer format.
  • The Catalog format as it stands today is a pain to edit manually, due to its inclusion of hashes. (This has been remarked upon by essentially every user to date.)
  • We also derive very little value from the use of hashes in the middle of the Catalog data, because in practice, we find we're continuously embedding the Catalog data in other version control systems, which already contain their own checksums. While there is elegance to having our own hashing scheme provide our own bespoke incrementally verifiable trees, there is little strong demonstration of that providing direct value (and very many demonstrations of it creating problematic friction).
  • The Catalog format seems like it could become more generally usable (and thus more appealing for others to interact

warpfork

Text Catalogs: an RFC

Abstract

This document proposes "Text Catalogs" -- "tcat" for short -- a convention for file naming and content organization for document stores. Text Catalogs are specifically aimed to be legible to humans, friendly to version control,

warpfork

#!/bin/bash
set -euo pipefail
#set -x


## HOW TO HOLD IT:
##
##   Give the program you want to zapp as the first argument.
##
##   For the couple of different usage patterns:

warpfork

WarpVer Versioning

A software versioning scheme.

It's not semver ™️.

The Rules

warpfork

Static Site Generator Roundup 2021

Criteria & Considerations

• does it consume markdown?
• is it fast?
  • (a "hello world" site should take milliseconds or less to build. Definitely not full seconds; certainly not multiple full seconds.)
• is it reasonably shipped?

warpfork

IPLD Strings

For an IPLD library to have "Complete" support for the IPLD Data Model, Strings MUST support the full range of 8-bit bytes. Strings SHOULD use UTF-8, and library designers may encourage this however they see fit. In particular, NFC-normalized form is encouraged. However, this MUST NOT result in the inability of a library to handle non-UTF-8 byte sequences where Strings are handled, nor should libraries apply normalizations to data they read. (In other words: using normalized forms when creating new data is encouraged,

warpfork

Strings in IPLD

"Strings" are a familiar concept in how programs handling digital data. However, for all its familiarity, it is also subtle. Solid systems require clear specifications. In this document, we will describe Strings in IPLD, where they appear, and what specifications we make about their domain.

Prerequisites

warpfork

Data Model: Order of Maps

Maps in IPLD are typically order-preserving: if you put data into them in some order, you can expect that the iterators will return that information in the same order; and transformation functions will generally operate on data without reordering it.

However, there are also several exceptions to this rule, so it's important to be aware of what to expect in the different scenarios.

warpfork

This is a radical rethink of the linking situation for the go-ipld libraries.

• It gets almost all logic detached from the Link/LinkPrototype implementations. This will probably detangle implementations greatly.
• There's a LinkSystem type that handles the composition of different codecs, hashers, etc.
• It's now possible to do your own codec and hasher registries by creating your own LinkSystem instance. Previously this was impossible.
• We can attach many more helper methods to LinkSystem. This should improve end-user usability. Previously there was no clear place to put them (without burdening the Link/LinkBuilder implementations with lots of boilerplate, which was nonelegant in many ways).

There are a couple of groups of interfaces below which would come from different places:

1. Link and LinkPrototype would be implemented by something like CIDs. (A project probably won't actually have a variety of implementations of these, one should be enough; but they're still interfaces for library agility

warpfork

Filesystems & IPLD

Introduction

IPLD is a system for creating decentralized systems based on content-addressable data primitives. One of the things people frequently want to do when building applications is handle files, and describe filesystems. As a result, we've compiled some thoughts and recommendations about how to describe filesystems in IPLD, and also produced some specifications that we suggest systems use and build upon.