Cene packages

gistfile1.txt

When someone shares or installs a Cene library, it consists of several parts, which together (in uncompressed form) determine its hash:

- A UUID indicating what spec of the Cene language is intended.
- A source file tree.
- A string containing a single Cene expression to evaluate to get a function that takes a reference to the source file tree, monadically performs macroexpansion side effects in its own sandbox (where every name is based on the library's overall hash), and results in a sink value. The side effects may include looking up other libraries by hash, which allows multiple libraries to import common names and constructors for communicating with each other.

The Cene language UUID accounts for built-in definitions, parser syntaxes, file tree archive formats, and macroexpansion side effect models that may vary between Cene versions. A UUID is preferable to a signature, a public key, or a a DNS host for this purpose; Cene implementations will inevitably implement divergent support for the same UUIDs, driven by goals of compatibility and realities of software flaws, so if the identifier called out a single vendor, it would eventually be dishonest. Meanwhile, by using UUIDs instead of version numbers or clever nicknames, we avoid letting the distribution format make an implicit value judgment between Cene's forks, branches, upgrades, and standardization efforts. Nevertheless, we allow this to be an arbitrary string in case certain spinoffs of Cene pursue a purely human-writable distribution format without UUIDs or hashes.

In the future, it's possible some libraries will export and import values of types other than sink. However, the most likely approach will be for the sink type to have as a special case an encapsulated constructor of well-typed programs.

With this library model in mind, it should be simple to make a command line tool to read and write a file containing a simple subset of Cene code:

  (def-imported-sink-by-hash foo /import-hash-sha-3 ABCDEF0123456789ABCDEF0123456789)
  (def-imported-sink-by-hash bar /import-hash-sha-3 CDEF0123456789ABCDEF0123456789AB)
  (def-imported-sink-by-hash baz /import-hash-sha-3 EF0123456789ABCDEF0123456789ABCD)

This file can be copied into the source directory before running a build. During the build, it can be interpreted as Cene code like any other .cene file in the project.

The command line tool may be extended to provide support for fetching a library of a known hash, displaying the library version information corresponding to a hash, fetching a library by a more friendly name, etc. It's possible these extensions may even be implemented by Cene libraries.

Into the future, a common distribution format for Cene libraries will probably have a few features: It will bundle several libraries together, often including every dependency or at least a hint as to where to get every dependency. It will recommend that certain libraries be installed into the command line tool for easier management of the other libraries in the bundle. In some cases it may bundle or hint at dependencies useful for developing new versions of the other libraries in the bundle. The bundle will likely be compressed as a whole.

The Era module system uses code signing to allow definitions to be shared between certain modules without being exposed to others that don't know the implementation details. In this Cene library system, every dependency is based on a hash, essentially proving that the importer knows the exporter's implementation details, so this does not contradict the Era approach. It may someday be important to flesh this out to include the flexibility of the Era approach, particularly if we represent dependencies on an open network or dependencies between complex libraries that undergo frequent security releases independently of each other. In those conditions, a network code loader or a command line tool that merely obtains the most up-to-date library versions automatically is inviting those two dependencies to use the codebase as their battleground.


(The following API design is sort of outdated now. The concept of how to fetch dependencies is now mostly a matter for the command line tool to deal with. Nevertheless, it could still be useful for formalizing ways to extend the command line tool using Cene code, and if we're going to design what the command line tool can do, we might as well design a Cene interface for it at the same time.)


## Revisions to the existing CLI API.

(defn output-path-blob output-path possibly-encapsulated-blob ...)

(defn encapsulated-blob-utf-8 possibly-encapsulated-string ...)

Remove `output-path-blob-utf-8`



(defn dependency-locator-archive-url url-string ...)

(defn dependency-locator-default first second ...)

(defn dependency-locator-give-up - ...)

(defn dependency-verifier-sha-3 hash-string ...)

(defn dependency-verifier-signed-by-rsa public-key-string ...)

(defn dependency-verifier-comes-from-archive-url url-string ...)

(defn fuse-dependency-verifiers-by-intersection - ...)

(defn dependency-verifier-empty - ...)


(defn package-import dependency-locator dependency-verifier scope ...)
Given a dependency locator, a dependency verifier, and a scope, monadically obtains the dependency's code and runs it in that scope with its own input directory.

(defn picky-packaged-cene-quine mode cexpr ...)
Given the current mode and a compiled expression, returns an encapsulated blob containing an archive of Cene code. TODO: Explain.

Over the past couple of days, I've written some notes that refer to these, and I'm making those public, so I'm making these public as well.