- Provide Fabric operators with a mechanism to build and launch chaincode outside of Docker
- Begin the process of decoupling and extracting chaincode build processing from the peer
Our current chaincode deployment model looks a lot like a PaaS; chaincode developers write code in the language of their choice and push it to the peer for execution. The peer, on demand, transforms the code into an executable package, launches the chaincode in an execution environment, and routes requests to the running chaincode.
Unfortunately, we have implemented this by hard coding the logic for packaging, building, and running chaincode inside Docker containers. In addition to making it difficult to experiment with new chaincode types, the implementation requires operators to grant the peer access to a Docker daemon. Since Docker daemon access enables the peer to perform a number of privileged actions, it's strongly discouraged in security conscious environments. This binding also prevents operators from using platforms like Kubernetes to host chaincode.
Let's begin by looking at some of the technology in Cloud Foundry that supports staging and execution of applications.
A buildpack is a software package that implements the process of transforming user provided source assets to executable code. The contract is simple, but flexible enough to support pure binaries, scripting languages, and complicated build processes that use autoconf and make.
The buildpack contract is comprised of three scripts: detect, compile, and release. detect is used to determine if a buildpack is can be used with source assets provided by a user, compile is used to transform the assets into something that can be executed, and release is used to provide the start command to the platform.
Cloud Foundry uses lifecycle implementations to manage applications. Like a buildpack, a lifecycle consists of three executables: builder, launcher, and healthcheck. The builder is responsible for preparing an application for execution, the launcher is responsible for starting an application, and the healthcheck is called regularly to monitor the launched application for liveness.
In Cloud Foundry, two implementations of lifecycle exist: docker and buildpack. The buildpack lifecycle uses buildpack processing to build and launch an application while the docker lifecycle is used extract metadata from an image and launch it in inside a garden container.
Our proposal is to adapt patterns used by Cloud Foundry for building and running applications to the chaincode model of the peer. These patterns will introduce seams that enable operators to build and launch chaincode with the technology of their choice and, eventually, will enable us to remove hard code chaincode platforms from the peer.
Combining aspects of the buildpack and lifecycle models in Cloud Foundry, the proposed peer chaincode contract consists of three process based entry points: detect, build, and launch.
The purpose of detect is to determine if the external builder should be used with the chaincode. The executable will be called by the peer with one argument: the path to a source directory containing the contents of the chaincode package. If detect exits with a return code of 0, the external builder will be used to build the chaincode. Any other exit code will indicate the peer's internal builder should be used.
Any modifications to the contents of the source directory may be discarded. Messages written to stderr will be written to the peer's log.
When detect has returned with a 0 return code, the build executable will be called to transform the chaincode package into something that can be launched by launch.
The executable will be called with two arguments: the first is the path to a source directory containing the contents of the chaincode package, and the second is the path to a build directory where the build can cache information or hold build outputs. A successful build operation must result in a 0 return code from build and any other return code will be treated as a failure.
When build is successful, the contents of the build directory will be saved on the peer's file system; any modifications to the source may be discarded. Messages written to stderr will be written to the peer's log.
A typical implementation of build might be to generate a build context from the input source, perform an unprivileged build using a tool like Google's kaniko, and then populate the docker image reference to the build directory.
The purpose of launch is to locate or start chaincode on behalf of the peer. The launch executable is called with three arguments: the first is the path to a directory containing chaincode package source, the second is the path to a directory containing the contents of the build tree, and the third is the path to a directory containing chaincode.json and any artifacts referenced by that file. This file will be used by the peer to provide metadata like the peer chaincode server address and assets such as generated TLS keys and certificates required for chaincode registration. It is up to the launcher to provide this information to the launched chaincode as appropriate.
The launch operation is considered successful when it exits with a return code of 0; any other return code is considered a failure. Prior to exiting, the launcher may write a JSON encoded message to the peer. This message is similar to chaincode.java in structure but is used to inform the peer about the chaincode invocation contract. In particular, it can specify the address of the running chaincode and provide TLS keys and credentials necessary to connect to chaincode. When this information is made available, the peer will consider registration complete and invoke chaincode at the target address using the provided credentials. If this information is not provided, the legacy chaincode model is assumed and the peer will wait for the chaincode instance to register.
A failure to parse the JSON message provided from launch to the peer will be treated as a launch failure. Fields that are unknown to the peer are ignored. Messages written to stderr will be written to the peer's log.
A typical implementation of launch might launch a container in a Kubernetes bare pod with the appropriate start command, environment and key material necessary to register with the peer's chaincode server. Once the pod object has been created, the launcher would return 0 with an empty JSON message of {} to stdout. A message that includes the ID of the new pod would be written to stderr and captured in the peer's log. The peer would then wait for chaincode registration to complete.
An alternative implementation of launch might retrieve the appropriate Kubernetes service definition and credentials to invoke the desired chaincode via some service, populate a message with the target information, write the message to stdout, and exit with a return code of 0. The peer would then parse the message and invoke the chaincode at the provided address with the provided credentials.
The peer configuration will be augmented to include a reference to a directory on the peer's local file system that contains the detect, build, and launch executables. These executables
When the feature is not configured, the peer will fallback to the existing implementation that is embedded within the peer.
The proposed launch mechanism does not provide any mechanism to include externally launched chaincode in the peer logs. If log aggregation is required, the peer and chaincode should be configured to direct logs to a common destination that can merge the events.
From the Configuration parameter it seems you assume there will be a single set of executables? Doesn't this somewhat get in the way of adding new chaincode types? I would have expected there would be something like a chaincode-type registry which would map chaincode types to their handler?