A list of metadata objects describing Nulecule apps/components easily accessible by users (developers, ops).
You are reading an architecture overview for an Index implementation I proposed in two pull requests:
And description of what has been proposed (and partially done) before regarding Index and what should be picked up again.
Please go and read those PRs now and come back then:)
Nor Nulecule neither Atomic App requires central server for application distribution. This give great freedom to users and developers, but makes hard to find applications and components scattered accross various registries and Github repositories and appearing under weird complicated names.
To make apps and components based on Nulecule specification easily discoverable and usable, we want to provide users with a way how to list, search, filter, get info... about them. The Nulecule Index will let users take all these actions.
In addition if CentOS infrastructure will be used and plugged in (as described below), we'll be able to also provide a building and testing capabilities for the apps and components submitted to the Index.
The links to POC implementation can be found in the above paragraph. The demo fo this is captured in the following video.
The Index architecture is extremely simple. It relies on an image containing the generated index being accessible. That means there doesn't have to a single index but anybody can introduce his own and users can choose or even combine multiple of them.
The way how I currently build the index file is by atomicapp index generate http://github.com/projectatomic/nulecule-library. This allows anybody to create a respository containing subdirectories containing Nulecules to build their own index. Also the Index is completely unrelated to a way how the Nulecule metadata images are built.
For Nulecule Library, we currently use Docker Hub Automated Builds, but we can switch to any other infrastructure for building in the future. The only problem I see is that we either have to make some assumptions on image naming (which is the way I chose for now) or we need to specify the image name somewhere. Let me make a list:
- Assumption based - subdirectory name == image name
- Exact name specifies
- Add a file (
index_entry.yaml?) which carries the image name (adds work for developers) - Add the expected image name to Nulecule spec (is nice if there is a special tooling like
atomicapp-builderused, might be dangerous otherwise)
Problem of this solution is that the Index is very centralized - if you want to be part of it, you need to add your work to someone else's repository. That might not be acceptable for some. I'd like to try to solve it with remote_entriey.yaml file.
This file should contain links - similarly to what was proposed here https://github.com/kbsingh/cccp-index
As already stated above, we use Docker Hub Automated Builds to build our apps at the moment. That's ok, but it would be much nicer to be able to add testing and other bits to the build workflow. This might be solved by CentOS infrastructure.
- Nulecule Library
- A repository containing Nulecule files and provider manifests used as a source for building apps
- Scheduler
- Checking the library 1. Polling - basically a cron job checking changes in the library 2. Web hooks - no polling, scheduler would be notified on every push to library
- Submiting builds of changed apps
- atomicapp-builder - builds apps and components (and potentially also the images runned in orchestration, see below)
- Test runners - apps and components need to be tested (see below)
- Registry
This tool lives in atomicapp-builder repository and it's goal is rebuilding Nulecule applications and securing all the dependencies are available in a required version (and building them in case they are not).
This is achieved by consuming Index metadata and matching them to the dependencies of the application. In case any dependency is missing, atomicapp-builder should be able to find a link to the sources in the Index and use those to build the missing dependency.
Another added value of atomicapp-builder is a proposal @bkabrda provided, which didn't make it to the Nulecule spec (yet). Following image describes the structure and relationships of various Nulecule/Atomic App elements
As you can see, Nulecule image contains only a metadata for an application. The actual running software is represented by the Binary Image. With current spec, we are not able to tie the metadata and binary images together. In other words, we simply expect binary images to be available at the point of deployment.
With the addition, @bkabrda propsosed, atomicapp-builder (after adding this feature) would be able to check and build not only the metadata images, but also binary images. That would basically mean a single command build of the whole application.
We currently have some tests for Atomic App itself, but very little for the applications. Given we are going to use CentOS infrastructure we can quite easily add bits for running test after each successful build of a Nulecule application.
The question is where the pointers to the tests should live - in Nulecule file or in the index. I think both are viable, although having them in Nulecule file would allow users to easily test even during the development phase.
Registry is simply a target for the build results - currently Docker images. That means it's a Docker registry where the infrastructure is able to push images. Images have to named based on the Index metadata so that the apps and components are reachable.