I'd like to announce a preliminary version of a "Friday Labs" project I've been working on for a little while.
It's called gocharm, and it is a framework for writing Juju charms in Go. I like Go, and that's one reason for me to do this, but I am also hoping that gocharm contributes more than just a new language to the Juju ecosystem. Here are some areas where I think it has something new to offer:
- Charm maintenance
- Modularity
- Persistent state
- Dependencies
- Charm-provided services
I'll go into my thoughts on these subjects below.
To take it for a quick test spin (assumes a working juju environment and a valid JUJU_REPOSITORY environment variable):
# install gocharm
go get -d github.com/juju/gocharm/cmd/gocharm
godeps -u $GOPATH/src/github.com/juju/gocharm/dependencies.tsv
go install github.com/juju/gocharm/...
# build a charm
gocharm github.com/juju/gocharm/example-charms/helloworld
# deploy it
juju deploy local:trusty/helloworld
It is common practice to write a charm as a single piece of code that is called by all the hooks. When a new relation is added to a charm, the relevant symbolic links must be created in the hooks directory, and relation information added to metadata.yaml. Although relatively trivial to do, this has a nuisance factor and, more importantly, there is significant potential for the hooks and the metadata to get out of sync with the code, and for bugs to result.
In gocharm, relation metadata in metadata.yaml, the config.yaml file and the whole hooks directory are automatically generated from the code when the charm is built. This means that there's much less chance of them getting out of sync, and it also opens the door to...
I have written one or two minor charms before, and I have studied the code for a few more. One thing that concerned me about the code that I saw and that I wrote was its lack of modularity. Every charm that provides an http relation, for example, implements all the required logic for that relation. While that's not a great deal in most cases, when this is repeated for a bunch of relations, the code becomes hard to follow, because the logic for handling particular relations is spread out. A config-changed hook, for example, needs to handle configuration options relating to all the various relations.
Ideally, if some charm provides a relation, I don't want to write code that knows all the details of that relation. I want to import a package that already knows about that relation and provides me with a few relevant methods to make it easy to use.
The way gocharm addresses that is by allowing you to register several functions for the same hook. This means that several independent pieces of code can register a config-changed hook for example - the code for all them will be invoked (in sequence) when the service configuration changes.
Since relation and config metadata are automatically generated, this means that a reusable package can create relations and configuration options and register hooks to be executed for all of them. It doesn't matter that other packages may be registering functions to be called in response to the same hooks - each package can do what it needs to.
Ideally we would have relation implementations for all the relation interfaces out there. I have made a tiny start towards that by implementing some relations in charmbits. If you import the httprelation package for example, and register a provider, the http-relation-joined and config-changed hooks will be registered, the appropriate relation, config metadata and hook files created.
For a hint at where this approach could lead, see this example "hello world" charm, which defines a web service and implements a charm that serves it, including configurable http port, https port and https certificates, all in under 30 lines of code. It's easy to compose that functionality too. It takes only a few more lines to add a configuration option.
Sometimes it is useful for a charm to remember things between hook invocations. Using gocharm, when you pass a pointer to RegisterContext all the state pointed to will be saved persistently. Nothing else is required. In practice, this makes maintaining persistent state as easy as using an instance variable.
Sometimes you don't want to deal with dependencies. When the functionality your charm requires is all available in the charm itself, why not just embed everything in the charm?
With cross-compiled Go, this becomes straightforward. By default, the Go binary that runs the hooks is compiled when the charm is created, and included in the charm. This approach does have its down sides too though - sometimes you don't know what architecture you're going to deploy onto, for example; and having only the binary can make the charm harder to debug, because every code change will require the binary to be pushed up to the environment. With this in mind, the gocharm command has a -source flag. In this mode, the binary is not included with the charm. Instead, all the source code is "vendored" into the charm including all dependencies. At charm install time, the Go compiler is installed and the charm binary built. It's also possible to configure it so that the binary is built every time a hook runs.
There is an issue with charm size - Go binaries are quite big. I intend to compress the executable when it's built, and uncompress at charm install and upgrade time. In the meantime, the goupx command can be useful for compressing the binaries before deploying the charm.
The Go charm code can not only be used to run hooks. Gocharm also supports long-running services, such as web servers, written using the charm code itself. This makes it quite convenient to write single-purpose charms that provide a given service when that service is written in Go.
Currently the service runs directly from within the hook binary in the charm directory, but this is potentially problematic with regard to charm upgrades. In the future, the binary will be copied to another location and replaced while the service is stopped.
I'd like to make a really convincing testing story for Go charms. For charms where both the hook logic and the charm service is written in Go, It should be possible to test almost all of the logic of the charm without deploying it, giving early reassurance that the charm will work. I have made some steps in that direction but there is a lot more to be done here.
When a service run by a charm needs to be upgraded, it is customary to restart the service. The integrated service functionality offered by gocharm makes it potentially possible to automatically provide zero-downtime upgrades, where a new server is brought up before the old one is torn down, gracefully switching from one to the other.
Currently there are implementations of the http, mongodb and elasticsearch relation interfaces. It would be great to have reference implementations for as many relation interfaces as possible.
I hope that this will be useful. For those that feel that Go is an inappropriate language for writing charms, I don't think there's much that's being done with gocharm that can't be done in other languages too. Perhaps it might be worth taking some of the ideas here (I'm thinking in particular of the modularity aspect) and implementing them in another language, such as Python.
Gocharm is still in its early stages. Contributions welcome!