This is a recap of what is changing in the mirage world in order to switch to the dune build system. It should lead to more pleasant worlflows and paves the way for cross-compilation.
- Stop generating an empty
myocamlbuild.ml - Generate a
dunefile specifying the set of rules needed to build an unikernel. - Generate a
dune-projectfile for the dune language and release compilation mode. - Generate a
dune-workspacefile that sets up compilation contexts with the correct C flags for each target.
- Execute a
dune build @<target>which directly generates the unikernel according to the rules defined in the configure step.
Executing the config.ml file was done by compiling and dynlinking it against
itself: this becomes problematic when the library's source code is modified as
symbols may get their names changed. The goal is to replace this by a 2-stage
build using dune: the config.ml is compiled and then executed to finish the
configuration step.
These packages expose the base layer of xen an freestanding targets, with a custom ocaml runtime and a custom libc. They expose the C and ld flags as pkg-config variables, this necessary to correctly build C stubs and link the unikernel. Dune offers the possibility to gather and set these flags automatically. The dune-workspace file indeed allows to set flags globally and enables a straightforward compilation scheme.
Most projects that have C stubs can simply rely on workspace flags set up by
the Mirage cli. In dune files you just need to have a c_names field to list
your C files and make sure to have c_flags :standard ... to include the
workspace compilation flags.
When the C stubs compilation need to depend on the target (for example
zarith-xen need gmp-xen and custom compilation machinery) it's possible to
fetch the flags using :include:
- xen:
:include %{lib:mirage-xen-ocaml:cflags - freestanding:
:include %{lib:ocaml-freestanding:cflags}
Virtual libraries are needed when a library is compiled with different dependencies and/or have different implementations depending on the selected backend.
mirage-os-shim is the shared interface between mirage-xen, mirage-unix
and mirage-solo5. For now it exposes an OS module with time and scheduling
primitives.
The dune file for mirage-os-shim is
(library
(name mirage_os)
(public_name mirage-os-shim)
(virtual_modules OS)
(libraries lwt))
and implementations can simply refer to it by using
(implements mirage-os-shim).
zarith is a numerical library that relies on GMP. As such there's no way to
transparently cross-compile GMP so gmp-xen and gmp-freestanding packages
have been created. So in order to have a portable zarith to build against we
have to expose zarith as a virtual library and then have zarith-xen,
zarith-unix and zarith-freestanding as implementations.
In ocaml 4.07 Bigarray has mostly been moved to Stdlib.Bigarray and the
bigarray library becomes Stdlib.Bigarray + Unix.ba_map_file. This means
depending on bigarray also grabs an unwanted unix dependency in the closure.
Mirage/OCamlbuild dealt with that using a -dontlink option that just ignores
these libraries. We can't do that (and we don't want to do that) in dune so we
need to track down and eliminate these dependencies to unix in packages that
transitively have them: sexplib, ocplib-endian.
The specific solution for bigarray was to create a bigarray-compat wrapper
that exposes Stdlib.Bigarray after 4.07 but still grabs unix when OCaml is
less than 4.06, in order to keep compatibility for non-mirage projects.
In order to corrrectly build packages with C stubs we need to keep the sources locally. The tool was initially designed by @avsm, and then contributed to and maintained by @NathanReb, @Julow, @TheLortex.
Workflow:
mirage config -t hvt: configure the unikernel and create an opam file with dependencies.duniverse init: read the opam file and compute the list of packages to download in order to build the unikernel.duniverse opam-install: install dune-incompatible packages in the global opam switchduniverse pull: fetch the whole dependency tree in aduniverse/directory so that dune can use them to build the unikernel. The first pull is slow but a cache mechanism has been developped so that subsequent pulls are fastermirage build: build the unikernel for the given target.