Here are some rough notes on things observed in the codebase - not by the Espresso authors so may all be totally wrong.
Espresso is mostly written in Java but has some C components as well. These serve two purposes:
- The espresso engine can be compiled with native-image down to a shared library so it no longer needs hotspot and runs AOT. This is called “libespresso”
- Native code components from OpenJDK that use internal HotSpot APIs can be loaded. This allows a high degree of code re-use and means that unlike most JVM implementations, this one doesn’t need to implement the JDK class library, not even tricky parts in java.base or internal modules - in fact the classes you’d find in the java.base module in a regular OpenJDK should work as long as it’s of the right versions.
OpenJDK native code components include libjava, libjimage etc. These can be loaded as genuinely native components and also (it appears) via Sulong, which on GraalVM EE would give you sandboxing of these components too.
A part of how Espresso works is by carefully loading dynamic libraries in exactly the right order, to ensure that later components link against the earlier components. On HotSpot there is a problem that the JVM components are already loaded, and thus they must be de-conflicted. Unfortunately only Linux exposes sufficiently powerful APIs from its platform dynamic linker to allow this (although maybe something can be hacked up on Windows/macOS). That’s why Espresso-on-HotSpot is limited to Linux, and by implication, to hack on Espresso you will need to do it on Linux. Or at minimum use a container.
This is the name of the Espresso engine when compiled with native image. It is not directly usable by existing software that wants to dynamically load a JVM though, because it exports a small Espresso-internal interface along with the usual C methods to manage isolates.
(for extreme Englishmen like me who don’t drink coffee at all, a mokapot is a type of coffee brewing kettle).
This is the name of what becomes “libjvm.so” in the Espresso world. On HotSpot libjvm exposes two APIs, one is the standard JNI functions to create a new VM, initialise and control it. The other is an internal API used to implement the java.base module, these functions start with JVM_. Mokapot provides both.
The JVM_ functions aren’t normal JNI calls. Sometimes calls to them come from a JNI binding layer called libjava, which comes from OpenJDK and is used as-is on Espresso. And sometimes calls to it are emitted directly from JIT compiled code.
Mokapot doesn’t have much real logic in it. There’s some stuff to lookup tables with threads via a similar attach mechanism to that used in JNI but it’s ultimately just a forwarding layer. A few JVM_ calls are implemented “natively”, by binding to the operating system code. Others are sent upwards, back into Java code via a fairly complicated route that depends on whether you’re using the AOT compiled libespresso (it has to traverse the SubstrateVM C binding layer, potentially JNI, TruffleNFI and other convoluted details), or running on HotSpot, or - probably - using Sulong.
A significant source of complexity in this part of Espresso is the numerous combinations of interfaces between Java and C. There are different mechanisms for SubstrateVM, hotspot, JNI, Sulong/Truffle FFI interop etc. Everything related to the transitions between native components and Java code is rather confusing as a consequence. Mokapot can be used in multiple modes, these are identified by coffee related names:
- "Ristretto" - loaded from another host JVM, e.g. by creating a polyglot context.
- "Latte" - loaded from native code using the normal JNI API, like the java launcher, or a C++ program that wants to embed a JVM.
- "Americano" - Latte composed with Ristretto. This is used (I think) when host Java code creates an Espresso JVM, and then native components may also need to interact with it, but it's not really clear.
There's also "Nespresso" which appears to be related to JNI rather than being related to how Espresso is loaded, but that isn't really documented.
The action starts in VM.java. This is where the JVM internal API is implemented and is also called into from Truffle nodes.
VM holds a reference to a separate data holder class called EspressoContext (all VM mutable state like currently loaded
classes, VM properties etc), and the context in turn holds a reference to a Meta which is used to access guest world (loaded
into Espresso) objects from the host world. Both of these can be used from partially evaluated code.
The VM object is not actually what's run. There are a series of annotation processors that generate more glue code that ends up creating a VMImpl
There's also InterpreterToVM which implements some of the core bytecode logic. It's used for things that aren't related to
frame or local variable management like locking, reflection, allocations, and casting.
Whilst OpenJDK code is mostly left alone and interacts with Espresso via the JVM_* private internal API, there are a few places where this is insufficient, mostly for performance reasons. Espresso has the ability to 'hot patch' the class libraries via a substitutions mechanism. This is similar but different to the SubstrateVM substitution mechanism. As of writing this mechanism is used to hot-patch a few basic methods like the java.lang.Object methods, Unsafe, reflection and a few other things.
Each Java method is a single BytecodeNode. The node holds a reference to a BytecodeStream which just makes it easier to decode bytecodes from the underlying binary. The core of the node is a big switch statement. Because of the @ExplodeLoop annotation when this node is compiled the loop is fully unrolled (the loop body is just copied over and over for each iteration) and optimised, deleting all the parts of each loop body that are for the 'wrong' bytecode. Thus this ends up gluing together the method bodies for each opcode into a single native code output method.
There's also the @BytecodeInterpreterSwitch annotation. The goal of this is to help Graal compile the BytecodeNode class when it's not partially evaluating - i.e. converting this file into the equivalent of the hand-coded assembly interpreter in HotSpot. Currently Espresso doesn't use it as it doesn't have quite the right behaviour.
As the interpreter runs some bytecodes are replaced with a fake QUICK opcode. This means the logic shouldn't be done by hard-coded logic in the switch - it should be delegated to another Truffle node. The indirection means that bytecode has a node and thus can benefit from the Truffle engine's support for swapping specialised nodes in and out at runime. Quite a lot of bytecodes are handled by nodes rather than simple method calls in the switch - invokes, checkcasts, and inlined. Additionally array loads and stores are handled by this mechanism, this is because they may be operating on either an Espresso array or a "foreign" array which needs to be routed via the Truffle polyglot interop mechanism. For example if a Java program running in Espresso is passed a pointer to a Ruby array, and then reads from it in a way that needs to be high performance, the bytecode interpreter will translate array load/store bytecodes into 'quickened' nodes that end up specialised for the type of array being accessed.
Although Espresso claims to be a full JVM, it lacks any garbage collector. Instead it relies on the host JVM GC, either HotSpot's or the one linked in to the compiled native image.
One consequence of this is you can't customise the way GC works by changing Espresso. Another is that any time guest data must be represented in Espresso, it's done using two arrays (Object[] and byte[]/long[]). The latter is used for every local variable that's not a pointer. The former is for pointers. This split layout is the only way that a set of Java variables can be meta-expressed in the Java language, as there's no low level interface for expressing that an array of longs can contain a mix of pointers and non-pointers. In HotSpot this is all handled by "oop maps" but there's no equivalent of JVMCI for garbage collectors, so for now this is the best Espresso can do. That's not a very efficient representation, so we may see improvements here in future.
There's a class called EspressoFrame but it's just a set of static helper methods. Critically this is not related to a Truffle VirtualFrame. Every method in Espresso uses a Truffle stack frame that has the two arrays in local variables as described above. Thus the Java frame isn't mapped directly to a Truffle VirtualFrame.
Every guest object is a StaticObject. The layout is not quite as memory efficient as in HotSpot. Each guest object is made up of several pointers in the real GC-visible heap object: a pointer to the ObjectKlass (i.e. the Class inside the guest world), pointers to Object[] and byte[] for the primitive fields (same approach as with frames), and a pointer to the lock.
StaticObject has quite a lot of code in it for polyglot interop purposes, and there are static utilities to wrap/transport primitive arrays into the guest world.
The core of any JVM is a model of the (partially) loaded program. This is in the vaguely named com.oracle.truffle.espresso.impl package. There are objects here for every element of the guest program - klasses (klass = guest class), methods, fields, arrays, class loaders and so on. This is the code that allows a program to reflect itself, be linked on the fly, be debugged, and for the program structure to be changed on the fly via hotswap.
Symbol is basically a String with a few extra features: they're always interned so they can be compared quickly using identity equals (==), they have a type variable (that isn't used for anything) but which is useful to make the code easier to read, the contents are partially evaluatable and a few other benefits. They're used to hold strings that define parts of the guest program like class names, method/field names, etc.
There are a few more parts that are ordinary Java implementations of straightforward algorithms: classfile parsing, liveness analysis etc. The latter is used to null out object references once the guest program no longer needs them, this is to avoid keeping guest objects hanging around even after they become garbage, simply because of refs in the Object[] used for local variables that aren't going to be used anymore.
And that's it.