JDK Enhancement Proposal: FFI

ffi-jep.md

Title: Foreign Function Interface Author: Charles Oliver Nutter Organization: Red Hat Owner: Charles Oliver Nutter Created: 2014/01/28 Type: Feature State: Draft Exposure: Open Component: --/-- Scope: JDK JSR: TBD Discussion: https://groups.google.com/forum/#!forum/jvm-ffi Start: 2014/Q2 Effort: M Duration: L Template: 1.0

Summary

In order to form a more perfect union of JDK with underlying native operating systems and hardware, this proposal adds a built-in Foreign Function Interface to OpenJDK. This interface provides the ability to bind native libraries (function endpoints in shared libs and OS kernel) and directly manage blocks of native memory.

Goals

• Provide a foreign function interface at the Java level, similar to JNA (Java Native Access) or JNR (Java Native Runtime).
• Optimize calls to native functions and management of native memory at the JVM level (optional).
• Support a future JSR for a standard Java FFI (aiming for Java 9).

Non-Goals

• Mechanisms for explicitly defining the managed and unmanaged structure of standard Java objects
• Other CPU-related improvements to Java object management, like cache line control, GPU/vectorization, etc
• Direct modifications or improvements to JNI itself. These may come during the JSR process, however, and JVM enhancements for FFI may require internal changes.

Success Metrics

Success will be defined by having an FFI API at JDK level sufficient for implementing large-scale native-backed features like NIO, advanced filesystem metadata, process management, and the like. This FFI API should ideally become preferred over writing JNI backends for each platform when a new native-level feature is required.

Motivation

The enhancement of the JDK (and of Java in general) by adding native-level features (new filesystem structures, different IO channels, new crypto backends) has been stymied by the need to write JNI code to back those features for every new platform. The expertise necessary to write proper JNI is orthogonal to the expertise needed to write good Java code or call native libraries, but JNI has been the only path toward bridging those two worlds. This API will provide a tool enablin any JDK developer with understanding of Java and of a specific native library to bind that library as a JDK API in furtherance of JDK and Java features.

My hope is that this JEP will make it easier to add new OS, hardware, and native library-level features to Java and the JDK as well as enabling a standard Java FFI for use in the wider Java world.

Description

For years, Java and JDK developers have had only one trusted way to add native-level features to Java applications: the Java Native Interface. This interface governs the boundary between the managed environment of the JVM and the unmanaged environment of native code, providing explicit protocols for data marshaling, object lifecycle management, upcalls back into Java, and native JVM tooling and management APIs. JNI does provide a strict, unmistakable demarcation between managed and native code; it also makes that boundary incredibly painful to cross, even for the most trivial cases.

We believe the following aspects of JNI are most painful to developers:

• Requiring developers to write C code means they have to have expertise of a world completely different from Java. In many cases, the native binding required is a trivial function call with no side effects and little to no data-marshaling complications, but in every case JNI requires developers to be C programmers.
• Using JNI requires expertise usually not found in typical C and Java developers, since the developer must have at least some understanding of how the JVM manages memory and code (object lifecycles, GC complications, Java class layout, JVM lifecycle). JNI makes it possible to do the right thing, but much easier to do the wrong thing.
• Beyond simply writing C code, developers must be able to build that code for each platform they wish to support, or provide appropriate tooling for end-users to do the same. This is required despite the fact that the JDK itself is provided across dozens of platforms and platform-specific experts have already done significant work to make the JVM run and access native code across those platforms. This requirement makes JNI more detrimental to write-once, run-anywhere than an FFI API, since the latter will be much more likely to work across many platforms with only a single binary, and fixing incompatibilities will usually still only require a new build and release of that one binary.
• Even after crafting perfect JNI code and providing builds for every platform, the performance of JNI-based libraries is usually very poor compared to the same library bound into a native application. This stems from the inescapable rigidity of JNI's manaaged/unmanaged demarcation and a complete inability of the JVM's own optimizations to see through JNI calls. In many cases, JNI downcalls to trivial functions could be done directly from jitted Java code, since they do not require memory gymnastics and do not interfere with JVM internals. JNI's guarantees make it impossible to make native calls as lightweight as they could be.
• Finally, JNI acts as an opaque boundary for security. The JDK only knows about permission to load a specific library; it does not know what calls the functions in that library might use or whether the code in that library could compromise the stability or security of the JVM. This invites mistakes from JNI developers who are not expert C programmers or who simply don't understand the JVM-level aspects of security.

The detriments of JNI would be addressed by providing a built-in FFI API at the JDK level. FFI would be easier for Java developers to write, not require as much knowledge of JVM internals, favor correct implementation or fast-fail over latent bugs, and eliminate the requirement for per-platform build expertise.

The JDK FFI API will provide the following to JDK developers:

1. A metadata system to describe native library calls (call protocol, argument list structure, argument types, return type) and native memory structure (size, layout, typing, lifecycle).
2. Mechanisms for discovering and loading native libraries. These capabilities may be provided by current System.loadLibrary or may include additional enhancements for locating platform or version-specific binaries appropriate to the host system.
3. Mechanisms for binding, based on metadata, a given library/function coordinate to a Java endpoint, likely via a user-defined interface backed by plumbing to make the native downcall.
4. Mechanisms for binding, based on metadata, a specific memory structure (layout, endianness, logical types) to a Java endpoint, either via a user-defined interface or a user-defined class, in both cases backed by plumbing to manage a real block of native memory.
5. Appropriate support code for marshaling Java data types to native data types and vice-versa. This will in some cases require the creation of FFI-specific types to support bit widths and numeric signs that Java can't represent.

Optionally, this JEP will build additional support for the above features via:

1. JVM-level awareness of FFI downcalls. This could include: JIT optimization of those calls, JVM/GC-level awareness of native memory, protection against illegal native memory accesses (SEGV faults), and mechanisms to opt out of JNI safeguards known to be unnecessary in specific cases (safepoint boundaries, blocking call guarantees, object lifecycle management, etc).
2. Tooling at either build time or run time for reflectively gathering function and memory metadata from native libraries. This would aid the initial binding of a library by providing a way to generate that binding at the Java level rather than requiring hand-implementation and tweaking for each platform. Prior work here includes the ffi-gen library for (J)Ruby, which uses clang (LLVM C compiler) metadata APIs for generaing Ruby FFI code.
3. The JVM security subsystem should understand specific library/fuction coordinates. It should be possible to set up security policies that allow binding only specific functions in specific libraries, rather than just the coarse-grained library-level permissions that exist today.

The level of abstraction for the JDK FFI is TBD; at minimum, it must understand how to:

1. Load a library
2. Invoke a function at some offset in that library
3. Pass bit-appropriate arguments to that library (width, endianness) and receive bit-appropriate values back; typing is not relevant at this level beyond describing bit width and structure
4. Minimally manage native memory: allocation, deallocation, access, and passing to/receiving from native calls

Therefore, we have some open discussion on how far to go with this API.

Alternatives

The need for a Java FFI has spawned several libraries. Among these, Java Native Access is the most pervasively used, and Java Native Runtime is perhaps the most comprehensive and advanced. JNR is likely to form the basis for this JEP, since it implements various levels of abstraction, provides function and memory metadata, abstracts away library and function binding, and has been in heavy use by at least the JRuby project (and its users) for at least the last five years.

Testing

In order to test this API, we will need to add representative native endpoints (as C code) to provide sufficient coverage of all call protocols, type marshalling, and memory management features. Many of these capabilities may exist in current JDK and Java test suites. The JNR library also has an existing suite of tests that could (will) be incorporated into JDK.

Risks and Assumptions

JNR's original author has recently gone on hiatus from open-source development. However, he believes he will be able to assist us in this effort.

JNR's license may or may not be compatible with OpenJDK, but it is negotiable.

Dependences

No known dependencies on other JEPs or JSRs. Not known to block any other JEPs or JSRs.

Impact

• Compatibility: FFI will increase the work required to guarantee cross-platform compatibility of OpenJDK. However, OpenJDK's supported platforms already support FFI via JNR.
• Security: FFI opens up the potential for user-level code to access untrusted native functions and read or write normally-inaccessble areas of native memory. This capability is no more extreme than that provided by existing APIs (Unsafe et al). It will be possible to explicitly define security controls based on library/function coordinates; this capability represents an improvement over coarsed-graind System.loadLibrary controls.
• Performance/scalability: Performance of native bindings should improve, if those bindings are written to use FFI. It is an open question whether existing JNI-based features should be rewritten to use FFI, and in what timeframe that should happen.
• Portability: There are obvious portability concerns, but no more than exist in current builds of OpenJDK. JDK-based code that uses FFI will still need testing across supported platforms to ensure functionality.
• Documentation: Ideally this API will become the preferred way to bind native code and memory, and so developer documentation should provide everything needed for JDK developers to use this API instead of JNI.
• TCK: A future Java FFI JSR will obviously need additions to the TCK, but ideally this will be little more than the testing provided by the JDK-level FFI.

Comments and replies from Ioannis Tsakpinis of LWJGL:

Just read the JEP, I think it's great and I don't have anything to add at
this point. The only thing I found a bit odd was mention of (new?) native
memory management. Which NIO's direct buffers basically is. Correct me if
I'm wrong, but my guess is that you're talking about something lower-level
(an official Unsafe?), something that could have been used together with the
FFI to implement everything in NIO. And not the other way around, like my
suggestion above to use direct buffers as a pointer abstraction in the FFI.
Btw (you've probably seen this already) Oracle is already open to
discussions about Unsafe:
https://blogs.oracle.com/dave/entry/do_you_use_sun_misc

Will have a look at that blog post.

WRT native memory, I agree that this isn't much more than direct ByteBuffer in implementation. What JNA and JNR provide is a Java-level abstraction atop that so you don't have to manually marshal values to/from memory. So the JEP (and eventual JSR) would essentially make it possible for user code to define a "super-ByteBuffer" that is backed by native memory but knows about struct layout and value marshaling (and perhaps lifecycle).

Oh, a thought I just had. Should the JEP include something about callback
functions? Right now that's a bit of a pain for us, callbacks from C to Java
code is the only functionality we have to code manually in LWJGL. I know
there's dynamic native code generation magic that can be used to handle this
(JNA & JNR do it too iirc), but I wonder if there's a better way with a
built-in FFI in the JVM. Threading issues exist here too, callbacks can
either be synchronous (from a JVM thread) or asynchronous (from a native
thread). Anyway, this isn't too important for us (not many callbacks to
support), but it'd be nice to get rid of JNI completely.

I don't know if that's the level of detail we need to have in the JEP, but it's a good thought. I also don't call out how call protocols should work, how specific types should marshal. Callbacks may be a big enough item to mention explicitly.

@headius Interesting proposal!

Few questions:

a) How is the "selection" of native method calls implemented from a language perspective?
E. g. is it possible to build a type provider on the things provided in this JEP which reads the header file and provides typed method signatures based on that information instead of having to describe the method signature manually?

@CLibrary("math.c")
object cmath

val x: Double = cmath.sin(123.4) // Type of method has been computed as cmath(x: Double)Double

b) Any plan to keep the specification/implementation/TCK as open as possible, so that alternative runtimes can benefit from it?

Thanks!