c-from-java.md

Here's an example of using C code from Java, via GraalVM.

I want to run the xxHash algorithm for some reason (I don't know anything about this algorithm, I just know I found a single-file C implementation of it). There's a Java port, but let's say there isn't for the sake of argument, so I want to run the original C version rather than doing the port myself.

I clone it from https://github.com/Cyan4973/xxHash.

I can then compile the native version.

$ make -C xxHash

Now unfortunately I do need to patch the C code in just two places to run well for us. It uses a memcpy on a Java object, which isn't going to work with the native function, so we substitute it for a version which does work on managed objects.

diff --git a/xxhash.c b/xxhash.c
index da06ea7..daa2a6d 100644
--- a/xxhash.c
+++ b/xxhash.c
@@ -116,6 +116,7 @@ static void* XXH_memcpy(void* dest, const void* src, size_t size) { return memcp
 #define XXH_STATIC_LINKING_ONLY
 #include "xxhash.h"
 
+void *truffle_managed_memcpy(void *destination, const void *source, size_t count);
 
 /* *************************************
 *  Compiler Specific Options
@@ -174,7 +175,7 @@ static U32 XXH_read32(const void* ptr) { return ((const unalign*)ptr)->u32; }
 static U32 XXH_read32(const void* memPtr)
 {
     U32 val;
-    memcpy(&val, memPtr, sizeof(val));
+    truffle_managed_memcpy(&val, memPtr, sizeof(val));
     return val;
 }
 
@@ -617,7 +618,7 @@ static U64 XXH_read64(const void* ptr) { return ((const unalign64*)ptr)->u64; }
 static U64 XXH_read64(const void* memPtr)
 {
     U64 val;
-    memcpy(&val, memPtr, sizeof(val));
+    truffle_managed_memcpy(&val, memPtr, sizeof(val));
     return val;
 }

Now we can compile with clang, emitting bitcode rather than machine code. We also want to run the mem2reg optimisation, to keep more things in managed registers rather than native memory.

$ clang -c -emit-llvm xxhash/xxhash.c -o xxhash/xxhash.bc
$ opt xxhash/xxhash.bc -mem2reg -o xxhash/xxhash.bc

We can write a Java program to use this bitcode file.

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import org.graalvm.polyglot.Context;
import org.graalvm.polyglot.Value;
import org.graalvm.polyglot.Source;

public class XXHash {
  
    public static void main(String[] args) {
        // Create a polyglot (GraalVM) context
        
        try (Context context = Context.newBuilder().allowNativeAccess(true).build()) {
            // 'Evaluate' the bitcode file to produce a library
            
            final Value library;

            try {
                library = context.eval(Source.newBuilder("llvm", new File("xxHash/xxhash.bc")).build());
            } catch (IOException e) {
                e.printStackTrace();
                return;
            }
            
            // Read a function from the library

            final Value XXH64 = library.getMember("XXH64");
            
            // Loop through each argument

            for (String arg : args) {
                // Read bytes from each argument file
                
                final byte[] bytes;

                try {
                    bytes = Files.readAllBytes(Paths.get(arg));
                } catch (IOException e) {
                    e.printStackTrace();
                    continue;
                }
                
                // Call the XXH64 function

                final Value result = XXH64.execute(context.asValue(bytes), bytes.length, 0);

                // Print output

                System.out.printf("%x  %s%n", result.asLong(), arg);
            }
        }
    }
  
}

We can compile that as normal, and then run it as a Java program. I'm using GraalVM to run these commands.

$ export PATH=~/Documents/graalvm-ee-1.0.0-rc3/Contents/Home/bin:$PATH
$ javac XXHash.java
$ java XXHash xxHash/xxhash.c xxHash/xxhash.h
ef3b7b633d0ccb14  xxHash/xxhash.c
eef6b55d7f6a374d  xxHash/xxhash.h

Compare that to the native output.

$ xxHash/xxhsum xxHash/xxhash.c xxHash/xxhash.h
ef3b7b633d0ccb14  xxHash/xxhash.c
eef6b55d7f6a374d  xxHash/xxhash.h

Hopefully the problem abuot the managed memcpy will go away.