Skip to content

Instantly share code, notes, and snippets.

@seanjensengrey
Last active April 2, 2022 04:08
Show Gist options
  • Star 56 You must be signed in to star a gist
  • Fork 10 You must be signed in to fork a gist
  • Save seanjensengrey/f5d73bbdf22cfa1ad463 to your computer and use it in GitHub Desktop.
Save seanjensengrey/f5d73bbdf22cfa1ad463 to your computer and use it in GitHub Desktop.
Calling Rust from Python/PyPy using CFFI (C Foreign Function Interface)

This is a small demo of how to create a library in Rust and call it from Python (both CPython and PyPy) using the CFFI instead of ctypes.

Based on http://harkablog.com/calling-rust-from-c-and-python.html (dead) which used ctypes

CFFI is nice because:

  • Reads C declarations (parses headers)
  • Works in both CPython and PyPy (included with PyPy)
  • Lower call overhead than ctypes

Install Rust

from either of:

I recommend installing Rust via multirust so that you can compile projects that use unstable features (core) as well as testing for Rust 1.0,1.1, etc backwards compatibility.

The per-directory override mechanism is especially nice.

Build library

rustc treble.rs

This will create a libtreble.dylib

Run the Python client

python test.py 
<cffi.api.FFILibrary_./libtreble.dylib object at 0x1089d5490>
math from rust! 30
try:
from cffi import FFI
except ImportError:
print "pip install cffi, included with PyPy"
ffi = FFI()
lib = ffi.dlopen("./libtreble.dylib")
print lib
# <cffi.api.FFILibrary_./libtreble.dylib object at 0x107f440d0>
ffi.cdef('int treble(int);')
print "math from rust!", lib.treble(10)
#![crate_type = "dylib"]
// compile: rustc treble.rs
#[no_mangle]
pub extern fn treble(value: i32) -> i32 {
value * 3
}
@devdave
Copy link

devdave commented Aug 16, 2021

This is a really cool little test you wrote up.

@seanjensengrey
Copy link
Author

@devdave

I haven't touched this in years, but there is a repo over here https://github.com/seanjensengrey/rust-jna-example that expands on this for Java.

This repo shows how to expose a Levenshtein library using three methods

This example handles strings, which depending on the ratio of work to marshaling can solve many problems. After all, nearly every service call already has heavy string based overheads.

@KevinWhalen
Copy link

I like that this is basically "rustc and go" so I can try pulling out a single, bottle-necked function. If I had to combine large Python and large Rust projects, then PyO3 is 100% the way I would go. For porting one function, instead of putting it up as a FaaS/Lambda/micro-service, I really like this use of CFFI.

@seanjensengrey
Copy link
Author

seanjensengrey commented Jan 28, 2022

I think now I would compile the Levenshtein code to Wasm and load it into Python via Wasmtime. @KevinWhalen totally with you on safe inprocess servers. Composes like a web service, fast like a function call.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment