Skip to content

Instantly share code, notes, and snippets.

@tanishiking
Last active April 22, 2024 07:22
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tanishiking/f8edf2057136fb48ac387f6ad5775aa0 to your computer and use it in GitHub Desktop.
Save tanishiking/f8edf2057136fb48ac387f6ad5775aa0 to your computer and use it in GitHub Desktop.

Why WASI?

To communicate with the systems provided by the operating system, you need to call system calls. However, WebAssembly is designed as a complete sandbox, and Wasm modules cannot directly access OS system calls.

To call system calls from Wasm, you need to import functions that have access to system calls from the host environment.

#[link(wasm_import_module = "syscall")]
extern "C" {
    fn write(pointer: i32, length: i32);
}

#[no_mangle]
pub fn exec() {
    let msg = "Hello, World".as_bytes()
    unsafe {
        let p = mst.as_ptr() as i32;
        let l = msg.len() as i32;
        write(p, l);
    }
}
$ cargo build --release --target wasm32-unknown-unknown

$ wasm2wat .../hello.wat
(module
  (type (;0;) (func (param i32 i32)))
  (type (;1;) (func))
  (import "syscall" "write" (func $syscall_write (type 0)))
  (func $exec (type 1)
    i32.const ... ;; pointer
    i32.const ... ;; length
    call $syscall_write
  )
  ;; ...
)
function write(pointer, length) {
  console.log(textDecoder.decode(memory?.subarray(pointer, pointer + length)));
}
const imports = {
  syscall: { write }
};
const { instance } = await WebAssembly.instantiateStreaming(..., imports);
const memory = new Uint8Array(instance?.exports?.memory?.buffer);

For example, starting with version 1.11, Go supports building for WebAssembly. However, Go's Wasm modules define their own interface for interacting with the system (as of Go 1.21, it supports WASI https://go.dev/blog/wasi).

If you look at a Wasm module built from Go1.11 using GOARCH=wasm GOOS=js, you can see many import definitions.

(import "gojs" "syscall/js.finalizeRef" (func (;8;) (type 1)))

To use this Wasm module built from Go, you need to provide these imports from the host environment using an import object. (Go compiler generates the JS library to fill in the interfaces though).

The program executor needs to know how it requires to be executed, which compromises Wasm's portability.

WASI stands for WebAssembly System Interface, is a specification for the interface that Wasm modules use to access the system, with the goal of standardizing it. By standardizing the system interface, Wasm modules can be executed on any WASI-supporting runtime without needing to know which language was used to build the module.

For example, in Go 1.21, WASI preview1 is supported, allowing Wasm modules to be executed on WASI-supporting runtimes (such as wasmtime or Node.js) without needing to provide an implementation of the custom-defined interface previously required for executing Wasm modules.

What is WASI?

WASI (WebAssembly System Interface) is being standardized by the WebAssembly community group to enable running WebAssembly outside of browsers in a standarized way. WASI aims to define safe and portable WebAssembly APIs through the WebAssembly mechanism. Currently, the development of what is called Preview 1 has been completed, and the development of Preview 2 is underway.

For example, a Wasm module built for WASI in Go 1.21 includes imports like:

(import "wasi_snapshot_preview1" "fd_write" (func (;7;) (type 5)))
(import "wasi_snapshot_preview1" "random_get" (func (;8;) (type 3)))

These are the system interfaces defined by WASI Preview 1. By providing implementations of these interfaces, you can execute Wasm modules that use WASI.

The list of interfaces provided by WASI Preview 1 can be found here: https://github.com/WebAssembly/WASI/blob/main/legacy/preview1/docs.md

For example, fd_write is defined as fd_write(fd: fd, iovs: ciovec_array) -> Result<size, errno>, and in Wasm, it is defined as a function that takes four i32 arguments and returns one i32 value: (func (param i32 i32 i32 i32) (result i32)).

According to the following references,

  • 1st: file descriptor
  • 2nd: Starting address (on Wasm Linear Memory) of the iovs
  • 3rd: Size of the memory pointed to by iov_base
  • 4th: Return pointer (on Wasm Linear Memory)

references

We need to exchange data using the linear memory.

Kotlin/Wasm example

For example, Kotlin/Wasm defines print function based on WASI like this:

/**
 * Write to a file descriptor. Note: This is similar to `writev` in POSIX.
 */
@WasmImport("wasi_snapshot_preview1", "fd_write")
private external fun wasiRawFdWrite(descriptor: Int, scatterPtr: Int, scatterSize: Int, errorPtr: Int): Int

internal fun wasiPrintImpl(
    allocator: MemoryAllocator,
    data: ByteArray?,
    newLine: Boolean,
    useErrorStream: Boolean
) {
    val dataSize = data?.size ?: 0
    val memorySize = dataSize + (if (newLine) 1 else 0)
    if (memorySize == 0) return

    val ptr = allocator.allocate(memorySize)
    if (data != null) {
        var currentPtr = ptr
        for (el in data) {
            currentPtr.storeByte(el)
            currentPtr += 1
        }
    }
    if (newLine) {
        (ptr + dataSize).storeByte(0x0A)
    }

    val scatterPtr = allocator.allocate(8)
    (scatterPtr + 0).storeInt(ptr.address.toInt())
    (scatterPtr + 4).storeInt(memorySize)

    val rp0 = allocator.allocate(4)

    val ret =
        wasiRawFdWrite(
            descriptor = if (useErrorStream) STDERR else STDOUT,
            scatterPtr = scatterPtr.address.toInt(),
            scatterSize = 1,
            errorPtr = rp0.address.toInt()
        )

    if (ret != 0) {
        throw WasiError(WasiErrorCode.entries[ret])
    }
}

private fun printImpl(message: String?, useErrorStream: Boolean, newLine: Boolean) {
    withScopedMemoryAllocator { allocator ->
        wasiPrintImpl(
            allocator = allocator,
            data = message?.encodeToByteArray(),
            newLine = newLine,
            useErrorStream = useErrorStream,
        )
    }
}

https://github.com/JetBrains/kotlin/blob/4ee876a52ae0d9e31b2dfc45295a1ce576099a96/libraries/stdlib/wasm/wasi/src/kotlin/io.kt#L23-L73

  • It allocates the required memory using the allocator.allocate on Wasm Linear Memory and stores the data in the allocated memory.
  • It then allocates an 8-byte memory region to store the iovec structure
    • The first 4 bytes of the iovec structure are set to the starting address of the allocated memory containing the data
    • and the next 4 bytes are set to the size of the data.
  • It allocates another 4-byte memory region to store the error pointer
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment