A counterexample to register the callback functions to the external libraries

README.md

A counterexample to register the callback functions to the external libraries

The key point for this gist repo is to indicate:

Don't misuse the pointers of the instances allocated in functions stack since they will be gone after finishing the function calling.

The sample code here is to demonstrate the problem I have when I am developing a Rust audio library. To play sounds, it needs to register callbacks to the underlying OS through audio APIs and then the callbacks will be fired periodically. The callback usually will come with at least 2 parameters:

1. a buffer that will be filled with audio data
2. a pointer indicating what variable calls the callback.

To create Rust APIs that can play audio, I create a struct for audio stream named Stream and store all the stream related settings like sampling rate or channels in that struct. There is a struct method named new that will create a Stream instance that will be returnd. Before the instance is returned, I use the pointer of the instance as the callback's target pointer so that the callback knows what Stream it belongs. When the callback is fired and run, the target pointer is dereferenced to a Stream instance and then use it to call some functions to get the audio data to fill the buffer come with it. The process here looks logically reasonable, but it doesn't work. When the program runs, it causes a segmentation fault.

The root cause of the segmentation fault is that the callback function tries to convert a invalid pointer to a Stream instance. The target pointer comes with the callback is pointed to a memory chunk that is abandoned after the struct method new(stream::new(...)) is called. Suppose we have a code like: let s = Stream::new(...). What let s = Stream::new(...) does is to copy the returned value from stream::new(...) and then abandon the stack memory to call stream::new(...). That's how normal function works in stack(if we don't have any variables in heap). That is, the memory of the created Stream instance that I use its address as the callback's target pointer is marked unused after executing stream::new(...). That's why the program fails.

This is not a Rust-specific problem. It's a common problem for all languages. But if you do the same things as what stream::new(...) does in C++'s constructor, it works. Since the address of this is same as the address of the variable that calls constructor. Notice that the stream::new(...) is not the constructor. In stead, it works like a static class/struct function that creates a class/struct instance and returns it. The key point is that you need to make sure the pointer points to the valid address you want.

Sample code

To simulate the callback from underlying OS API, I create a external library to register and fire callbacks.

• The external library that simulates the callback behavior:
  • ext.h
  • ext.c
• Rust examples:
  • problem.rs: Demonstrate the problem mentioned above
  • solution.rs: Solution for the problem

Simpler examples (with comparison with other languages)

Here is a simpler version of this problem, without exteranl library. In that repo, it also provides the comparison with how it works in other languages. See more details there.

References

• Is it safe to use the “this” pointer in an initialization list?
• Should you use the this pointer in the constructor?
• When is it safe to call this-> in constructor and destructor

ext.c

#include "ext.h"
#include <assert.h> // assert
#include <stdio.h>  // printf
#include <stdlib.h> // callac, free

// Private APIs
// ============================================================================
const unsigned int TYPES = SHORT + 1;
size_t get_type_size(Type type) {
  assert(type < TYPES);
  static size_t sizes[TYPES] = {
    sizeof(float),
    sizeof(short)
  };
  return sizes[type];
}

void print_data(unsigned char* ptr, size_t bytes) {
  printf("0x");
  while (bytes) {
    printf("%02x", ptr[--bytes]);
  }
  printf("\n");
}

void read_data(void* buffer, size_t item_size, size_t items) {
  printf("%s > read data:\n", __FILE__);
  for (size_t i = 0 ; i < items ; ++i) {
    printf("%zu: ", i);
    unsigned char* ptr = buffer + i * item_size;
    print_data(ptr, item_size);
  }
  // Use buffer data to do something here ...
}

typedef struct {
  void* target;
  Callback callback;
} CallbackStruct;

typedef struct {
  Type type;
  CallbackStruct cbs;
} Resource;

static Resource resource;

// Public APIs
// ============================================================================
Status set_type(Type type) {
  if (type >= TYPES) {
    return BAD_TYPE;
  }

  resource.type = type;
  return OK;
}

Status register_callback(void* target, Callback callback) {
  if (!callback) {
    return NO_CALLBACK;
  }
  if (!target) {
    return NO_CALLBACK_TARGET;
  }

  resource.cbs.target = target;
  resource.cbs.callback = callback;
  return OK;
}

Status trigger_callback() {
  if (!resource.cbs.callback) {
    return NO_CALLBACK;
  }
  if (!resource.cbs.target) {
    return NO_CALLBACK_TARGET;
  }

  size_t items = 10;

  size_t item_size = get_type_size(resource.type);
  assert(item_size);

  void* buffer = calloc(items, item_size);
  resource.cbs.callback(resource.cbs.target, resource.type, buffer, item_size, items);
  read_data(buffer, item_size, items);

  free(buffer);
  return OK;
}

ext.h

#ifndef EXT_H
#define EXT_H

#include <stddef.h> // size_t

typedef enum {
  FLOAT,  // float
  SHORT   // short
} Type;

typedef enum {
  OK,
  BAD_TYPE,
  NO_CALLBACK,
  NO_CALLBACK_TARGET,
} Status;

// The buffer size is based on the defined type set via `set_type`.
typedef void (*Callback)(
  void*,  // target
  Type,   // data_type
  void*,  // buffer
  size_t, // item_size
  size_t  // items
);

Status set_type(Type type);
Status register_callback(void* target, Callback callback);
Status trigger_callback();

#endif // EXT_H

makefile

all:
				gcc -shared -fPIC ext.c -o libext.so
				rustc solution.rs -L.
				LD_LIBRARY_PATH=. ./solution
				rustc problem.rs -L.
				LD_LIBRARY_PATH=. ./problem

clean:
				rm problem
				rm solution
				rm libext.so

problem.rs

// A module containing all the converted types from external library.
mod sys {
    use std::os::raw::c_void;

    pub type Type = i32;
    pub const FLOAT: Type = 0;
    pub const SHORT: Type = 1;

    pub type Status = i32;
    pub const OK: Status = 0;
    pub const BAD_TYPE: Status = 1;
    pub const NO_CALLBACK: Status = 2;
    pub const NO_CALLBACK_TARGET: Status = 3;

    pub type Callback = extern fn(
        *mut c_void,    // target
        Type,           // data_type
        *mut c_void,    // buffer
        usize,          // item_size
        usize,          // items
    );

    #[link(name = "ext")]
    extern "C" {
        pub fn set_type(data_type: Type) -> Status;
        pub fn register_callback(
            target: *mut c_void,
            callback: Callback,
        ) -> Status;
        pub fn trigger_callback() -> Status;
    }
}

// An adapter layer to call external library.
mod ext {
    use std::mem::size_of;
    use std::os::raw::c_void;
    use super::sys;

    #[derive(Debug)]
    pub enum Error {
        BadType,
        NoCallback,
        NoCallbackTarget,
    }
    impl From<sys::Status> for Error {
        fn from(status: sys::Status) -> Error {
            match status {
                sys::BAD_TYPE => Error::BadType,
                sys::NO_CALLBACK => Error::NoCallback,
                sys::NO_CALLBACK_TARGET => Error::NoCallbackTarget,
                s => panic!("Uncatched error status: {}", s),
            }
        }
    }

    #[derive(Clone, Debug, PartialEq)]
    pub enum Type {
        Float,
        Short,
    }
    impl Type {
        pub fn byte_size(&self) -> usize {
            match self {
                Type::Float => size_of::<f32>(),
                Type::Short => size_of::<i16>(),
            }
        }
    }
    impl From<Type> for sys::Type {
        fn from(data_type: Type) -> sys::Type {
            match data_type {
                Type::Float => sys::FLOAT,
                Type::Short => sys::SHORT,
            }
        }
    }
    impl From<sys::Type> for Type {
        fn from(data_type: sys::Type) -> Type {
            match data_type {
                sys::FLOAT => Type::Float,
                sys::SHORT => Type::Short,
                t => panic!("Unknown type: {}", t),
            }
        }
    }

    pub trait Render {
        fn render(
            &self,
            data_type: Type,
            buffer: *mut c_void,
            item_size: usize,
            items: usize
        );
    }

    extern "C" fn callback_from_ext_lib<R>(
        target: *mut c_void,
        data_type: sys::Type,
        buffer: *mut c_void,
        item_size: usize,
        items: usize
    ) where R: Render {
        println!("{} > (callback_from_ext_lib) target address @ {:p}", file!(), target);
        unsafe {
            let callback_render_ref = &*(target as *mut R);
            callback_render_ref.render(data_type.into(), buffer, item_size, items);
        }
    }

    pub fn set_type(data_type: Type) -> Result<(), Error> {
        let status = unsafe { sys::set_type(data_type.into()) };
        convert_to_result(status)
    }

    pub fn register_callback<T>(target: *mut T) -> Result<(), Error> where T: Render {
        println!("{} > (register_callback) target address @ {:p}", file!(), target);
        let status = unsafe {
            sys::register_callback(
                target as *mut c_void,
                callback_from_ext_lib::<T>
            )
        };
        convert_to_result(status)
    }

    pub fn trigger_callback() -> Result<(), Error> {
        let status = unsafe { sys::trigger_callback() };
        convert_to_result(status)
    }

    fn convert_to_result(status: sys::Status) -> Result<(), Error> {
        match status {
            sys::OK => Ok(()),
            s => Err(s.into()),
        }
    }
}

mod callback {
    use std::mem::size_of;
    use std::os::raw::c_void;
    use std::slice;
    use super::ext;

    #[derive(Debug)]
    pub enum Error {
        BadType,
        Ext(ext::Error),
    }
    impl From<ext::Error> for Error {
        fn from(e: ext::Error) -> Error {
            Error::Ext(e)
        }
    }

    pub type CallbackArgs<'a, T> = &'a mut [T];
    type CallackFunc<T> = fn(CallbackArgs<T>);

    pub struct CallbackRender<T> {
        data_type: ext::Type,
        callback: fn(&mut [T]),
    }

    impl<T> CallbackRender<T> {
        pub fn new(data_type: ext::Type, callback: CallackFunc<T>) -> Result<Self, Error> {
            if data_type.byte_size() != size_of::<T>() {
                return Err(Error::BadType);
            }
            ext::set_type(data_type.clone())?;
            let mut render = CallbackRender { data_type, callback };
            println!("{} > (CallbackRender::new) render address @ {:p}", file!(), &render);
            render.init()?;
            Ok(render)
        }

        fn init(&mut self) -> Result<(), Error> {
            println!("{} > (CallbackRender::init) render address @ {:p}", file!(), self);
            ext::register_callback(self as *mut Self)?;
            Ok(())
        }

        pub fn trigger(&self) -> Result<(), Error> {
            ext::trigger_callback()?;
            Ok(())
        }

        fn get_buffer_data(&self, buffer: &mut [T]) {
            (self.callback)(buffer);
        }
    }

    impl<T> ext::Render for CallbackRender<T> {
        fn render(
            &self,
            data_type: ext::Type,
            buffer: *mut c_void,
            item_size: usize,
            items: usize
        ) {
            assert_eq!(self.data_type, data_type);
            assert_eq!(self.data_type.byte_size(), item_size);
            let data_buffer = unsafe { slice::from_raw_parts_mut(buffer as *mut T, items) };
            self.get_buffer_data(data_buffer);
        }
    }
}

fn callback_float(buffer: callback::CallbackArgs<f32>) {
    for data in buffer.iter_mut() {
        *data = 3.14;
    }
}

fn callback_short(buffer: callback::CallbackArgs<i16>) {
    for (index, data) in buffer.iter_mut().enumerate() {
        *data = index as i16;
    }
}

fn test_float() {
    use callback::CallbackRender;
    let render = CallbackRender::new(ext::Type::Float, callback_float).unwrap();
    println!("{} > (test_float) render address @ {:p}", file!(), &render);
    render.trigger().unwrap();
}

fn test_short() {
    use callback::CallbackRender;
    let render = CallbackRender::new(ext::Type::Short, callback_short).unwrap();
    println!("{} > (test_short) render address @ {:p}", file!(), &render);
    render.trigger().unwrap();
}

fn main() {
    test_float();
    test_short();
}

solution.rs

// A module containing all the converted types from external library.
mod sys {
    use std::os::raw::c_void;

    pub type Type = i32;
    pub const FLOAT: Type = 0;
    pub const SHORT: Type = 1;

    pub type Status = i32;
    pub const OK: Status = 0;
    pub const BAD_TYPE: Status = 1;
    pub const NO_CALLBACK: Status = 2;
    pub const NO_CALLBACK_TARGET: Status = 3;

    pub type Callback = extern fn(
        *mut c_void,    // target
        Type,           // data_type
        *mut c_void,    // buffer
        usize,          // item_size
        usize,          // items
    );

    #[link(name = "ext")]
    extern "C" {
        pub fn set_type(data_type: Type) -> Status;
        pub fn register_callback(
            target: *mut c_void,
            callback: Callback,
        ) -> Status;
        pub fn trigger_callback() -> Status;
    }
}

// An adapter layer to call external library.
mod ext {
    use std::mem::size_of;
    use std::os::raw::c_void;
    use super::sys;

    #[derive(Debug)]
    pub enum Error {
        BadType,
        NoCallback,
        NoCallbackTarget,
    }
    impl From<sys::Status> for Error {
        fn from(status: sys::Status) -> Error {
            match status {
                sys::BAD_TYPE => Error::BadType,
                sys::NO_CALLBACK => Error::NoCallback,
                sys::NO_CALLBACK_TARGET => Error::NoCallbackTarget,
                s => panic!("Uncatched error status: {}", s),
            }
        }
    }

    #[derive(Clone, Debug, PartialEq)]
    pub enum Type {
        Float,
        Short,
    }
    impl Type {
        pub fn byte_size(&self) -> usize {
            match self {
                Type::Float => size_of::<f32>(),
                Type::Short => size_of::<i16>(),
            }
        }
    }
    impl From<Type> for sys::Type {
        fn from(data_type: Type) -> sys::Type {
            match data_type {
                Type::Float => sys::FLOAT,
                Type::Short => sys::SHORT,
            }
        }
    }
    impl From<sys::Type> for Type {
        fn from(data_type: sys::Type) -> Type {
            match data_type {
                sys::FLOAT => Type::Float,
                sys::SHORT => Type::Short,
                t => panic!("Unknown type: {}", t),
            }
        }
    }

    pub trait Render {
        fn render(
            &self,
            data_type: Type,
            buffer: *mut c_void,
            item_size: usize,
            items: usize
        );
    }

    extern "C" fn callback_from_ext_lib<R>(
        target: *mut c_void,
        data_type: sys::Type,
        buffer: *mut c_void,
        item_size: usize,
        items: usize
    ) where R: Render {
        println!("{} > (callback_from_ext_lib) target address @ {:p}", file!(), target);
        unsafe {
            let callback_render_ref = &*(target as *mut R);
            callback_render_ref.render(data_type.into(), buffer, item_size, items);
        }
    }

    pub fn set_type(data_type: Type) -> Result<(), Error> {
        let status = unsafe { sys::set_type(data_type.into()) };
        convert_to_result(status)
    }

    pub fn register_callback<T>(target: *mut T) -> Result<(), Error> where T: Render {
        println!("{} > (register_callback) target address @ {:p}", file!(), target);
        let status = unsafe {
            sys::register_callback(
                target as *mut c_void,
                callback_from_ext_lib::<T>
            )
        };
        convert_to_result(status)
    }

    pub fn trigger_callback() -> Result<(), Error> {
        let status = unsafe { sys::trigger_callback() };
        convert_to_result(status)
    }

    fn convert_to_result(status: sys::Status) -> Result<(), Error> {
        match status {
            sys::OK => Ok(()),
            s => Err(s.into()),
        }
    }
}

mod callback {
    use std::mem::size_of;
    use std::os::raw::c_void;
    use std::slice;
    use super::ext;

    #[derive(Debug)]
    pub enum Error {
        BadType,
        Ext(ext::Error),
    }
    impl From<ext::Error> for Error {
        fn from(e: ext::Error) -> Error {
            Error::Ext(e)
        }
    }

    pub type CallbackArgs<'a, T> = &'a mut [T];
    type CallackFunc<T> = fn(CallbackArgs<T>);

    pub struct CallbackRender<T> {
        data_type: ext::Type,
        callback: fn(&mut [T]),
    }

    impl<T> CallbackRender<T> {
        pub fn new(data_type: ext::Type, callback: CallackFunc<T>) -> Result<Self, Error> {
            if data_type.byte_size() != size_of::<T>() {
                return Err(Error::BadType);
            }
            ext::set_type(data_type.clone())?;
            let render = CallbackRender { data_type, callback };
            println!("{} > (CallbackRender::new) render address @ {:p}", file!(), &render);
            Ok(render)
        }

        pub fn init(&mut self) -> Result<(), Error> {
            println!("{} > (CallbackRender::init) render address @ {:p}", file!(), self);
            ext::register_callback(self as *mut Self)?;
            Ok(())
        }

        pub fn trigger(&self) -> Result<(), Error> {
            ext::trigger_callback()?;
            Ok(())
        }

        fn get_buffer_data(&self, buffer: &mut [T]) {
            (self.callback)(buffer);
        }
    }

    impl<T> ext::Render for CallbackRender<T> {
        fn render(
            &self,
            data_type: ext::Type,
            buffer: *mut c_void,
            item_size: usize,
            items: usize
        ) {
            assert_eq!(self.data_type, data_type);
            assert_eq!(self.data_type.byte_size(), item_size);
            let data_buffer = unsafe { slice::from_raw_parts_mut(buffer as *mut T, items) };
            self.get_buffer_data(data_buffer);
        }
    }
}

fn callback_float(buffer: callback::CallbackArgs<f32>) {
    for data in buffer.iter_mut() {
        *data = 3.14;
    }
}

fn callback_short(buffer: callback::CallbackArgs<i16>) {
    for (index, data) in buffer.iter_mut().enumerate() {
        *data = index as i16;
    }
}

fn test_float() {
    use callback::CallbackRender;
    let mut render = CallbackRender::new(ext::Type::Float, callback_float).unwrap();
    println!("{} > (test_float) render address @ {:p}", file!(), &render);
    render.init().unwrap();
    render.trigger().unwrap();
}

fn test_short() {
    use callback::CallbackRender;
    let mut render = CallbackRender::new(ext::Type::Short, callback_short).unwrap();
    println!("{} > (test_short) render address @ {:p}", file!(), &render);
    render.init().unwrap();
    render.trigger().unwrap();
}

fn main() {
    test_float();
    test_short();
}