Work in Progress Looking for feedback on the design and code.
From what I can tell, there are some issues with using realtime_tools::RealtimeBuffer in our ros2_control hardware interfaces. It seems this RealtimeBuffer is only useful for writing to a realtime based on the use of the new_data_available internal variable. For reading state from the realtime thread we dont have an existing RealtimeBuffer.
The internal implementation of the "swapping double buffer pointers" works for both modes, only the interface (read vs write) to this swapping buffer needs to change. Therefor I refactored the swapping double buffers out into the MemoryBarrier class, which also defines a sub-type called DirectAccess<> to protect direct access to the buffers.
The realtime thread is expected to use the DirectAccess class to get direct access to the buffers without memory copies. However, the non-realtime thread should use either WriteBarrier or ReadBarrier. These two classes control the direction of data flow to/from the realtime thread, respectively. IMO It makes sense that the reads and writes be through seperate class objects and not via some bi-directional interface.
TL;DR - skip to the examples at the bottom of this README. The usage is simple.
The following descriptions are the brief overview. There are more methods and fields that you can dig into in source but not important to the story.
MemoryBarrier implements a double buffer swapping mechanism for interchanging data between threads. It allocates two T objects on the heap. When MemoryBarrier::swap() is called the two pointers are swapped. Generally one pointer is accessed from the non-realtime thread and the other is used by the realtime thread.
You should not use this class directly, you will work with ReadBarrier, WriteBarrier and DirectAccess classes only.
Methods: bool new_data_available() - read only. returns true if flag indicates new data available. initialize(T value) - initializes both sides of the memory buffer to the same value. memory() - access the MemoryBarrier object
Protected Fields: T* nrt_ - the non-RealTime buffer T* rt_ - the RealTime buffer bool new_data_available_ - flag indicating if new data is ready (context depends on direction)
ReadBarrier implements reading data from a realtime thread using a MemoryBarrier. The default constructor will create a new memory barrier for use. There is an alternate constructor if you want to create your own MemoryBarrier explicitly.
Methods: pull(T& val) - swap RT buffer for non-RT, copy the new data into val and reset the new_data_available flag. current(T& val) - copy the current data into val. No swap is performed and the new_data_available flag is unaffected. memory() - access the MemoryBarrier object
WriteBarrier implements writing data to a realtime thread from a non-realtime thread using a MemoryBarrier. The default constructor will create a new memory barrier for use. There is an alternate constructor if you want to create your own MemoryBarrier explicitly.
Methods: push(T& val) - swap RT buffer for non-RT, copy the new data from val into non-RT buffer and set the new_data_available flag. current(T& val) - copy the current data into val. No swap is performed and the new_data_available flag is unaffected. memory() - access the MemoryBarrier object
MemoryBarrier class gives no access to the buffers itself, all read/write calls to the buffers are done by creating a DirectAccess<> object that protects access to the buffers through a mutex with behaviour defined by the lock_strategy. The DirectAccess class is a simply object and should be allocated on the stack not heap so as to prevent calls to malloc().
The ReadBarrier and WriteBarrier classes also use a DirectAccess object to access the MemoryBarrier but only long enough to perform the non-realtime memory copy.
Parameters: access_mode - either realtime or non_realtime. lock_strategy - can be try_lock or wait_lock. If not specified defaults to try_lock when access_mode=realtime and wait_lock when non_realtime.
Methods: swap() - rotate the double buffer pointers. non-RT and RT buffer is swapped. new_data_available(bool) - sets the flag indicating if new data is available to the destination thread...whether that be to the RT thread (Write) or to the non-RT (Read) side. reset() - detaches from a MemoryBarrier, unlocking it immediately. The destructor also does this for you.
Smart_ptr semantics: DirectAccess also implements all the *, ->, and get() methods like smart_ptr for dereferencing the memory buffer. These are all inline so you can access the underlying memory efficiently.
Some types are used to configure the templates above. Good defaults are already chosen based on RT/non-RT access modes so you should not have to bother with these. They are:
Determine what kind of locking strategy should be performed when accessing the memory barrier. wait_lock will wait indefinately for the lock to free up whereas try_lock will only try once and fail if MemoryBarrier is currently locked.
Indicates an access mode. Can be used to determine the proper locking_strategy or to indicate if you want DirectAccess to the RT or non-RT memory buffer.
- I chose pull() and push() names because I felt read/write doesnt clearly indicate if a swap is performed, where push/pull for me is more clear. For example, ReadBarrier::read() did swap yet WriteBarrier::read() did not. Instead, WriteBarrier::read became current() and ReadBarrier::read became pull() and I feel this is less confusing.
- There is also a polarity field in MemoryBarrier, this is currently unused and will probably go.
- Should this not be called Barriers? "memory barrier" is also used elsewhere in programming.
In your class header file declare a read and write barrier.
// Barrier is placed in realtime_tools namespace, hopefully it will end up there.
using realtime_tools;
typedef struct { ... } StateData;
typedef struct { ... } CommandData;
// you might want to declare data structs here if you are binding to handles.
// for this example, we'll keep it simple and declare inside the method.
// StateData state;
// CommandData command;
// declare a two-way data link to our realtime thread
ReadBarrier<StateData> state_rtb;
WriteBarrier<CommandData> command_rtb;In your non-RealTime read() method:
void read()
{
StateData state;
// read data from the realtime thread if there is new data
// will cause a swap of the internal buffers
if(state_rtb.pull(state_))
LOG("new data read");
}In your non-RealTime write() method:
void write()
{
StateData command;
// you might need to get the current values first, you can
// read values using current(T&) without affecting the MemoryBarrier.
// command_rtb.current(command);
// set our new command
command.position = 15;
command.velocity = 2.3;
// write data to our realtime thread and set the "new data" flag
// will cause a swap of the internal buffers
if(command_rtb.push(command))
LOG("data successfully written");
else
LOG("write failed, possibly a locking failure?");
}In your RealTime thread method:
void process_realtime()
{
// write state to non-RT
decltype(state_rtb)::DirectAccessType state(command_rtb);
// typeof(state) is MemoryBarrier<StateData>::DirectAccess<realtime, try_lock>
state->position = 8; // using smart_ptr-like semantics
state.new_data_available(true); // indicate to non-RT there is new state data
state.reset(); // unlock the state barrier
// read new commands from non-RT
decltype(command_rtb)::DirectAccessType command(command_rtb);
// typeof(command) is MemoryBarrier<CommandData>::DirectAccess<realtime, try_lock>
if(command.new_data_available()) {
// todo: write command->position and command->velocity to hardware
command.new_data_available(false); // indicate we read the commands
}
command.reset(); // redundant, would fall out of scope and reset anyway
}