To handle certain kinds of operations better (and not leak memory), NativeCall needs to be able to control the lifecycle of objects. Specifically, we need to be able to relinquish control of memory allocated on the Perl side (so that it can be handed to C code), and the ability to take responsibility for a chunk of memory (so that it will be freed when the object is GCed.
To do this, I suggest adding two new functions, adopt() and disown(), and reworking the string marshalling rules.
These two functions do what you'd expect. adopt() assumes control of the memory allocated by the object, indicating that it should be freed when the object is GCed. disown() releases control, so that the memory is untouched on GC. Neither function should apply recursively to the members of structs or arrays.
NativeCall objects created in Perl 6 will be adopted by default, and those returned from C will be disowned by default.
Implementation-wise, I expect to add a flag for adoption state in the dyncall REPR types and NQP ops for setting it. Zavolaj will then only need to expose the ops through the functions.
explicitly-manage() is terribly named, not very elegant, and I don't like it much. So we'll remove it and replace it with something else.
Under the new scheme, a normal Str object passed to a native function will be marshalled to a temporary string buffer which is sent to C and immediately freed after the function returns. No exceptions.
If you need more expressive handling than that, I suggest adding two methods to the CArray class: decode and encode to explicitly convert between CArray[int8] and Str. The first method would convert a CArray to a string and the latter munge a string into a byte representation. These CArrays can then be adopted and disowned as needed.
We'll probably also want to let the user explicitly set the length of a CArray. Often a library will do something along the lines of char *some_operation(..., int *bytes) where the length of the returned buffer is stored in the variable pointed at by bytes, which is information which it would be natural to store in the CArray along with the data.
Implementation-wise this will mean we can kill the CStr REPR (and explicitly-manage()!) and some reworking of the Perl 6 plumbing, but hopefully not very much on the NQP/6model level.
Rakudo's Buf implementation is some distance from what the specs mandate, and uses pir:: operations directly. I think a good way to fix both of these problems, and alleviate some of the troublesome parts of string handling in Zavolaj is to provide buffer primitives in NQP. I think these five ops will give us what we want:
nqp::buffer($bits, *@values)- Create a buffer whose elements have bitwidth $bits, and use @values for initial values.nqp::buffer_get($buf, $index)- Get the value at position $index.nqp::buffer_set($buf, $index, $value)- Set the value at position $index.nqp::buffer_elems($buf)- Get the number of elements in the buffer.nqp::buffer_elemwidth($buf)- Get the bitwidth of the buffer's elements.
The supported bitwidths, at least initially, should be 8, 16, 32 and 64. Fancier widths like 1, 2 and 4 can come later.
On NQP-Parrot this should probably be implemented as a custom PMC, similar to QRPA. On NQP-JVM, I think we'll need separate classes for each supported width (since int[] and byte[] are different types), but a properly factored abstract superclass should keep the code duplication to a minimum.