Importing Nim modules from C#

ImportNim.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Runtime.InteropServices;

namespace nim_sharp
{
    [StructLayout(LayoutKind.Sequential)]
    public unsafe struct NimSring
    {
        public Int32 Len;
        public Int32 Capacity;
        public char Text;
    }

    [StructLayout(LayoutKind.Sequential)]
    public unsafe struct NimSeq
    {
        public Int32 Len;
        public Int32 Capacity;
        public byte Data;
    }

    [StructLayout(LayoutKind.Sequential)]
    public unsafe struct NimType_ext
    {
        public Int32 x;
        public Int32 y;
        public NimSring* s;
    }

    public struct NimType
    {
        public int x;
        public int y;
        public string s;
    }

    class Program
    {
        [DllImport("nimlib.dll", CallingConvention = CallingConvention.Cdecl)]
        public static unsafe extern void populateArraySruct(int length, NimSeq** ArrayStructs);
 
        public static unsafe List<NimType> getResultsFromNim(int ArrL)
        {
            List<NimType> convertedResults = new List<NimType>(ArrL);
            
            NimSeq* nimResults = null;
            populateArraySruct(ArrL, &nimResults);
            
            NimType_ext* currentItem = (NimType_ext*) &(nimResults->Data);
            for (int i = 0; i < ArrL; i++, currentItem++)
            {
                convertedResults.Add(new NimType() {x = currentItem->x, y = currentItem->y, s = new string(&(currentItem->s->Text))});
            }
            return convertedResults;
        }

        static void Main(string[] args)
        {
            var list = getResultsFromNim(10);

            foreach (var elem in list) {
                Console.WriteLine("x: {0}, y: {1}, s: {2}", elem.x, elem.y, elem.s);
            }
        }
    }
}

nimlib.nim

# compile with nim c -app:lib nimlib.nim
# place next to the C# executable together with a compiled copy of nimrtl.dll

type
  NimTypeSeq = seq[NimType]
  NimType = object {.packed.}
    x: int
    y: int
    s: string
 
proc populateArraySruct(outputLength: int, output: var NimTypeSeq) {.exportc,dynlib.} =
  setupForeignThreadGc()

  output.newSeq(outputLength)
  
  for i in 0 .. <outputLength:
    output[i] = NimType(x: i, y: i*2, s: $i)
    

i am here, i need to test the code. so

• in #V1 i don't have the LPStr needs a using ?
  reading about it in MSN reveals it's actually a type char* but with some difference ?
• in #V2 is one of the advantages is that it will be managed by the nims GC?
• i will test the code if i'll understand the first point about LPStr before you will get to read this,
  and i will benchmark to see if there's any difference in performance
• last thing, i want to understand what did you offer about "one single managed C++ module"

l appreciate your kindness and the time you put into this, this is a building block for my applications i'm trying to come up with the most efficient dataType with the most efficient low level language
(for now i see nim as a very good candidate over D Rust Go & co' )
later in small steps as i learn Nim i will move more and more of my code from c# to be implemented in nim
so your help is much appreciated

i am newb also with github use , but i think that you could access the fork https://gist.github.com/zah/fe8f5956684abee6bec9/forks

something happened zah. i don't know how i had everything allmost everything was working , and at some point i was not satisfied with timing , tried to leave copying memory into the buffer , and do it only when needed to access meaning from c# rather in-front from nim cause it effected the performance, so as i tried to change some things, i messed up that bad that i can't even revert and fall-back to any , and i mean any version of the whole 48 - 72 hours of work i spent on it , and succeeded with most , now as if visual studio got some instructions from Microsoft to resist nim !!
maybe i should just rest and get back to the subject from scratch

but i will be so much happy if you could check on my fork and help me cause i think 5 minutes of your coding equals to 50 hours of mine . on that issue.

So, did you get it to work in the end?

I've checked the code in your fork, and you seem to be on the right track. Using char * instead of LPStr is perfectly fine and immediately converting the value to a native C# string is the most sensible thing to do here.

If we start with your original code on StackOverflow, the required changes are only minor:

public unsafe static class DataPackg
{
    [StructLayout(LayoutKind.Sequential)]
    public struct NimString
    {
        public uint Len;
        public uint Capacity;
        public char* Text;
    }

    [StructLayout(LayoutKind.Sequential)]
    public struct ExtenralTestC
    {
        public uint Id;
        public NimString NimStr;
    }

    public struct NativeTestC
    {
        public uint Id;
        public string Str;
    }
}

public static unsafe List<DataPackg.NativeTestC> PopulateLstPackC(int ArrL)
{
    DataPackg.ExternalTestC* PackUArrOut;
    List<DataPackg.TestC> RtLstPackU = new List<DataPackg.NativeTestC>(ArrL);
    c_returnDataPack((uint)ArrL, &PackUArrOut);
    DataPackg.ExternalTestC* CurrentPack = PackUArrOut;
    for (int i = 0; i < ArrL; i++, CurrentPack++)
    {
        RtLstPackU.Add(new DataPackg.NativeTestC() { StrVal = new string(CurrentPack.NimStr.Text), Id = CurrentPack->Id });
    }

    return RtLstPackU;
}

If this code is extremely performance critical and you must avoid any string copies, you can expose the rest of the C# program to the NimString type, but you have to make sure that the strings in the Nim module won't be de-allocated while C# is using them. There are several ways to do this - you can just keep a variable referencing them somewhere, or you can use the GC_ref proc from the system module to increase the reference count of the strings.

hey zah, i have reveled the bug, i now know at least what is the bug not how to solve it or who is the falt on nim or my settings for interop ( as they apply to any other language)
i have made new struct simple one no strings; a blittable struct two int members
say int a , int b;
then interoping and marshaling it back to c# they switch places,
say you print to console , you will see not in a stable order but they allmost liniar you try to print
struct.memberA and first itration returns the length of array (nice but it is not a variable nor a member in any place) then it prints A, B , A , B values , like it spins not in order.
memory is UN-ordered
did u ever encounter this behavior ?

Robbie, I think I'll have to look at your complete code before I can help you further.
If you are doing this on your job, you may try to convince your boss to let me access your workstation over Remote Desktop or TeamViewer. I'm one of the core Nim developers with many contributions to the compiler*, so it should be easy to convince them that I'm the right person for this.

• https://github.com/nim-lang/Nim/graphs/contributors

Otherwise, try to post a complete compiling minimal example somewhere, so I can take a look.

wow 3 c# types, and that line is complex.. i hope i will get to understand one day

NimType_ext* currentItem = (NimType_ext*) &(nimResults->Data);

thanks i will defenitly find the time first thing in the morning to start fresh
with the new implementation

thanks a great lot !
edit: aha.. just a cast..like you accesed the envelop.data

hey what's up , tried my best, played with current code, first this does not compile:
line# 57 in code above(C#) s = new string( &(currentItem->s->Text) )
would not compile can't convert x** to x*
and i made some changes tried what i could, at the end when i compiled + any kind of edit did not fix our
issue.

here is the BenchMark Cs project Folder
https://jumpshare.com/v/r1kI75HBfJEKnzlirt2z

I posted the code after compiling it and testing it with Visual Studio 2013 (.NET Framework 4.5). "Allow unsafe code" must be enabled. The Text variable must be a char as in my example, not char *.

i have replaced the char* to single char
and did some thing else - changed all to int - x,y,s
still i could see the outer struct ->data member

is 160, capacity 0, lengh 10k as i assigned
the outer seq is then accessed to data member and assigned to the
nimtype_ext* so now it' still exchanges "slots" between x, y , z
might be 2010 vs 2013, but i suspect something to do
with the nimrtl.dll,
thinking about it more, maybe the solution will come from your side,
upload the project folder , i will then compile only the cs. with my .net 4 vs2010
should not be problem unless ...only 4.5 is ok with nim i susspect not.

Here is my complete solution, including the build outputs:
https://dl.dropboxusercontent.com/u/151344/nim-sharp.zip

just downloaded it many thanks you're very kind.
i would love to help with nim when i would be able to.

ok now all i can say is it just works
and as soon as i alter the nimlib.nim, and compile :
nim c -d:release --app:Lib nimlib.nim

running nimsharp.exe gives an exception :

: System.BadImageFormatException: An attempt was made to load a program with an incorrect format.
(Exception from HRESULT: 0x8007000B)

thinking it's what i have change in source.nim (which is impossible)
i at this point undo all , and compile again, but as i suspected, it's nothing to do with the code, but the compilation. meaning some environment setting with nims compiler or the way i compile
i did nothing with the other nim dll maybe there is some missing step to do with nimrtl.dll

ps. and offcourse i forgot to point out as soon as i replace the original dll by overwriting nimlib.dll
everything is back to work as it did at first.

another clue is while i tried to compile nim when dll is in use i could see what it tried to execute
using x86_64-w64-mingw32 so maybe you could see the differences in my nim settings from this output .

as when it did not succeed gcc.exe plots out :

G:\RobDev\Documents\Visual Studio 2010\Projects\CPP_Interaction\csNim\NimSharp>
nim c -d:release --app:Lib nimlib.nim
Hint: system [Processing]
Hint: nimlib [Processing]
CC: nimlib
CC: system
Hint: [Link]
G:/RobDev/ProgramsDev/Nim-0.12.0/dist/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.9.1/../../../../x86_64-w64-mingw32/bin/ld.exe:
cannot open output file g:\robdev\documents\visual studio 2010\projects\cpp_interaction\csnim\nimsharp\nimlib.dll: Permission denied
Error: execution of an external program failed: 'gcc.exe -shared -o "g:\robdev\documents\visual studio 2010\projects\cpp_interaction\c
snim\nimsharp\nimlib.dll" "g:\robdev\documents\visual studio 2010\projects\cpp_interaction\csnim\nimsharp\nimcache\system.o" "g:\robdev
\documents\visual studio 2010\projects\cpp_interaction\csnim\nimsharp\nimcache\nimlib.o" '

and yet another clue which contrasts my opinion on the relation with nimrtl.dll is when it's removed from the folder it has no effect, so it has nothing to do with nimrtl.dll as if it does not use it or maybe when you compiled against nimrtl.dll it allready did what it needs to do with nimrtl.dll so now the nimlib.dll allready have what it needs ...could be possible i really don't know the relations between them so it's for someone with the knowledge to figure it out.

uninstalled nim 32/64 version which was a 64bit setup

installed full package of 32bit nim setup
now it does work encoding is wrong shows string as????
but members of struct are now aligned and i guess you are running 32bit too
and your project i have downloaded is 32bit am i right or am i right?

s = new string(&(currentItem->s->Text))});
this will throw an error bad pointer in 64 bit if it is clear for you as to why...

Yes, my builds were 32-bit and I'm using a Nim compiler compiled straight from the devel branch. On Windows, I also use the older 32-bit mingw distribution.

Basically, these problems are not very hard to trouble-shoot, but you have to be a bit more seasoned C/C++ developer. You can always examine the C code produced by Nim (placed in the nimcache folder) and you can use the "Memory" debug window in Visual Studio to see the actual data written by the Nim dll. From there, it's just a matter of comparing your conceptual model expectations with the reality you see in the debugger.

the only problem i have left as i mention in former post, as we only pass numeric char now it does work but encoding is wrong shows string as??? is this an issue or simple solution ? try concat $i as s= "abc" & $i

Nim strings are usually stored in UTF-8, although the language doesn't require that. In C#, the encoding is UCS-2 (UTF-16). Take a look at the following stackoverflow answer that features some conversion code:
http://stackoverflow.com/questions/10773440/conversion-in-net-native-utf-8-managed-string

StringFromNativeUtf8 is the one i think, i will take a time to test both ways to see if that is the solution, as this has to be the one, i will report to you as soon as i find the time to play with it again, thanks i think that concludes the subject of the string passing from nim to c# which was not a simple task for me, as soon as i will confront this, and do some more tests i will be more familiar and it will be simpler from that point, i'll let you know. thanks a lot as ususall !