blog.md

MasterMemory v3 - A Fast Read-Only In-Memory Database for C# with Source Generator Support

We've released MasterMemory v3! It finally supports Source Generators!

MasterMemory is a C# in-memory database that is fast, memory-efficient, and type-safe. It's 4700 times faster than using SQLite directly!

Originally, MasterMemory had an advanced design philosophy of generating C# code from C# code, doing Source Generator-like tasks in an era before Source Generators existed. When porting it now, I was impressed by how smoothly it could be ported and how the legacy code worked without any modifications. The times have finally caught up...

As such, database construction code and query portions are automatically generated by Source Generator from C# definitions like this:

[MemoryTable("person"), MessagePackObject(true)]
public record Person
{
    [PrimaryKey]
    public required int PersonId { get; init; }
    
    [SecondaryKey(0), NonUnique]
    [SecondaryKey(1, keyOrder: 1), NonUnique]
    public required int Age { get; init; }

    [SecondaryKey(2), NonUnique]
    [SecondaryKey(1, keyOrder: 0), NonUnique]
    public required Gender Gender { get; init; }

    public required string Name { get; init; }
}

Since it's generated as C# code, not only do all queries have input completion and type-safe return values, but it also contributes to better performance.

Since it's used as a read-only database, immutable class definitions are preferable, and with recent C# features like record, init, and required, it's become even more convenient to use as a Readonly Database. While required isn't available in Unity, record and init are, so there's no problem using it with Unity.

Note that the Unity version is now provided through NuGetForUnity. Also, it requires MessagePack for C# v3, which supports Source Generator.

Next

MasterMemory is actually quite widely used. I've started to see it being adopted in games more frequently. So I'm really happy that we've finally resolved the hassle of code generation from external tools, which had been causing quite some concern!

Migration from v2 to v3 shouldn't be too difficult. We deliberately avoided touching the quality of generated code, core functions, and method signatures, so it should work right out of the box just by removing the parts where you were running the command-line tool. Just make sure to set the namespace using assembly attributes.

Additionally, we've added support for records (which we hadn't done before!) and #nullable enable (which we also hadn't done before!), so the usability should be improved beyond just the generated parts.

In the future, we're considering adding MemoryPack support, modernizing the API further (it's currently netstandard2.0, so it's old), and making overall improvements (like replacing generated code parts such as ImmutableBuilder). There's a lot we can do, so I hope we can work on these improvements when the opportunity arises.