The new MessagePack website says that MessagePack for Lua is "coming soon". I thought it would be a good idea to make a quick recap on the state of MessagePack in Lua.
The historical implementation of MessagePack is mplua by Nobuyuki Kubota. It is the implementation linked from the MessagePack wiki. However, it is rather old (no updates for two years) and not very fast.
A bit less than a year ago I had to use MessagePack in LuaJIT and I wrote a FFI-based binding on the official C interface of MessagePack. I had to include a small C library that had to be compiled to wrap inline functions in the interface.
I did not like that design and it had performance issues in real use cases, mostly because I had to use calls that break the JIT mode in LuaJIT. I decided to re-implement MessagePack directly in LuaJIT with no calls to any external library but still using FFI ctypes. I called it luajit-msgpack-pure. This is the implementation I have been using to this day, it is a good solution for LuaJIT.
Starting from my code base, the awesome Kengo Nakajima ported it to pure Lua (5.1) -> lua-msgpack. His implementation is slower than luajit-msgpack-pure, mostly because there is no way to make a buffer as efficient as with FFI ctypes in pure Lua (EDIT: François Perrad proved this wrong).
Dissatisfied with the speed, he wrote lua-msgpack-native which uses a C library. I think this is probably the fastest implementation of MessagePack for any Lua platform as of now. It targets Luvit and can apparently also work with LuaJIT. I am not sure about PUC Lua (5.1, 5.2?).
A similar implementation is Salvatore Sanfilippo's lua-cmsgpack. It was originally written for the Redis project. It is a single self contained C file with no dependencies and targets Lua 5.1.
So, to sum things up:
- If you use LuaJIT and are OK with C modules -> lua-msgpack-native
- If you use PUC Lua 5.1 and are OK with C modules -> lua-cmsgpack
- If you want pure LuaJIT -> luajit-msgpack-pure
- If you want pure Lua 5.1 -> lua-msgpack
Regarding Lua 5.2: I don't now. I plan to try to make luajit-msgpack-pure compatible with Lua 5.2 + luaffi someday, it should not be too hard. Ping me if you need this! It is probably not too hard to port lua-cmsgpack to Lua 5.2 either.
Cool!
There is a similar situation with MessagePack for Go (historical implementation which is not actively maintained and another implementation).
Here let me explain why I wrote "coming soon.":
In my opinion, as an open source project, single project with many developpers is better than many projects with few developpers. In addition, from the users' point of view, "MessagePack" should be an unified project which provides a reliable implementation for each language to establish a good reputation and get widely adopted (Adoption is essentially important because msgpack is a data exchange format which should be universal).
But it's true that sometimes we can't avoid having multiple implementations. For example, the author (or major committers) may quit maintenance. It likely occurs in MessagePack projects because historically, most of them started as a weekend hack. In fact, I once suffered this issue with PHP and C# implementations, and they were replaced by other implementations.
Or, some languages need multiple implementations for a technical reason such as: LuaJIT C module/PCU Lua C module/Pure Lua, Cython/PyPy/Pure Python, CRuby/JRuby/mruby/Rubinius/Pure Ruby, etc.
For now, I don't have a perfect idea how to write that section. If you have an idea, I'd like you to let me know.