Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
OC Network Packet Serilisation
-- OC Network Packet Serialisation
-- This requires lua-MessagePack found at https://github.com/fperrad/lua-MessagePack
-- Note that is a dirty hacky hack! But this should work.
-- Future debuggers, take note:
-- there is a race condition that could happen with this requiring!
local msgpack
do
local previous_msgpack = package.loaded["msgpack"]
package.loaded["msgpack"] = nil
msgpack = require "msgpack"
package.loaded["msgpack"] = previous_msgpack
end
-- msgpack configuration
msgpack.set_array('with_hole')
msgpack.set_string('binary')
-- advanced stuff
msgpack.packers['function'] = function(buffer, func)
func(buffer)
end
local function STRING(str)
if str == nil then
return nil
else
return function(buffer)
msgpack.packers['_string'](buffer, str)
end
end
end
return {
ser = function(src, dst, prt, ...)
if (src ~= nil) and (type(src) ~= "string") then
error("bad arugment #1 (string expected, got " .. type(src) .. ")")
end
if (dst ~= nil) and (type(dst) ~= "string") then
error("bad arugment #2 (string expected, got " .. type(dst) .. ")")
end
if (prt ~= nil) then
if type(prt) == "number" then
if math.floor(prt) ~= prt then
error("bad argument #3 (not a whole number)")
end
elseif type(prt) ~= "integer" then
error("bad arugment #3 (integer or number expected, got " .. type(prt) .. ")")
end
if prt < 0 then
error("bad argument #3 (positive/unsigned number/integer expected)")
end
end
local dat = table.pack(...)
local dat_len = dat.n
dat.n = nil
local trail_nil = dat[dat_len] == nil
local all_nil = true
for _,v in pairs(dat) do
all_nil = all_nil and (v == nil)
end
if (dat_len == 0) or all_nil then
dat = nil
elseif dat_len == 1 then
dat = dat[1]
end
if (not trail_nil) or (dat_len == 0) then
dat_len = nil
end
return msgpack.pack({
[STRING("src")] = STRING(src),
[STRING("dst")] = STRING(dst),
[STRING("prt")] = prt,
[STRING("len")] = dat_len,
[STRING("dat")] = dat
})
end,
des = function(packed)
if type(packed) ~= "string" then
error("bad argument #1 (string expected, got " .. type(packed) .. ")")
end
local up = msgpack.unpack(packed)
if up.dat == nil then up.dat = {}
elseif type(up.dat) ~= "table" then up.dat = {up.dat} end
if up.len == nil then up.len = #up.dat end
return up.src, up.dst, up.prt, table.unpack(up.dat, 1, up.len)
end
}

OC Network Packet Serialisation

This document defines a method to serialise packets in a form that represents the packets in the built in OC network system. This does not define more than a way to serialise packets that OC uses, and details such as handling and routing are up to the implementer. In particular it is worth noting that packets that make no sense may be encoded in this format, and the handling of which is up to the user of this format.

Note that this document is written in British English, so some words, notability "serialise" may be spelled differently than US English.

OC Packet Format Description

Note that is is only a description of the existing system that this format tries to represent. It does not define anything new.

There are only three guaranteed values in an OC modem packet, the source address, the destination address, and the port. It is worth noting that under normal circumstances, the source address cannot be changed from the address of the "modem" that sent the message.

As a special case of the OC modem packet, there is the "tunnel". When sending, there is no port value, and when receiving, the port is reported as 0 on the receiving end. In this format, the port in this case will be represented as a null value.

After the guaranteed values, there are the optional values. By default there is a limit on the number of optional values that can be sent, as well as a total size limit (which also includes "overhead").
There can be a variable number of optional values. They can be arrays of bytes (strings), numbers, booleans, and nil.

It is worth noting, that unlike normal use in Lua, nils are preserved and transmitted. Most of the time, this will not matter, but is important to consider.

Terminology

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

  • An "implementation" is an implementation of this standard, most likely a library, that uses a MessagePack implementation / library to convert it to a more native / accessible data format. Given rules to follow to ensure that the packets will be converted consistently.
  • A "user" is the user of an implementation, such as a program. Given advice that will help with using this format.

Serialised Packets

The serialised form of packets is in "MessagePack" format. This is a binary format, and is described at https://github.com/msgpack/msgpack/blob/master/spec.md (last edited 2017-12-22, accessed 2017-05-12).

In this document, MessagePack data will be represented in the JSON format, as defined in RFC4627.

Packet Layout

The packet can be represented as a MessagePack Object. The purpose of each member of the object MUST determined by its name. Every value is optional in this specification, but users of this format MAY decide to add their own requirements.
For illustrative purposes, JSON is used in this document to explain the layout, but care should be taken that some strings are actually byte-arrays (MsgPack "bin"), rather than strings, specified later on. Also take care to note that MessagePack stores numbers differently to JSON. JSON is only used for terminology and illustrative examples!

Name Type for member/value
src String
dst String
prt Unsigned Integer (Number)
len Unsigned Integer (Number)
dat Any except object

The rules for dat are more complicated. It SHALL be any JSON type other than an Object. If it is an Array, it represents multiple values, each of which can be any JSON type other than an Object or an Array. If it is not an Array or a Null, it MUST be treated the same as if it was a single element array with only that value. If the value is null, it MUST NOT be considered the same as a single element array containing it. Note that any Lua strings in or as dat MUST be encoded as raw data / byte array / MsgPack "bin".
Similarly, note that src and dst SHOULD be encoded as strings / MsgPack "str" (UTF-8 encoded string).

Both prt and len SHOULD be encoded as MsgPack unsigned integers!

Any value MAY be null, which MUST be treated the same as if the member/value did not exist unless it is in the dat Array.
Implementations MAY NOT treat null and no value differently, however they SHOULD preserve the difference between null and nothing.

For null in Arrays, positions MUST be preserved if a null is between elements in an Array. For example the JSON/MessagePack Array ["a", null, "b"] MUST be converted to this Lua table {[1]="a", [3]="b"}, preserving the fact that there was a null in between.

There is no way to represent null at the end of an array in Lua, so ["z", null] MAY be converted to {[1] = "z"}. However, trailing nulls SHOULD be preserved if at all possible.
It is also highly RECOMMENDED to store the length of the array in the len value/member (in practice, this will probably be needed most of the time with trailing nulls, unless both sides can handle them).
The len value MUST be equal to or greater than the actual length of the array if the value for len exists (or is not null). If the len value is greater than the length of the dat array stored, then there MUST be an indication that there would have been nulls to the user, either padding the list with nulls if supported or returning the length value. If there is no dat (or the value for dat is only null), then the list SHALL be a list made of the number of nulls indicated in the value for len.

Example Layouts

Packet that uses every feature
{
	"src": "c5e8ccb1-d367-4bc4-a351-d6b23e2e38d6",
	"dst": "c33fc614-219c-4832-bd8e-e292a49b7dc0",
	"prt": 42,
	"len": 7,
	"dat": ["example", null, "packet", 6, 7, null]
}
Packet that uses common features
{
	"src": "c5e8ccb1-d367-4bc4-a351-d6b23e2e38d6",
	"dst": "c33fc614-219c-4832-bd8e-e292a49b7dc0",
	"prt": 42,
	"dat": ["example", "packet", 2]
}
Most minimal packet
{
}

Extensions

Due to the flexible format of MessagePack, it is possible to extend the packet to include more data. Implementations MUST accept packets with extensions that do not break any of the earlier rules, but users of those implementations MAY do whatever they want. Users SHOULD negotiate (outside the scope of this) for what extensions are used.

Authors and Credits

  • Skye M. - Original author of this document.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment