Skip to content

Instantly share code, notes, and snippets.

@shibukawa
Last active August 29, 2015 14:08
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save shibukawa/5d555ccd1d6c80cbdfd0 to your computer and use it in GitHub Desktop.
Save shibukawa/5d555ccd1d6c80cbdfd0 to your computer and use it in GitHub Desktop.
data format like JSON

alt binary Json idea

  • 100% compatible json and binary format
  • First priority environment is browsers
  • It can reduce size by using external string table (hash key etc)
  • Compressed by lz4
  • 16bit align
  • String should be utf-16le
  • Don't support streaming encode/decode

Type ID

0b 0xxxxxxx xxxxxxxx

INT15 int (15bit)

0b 110000xx xxxxxxxx

SOBJ object (under 1023 elements)

0b 110100xx xxxxxxxx

SINT16OBJ object (only have int16, under 1023 elements)

0b 110010xx xxxxxxxx

SINT32OBJ object (only have int32, under 1023 elements)

0b 110110xx xxxxxxxx

SNUM64OBJ object (only have num64, under 1023 elements)

0b 110001xx xxxxxxxx

SSTROBJ (only have string, under 1023 elements)

0b 111000xx xxxxxxxx

SLIST list (under 1023 elements)

0b 111100xx xxxxxxxx

SINT16LIST list (only have int16, under 1023 elements)

0b 111010xx xxxxxxxx

SINT32LIST list (only have int32, under 1023 elements)

0b 111110xx xxxxxxxx

SNUM64LIST object (only have num64, under 1023 elements)

0b 111001xx xxxxxxxx

SSTRLIST list (only have string, under 1023 elements)

0b 101000xx xxxxxxxx

SSTR string (8bit string table index)

0b 11111111 00000000

INT16 int (16bit)

0b 11111111 01000000

INT32 int (32bit)

0b 11111111 00100000

NUM64 number (64bit)

0b 11111111 01100000

STR string (16bit table index)

0b 11111111 00010000

FALSE false

0b 11111111 00001000

TRUE true

0b 11111111 01011000

NULL null

0b 11111111 010001xx

reserve

0b 11111111 10100000

STR string (16bit string table index)

0b 11111111 11000000

OBJ object (under 65535 elements)

0b 11111111 11010000

INT16LIST object (only have int16, under 65535 elements)

0b 11111111 11001000

INT32LIST object (only have int32, under 65535 elements)

0b 11111111 11011000

SNUM64OBJ object (only have num64, under 1023 elements)

0b 11111111 11000100

STROBJ object (only have string, under 65535 elements)

0b 11111111 11100000

LIST list (under 65535 elements)

0b 11111111 11110000

INT16LIST list (only have int16, under 65535 elements)

0b 11111111 11101000

INT32LIST list (only have int32, under 65535 elements)

0b 11111111 11111000

NUM64OBJ object (only have num64, under 1023 elements)

0b 11111111 11100100

STRLIST list (only have string, under 65535 elements)

todo

  • add 64bit number list/object

format BNF

BNF:

document ::=
   string_table, binaryjson

string_table ::=
   int16 strings

strings ::=
   int16 string strings*

binaryjson ::=
   SOBJ object_values
   SLIST array
   OBJ int16 object_values
   LIST int16 array_array

typedarray ::=
   SINT16LIST int16_array
   SINT32LIST int32_array
   SNUM64LIST num64_array
   SSTRLIST str_array
   INT16LIST int16 int16_array
   INT32LIST int16 int32_array
   NUM64LIST num64_array
   STRLIST int16 str_array

typedobject ::=
   SINT16OBJ int16_values
   SINT32OBJ int32_values
   SNUM64OBJ num64_values
   SSTROBJ str_values
   INT16OBJ int16 int16_values
   INT32OBJ int16 int32_values
   NUM64OBJ int16 num64_values
   STROBJ int16 str_values

object_values ::=
   int16 binaryjson values*
   int16 INT16 values*
   int16 INT32 values*
   int16 NUM64 values*
   int16 STR values*
   int16 TRUE values*
   int16 FALSE values*
   int16 NULL values*

int16_values ::=
   int16 int16 int16_array*

int32_values ::=
   int16 int32 int32_array*

num64_values ::=
   int16 num64 num64_array*

str_values ::=
   int16 int16 str_array*

array ::=
   binaryjson values*
   INT16 values*
   INT32 values*
   NUM64 values*
   STR values*
   TRUE values*
   FALSE values*
   NULL values*

int16_array ::=
   int16 int16_array*

int32_array ::=
   int32 int32_array*

num64_array ::=
   num64 num64_array*

str_array ::=
   int16 str_array*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment