Skip to content

Instantly share code, notes, and snippets.

@rewritten
Created December 5, 2023 17:25
Show Gist options
  • Save rewritten/2533573332d1de1e4b568def9c757c42 to your computer and use it in GitHub Desktop.
Save rewritten/2533573332d1de1e4b568def9c757c42 to your computer and use it in GitHub Desktop.
Proof of concept for the unnamed types issues in Avrora
Mix.install([:erlavro, :avrora, :jason])
record = %{"id" => 1, "name" => "test"}
record_schema = """
{
"type": "record",
"name": "TestRecord",
"fields": [
{"name": "id", "type": "int"},
{"name": "name", "type": "string"}
]
}
"""
union_schema = ~s(["null",#{record_schema}])
basic_schema = ~s("string")
# check that all schemas work with erlavro
# record:
decoder = :avro.make_simple_decoder(record_schema, record_type: :map)
encoder = :avro.make_simple_encoder(record_schema, [])
<<2, 8, 116, 101, 115, 116>> = IO.chardata_to_string(encoder.(record))
^record = decoder.(<<2, 8, 116, 101, 115, 116>>)
# union:
decoder = :avro.make_simple_decoder(union_schema, record_type: :map)
encoder = :avro.make_simple_encoder(union_schema, [])
<<2, 2, 8, 116, 101, 115, 116>> = IO.chardata_to_string(encoder.(record))
^record = decoder.(<<2, 2, 8, 116, 101, 115, 116>>)
<<0>> = IO.chardata_to_string(encoder.(nil))
:null = decoder.(<<0>>)
# basic:
decoder = :avro.make_simple_decoder(basic_schema, record_type: :map)
encoder = :avro.make_simple_encoder(basic_schema, [])
<<8, 116, 101, 115, 116>> = IO.chardata_to_string(encoder.("test"))
"test" = decoder.(<<8, 116, 101, 115, 116>>)
# check that all schemas work with avrora
# Start avrora and store the schemas in Avrora.Storage.Memory
Avrora.start_link()
### normal record works correctly
{:ok, avrora_record} = Avrora.Schema.Encoder.from_json(record_schema, %{})
Avrora.Storage.Memory.put("TestRecord", %{avrora_record | id: 1})
Avrora.Storage.Memory.put(1, %{avrora_record | id: 1})
# notice one header byte <<0>> and one 4-byte id <<0, 0, 0, 1>>
{:ok, <<0, 0, 0, 0, 1, 2, 8, 116, 101, 115, 116>>} =
Avrora.encode(record, schema_name: "TestRecord", format: :registry)
{:ok, %{"id" => 1, "name" => "test"}} = Avrora.decode(<<0, 0, 0, 0, 1, 2, 8, 116, 101, 115, 116>>)
### union does not work
{:error, :unnamed_type} = Avrora.Schema.Encoder.from_json(union_schema, %{})
### basic type does not work
{:error, :unnamed_type} = Avrora.Schema.Encoder.from_json(basic_schema, %{})
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment