Skip to content

Instantly share code, notes, and snippets.

@r39132
Created May 18, 2016 18:04
Show Gist options
  • Star 5 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save r39132/f94484efa68a8ca4d2d23a4260436c7a to your computer and use it in GitHub Desktop.
Save r39132/f94484efa68a8ca4d2d23a4260436c7a to your computer and use it in GitHub Desktop.
Writing Avro Without A Schema in Python
import avro.schema
import io, random
from avro.io import DatumWriter, DatumReader
import avro.io
# Path to user.avsc avro schema
schema_path="user.avsc"
schema = avro.schema.parse(open(schema_path).read())
for i in xrange(1):
writer = avro.io.DatumWriter(schema)
bytes_writer = io.BytesIO()
encoder = avro.io.BinaryEncoder(bytes_writer)
writer.write({"name": "123", "favorite_color": "111", "favorite_number": random.randint(0,10)}, encoder)
raw_bytes = bytes_writer.getvalue()
print(raw_bytes)
bytes_reader = io.BytesIO(raw_bytes)
decoder = avro.io.BinaryDecoder(bytes_reader)
reader = avro.io.DatumReader(schema)
user1 = reader.read(decoder)
print(" USER = {}".format(user1))
{"namespace": "example.avro",
"type": "record",
"name": "User",
"fields": [
{"name": "name", "type": "string"},
{"name": "favorite_number", "type": ["int", "null"]},
{"name": "favorite_color", "type": ["string", "null"]}
]
}
@r39132
Copy link
Author

r39132 commented May 18, 2016

The output is

123111
USER = {u'favorite_color': u'111', u'favorite_number': 1, u'name': u'123'}

@tonicebrian
Copy link

I think that line 8 with latests versions of Avro should be written as:

schema = avro.schema.Parse(open(schema_path).read())

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment