Skip to content

Instantly share code, notes, and snippets.

@mustafaakin
Last active February 27, 2018 10:35
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mustafaakin/e627fbebb364bcfe061f0cc89076bf53 to your computer and use it in GitHub Desktop.
Save mustafaakin/e627fbebb364bcfe061f0cc89076bf53 to your computer and use it in GitHub Desktop.
SchemaBuilder.RecordBuilder<Schema> builder = SchemaBuilder
.record("flow_logs").namespace("com.opsgenie");
schema = builder.fields()
.name("version").type().intType().intDefault(0)
.name("account").type().stringType().stringDefault("")
.name("interfaceId").type().stringType().stringDefault("")
.name("sourceAddress").type().stringType().stringDefault("")
// continues for remaining 10 fields..
ParquetWriter<FlowLogs> writer = AvroParquetWriter.<FlowLog>builder(p)
.withSchema(schema)
.withConf(conf)
.withCompressionCodec(CompressionCodecName.SNAPPY)
.withWriteMode(Mode.OVERWRITE)
.build();
GenericData.Record record = new GenericData.Record(schema);
record.put("version", log.getVersion());
record.put("account", log.getAccount());
record.put("interfaceId", log.getInterfaceId());
record.put("sourceAddress", log.getSourceAddress());
// continues for remaining 10 fields..
writer.write(record)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment