Skip to content

Instantly share code, notes, and snippets.

@rdblue
rdblue / TestProvidedConfiguration.java
Created September 12, 2014 22:53
Tests that hdfs-site.xml is correctly loaded in the master branch.
/*
* Copyright 2013 Cloudera Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
@rdblue
rdblue / gist:ca14f2a469f36d7f012c
Created May 17, 2014 00:35
Movie schema with column mappings.
{
"type" : "record",
"name" : "Movie",
"doc" : "Schema generated by Kite",
"fields" : [ {
"name" : "id",
"type" : "long"
}, {
"name" : "title",
"type" : [ "null", "string" ]
@rdblue
rdblue / TestAvroSchemaConverter.java
Last active August 29, 2015 13:56
Test to catch parquet-mr #12
@Test
public void testArrayOfRecords() throws Exception {
Schema innerRecord = Schema.createRecord("InnerRecord", null, null, false);
Schema optionalString = optional(Schema.create(Schema.Type.STRING));
innerRecord.setFields(Lists.newArrayList(
new Schema.Field("s1", optionalString, null, NullNode.getInstance()),
new Schema.Field("s2", optionalString, null, NullNode.getInstance())
));
Schema schema = Schema.createRecord("HasArray", null, null, false);
schema.setFields(Lists.newArrayList(