Skip to content

Instantly share code, notes, and snippets.

Created June 19, 2013 17:32
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save anonymous/5816180 to your computer and use it in GitHub Desktop.
Save anonymous/5816180 to your computer and use it in GitHub Desktop.
//Acknowledgement: Comparator code borrowed from https://github.com/nt/bulkload-tests/blob/master/src/test/java/ProgrammaticHFileGeneration.java
static Comparator<byte[]> byteArrComparator = new Comparator<byte[]>() {
public int compare(byte[] o1, byte[] o2) {
return Bytes.compareTo(o1, 0, o1.length, o2, 0, o2.length);
}
};
//create a writer
FileSystem hdfs = FileSystem.get(conf);
Path path = new Path(outputHFile);//TODO: add appropriate dfs path
HFile.WriterFactory fac = HFile.getWriterFactoryNoCache(conf);
Writer writer = fac.withPath(hdfs, path).create();
//write values to writer
byte[] rowBytes = Bytes.toBytes("10011-2-0000000000000000703");
Map<byte[], byte[]> keyValuePairs = new TreeMap<byte[], byte[]>(byteArrComparator);
keyValuePairs.put(Bytes.toBytes("dt"), Bytes.toBytes("dummy value 1"));
keyValuePairs.put(Bytes.toBytes("dth"), Bytes.toBytes("dummy value 2"));
for (byte[] key : keyValuePairs.keySet()){
KeyValue kv = new KeyValue(rowBytes, HbaseConstants.staticDataCF, key, keyValuePairs.get(key));
writer.append(kv);
}
//add fileinfo and close writer
writer.appendFileInfo(StoreFile.BULKLOAD_TIME_KEY, Bytes.toBytes(System.currentTimeMillis()));
writer.appendFileInfo(StoreFile.MAJOR_COMPACTION_KEY, Bytes.toBytes(true));
writer.close();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment