This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
protected long uploadGraph(ErgGraph graph) throws IOException { | |
try { | |
InputStream stream = ErgSerializer.serializeErgGraph(graph); | |
return filesClient.uploadAsFile(stream, graph.getErgID() + "", null).get().getFileId(); | |
} catch (Exception e) { | |
throw new IOException("Unable to cache ErgGraph"); | |
} | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
public static FilesClient getDefaultClient() { | |
return new FilesClient(new AppClient.Credentials() { | |
@Override public long getUserId() throws IOException {return 123L;} | |
@Override public byte[] getKey() throws IOException {throw new IOException("NOT IMPLEMENTED");} | |
}, | |
System.getProperty(Broker.PUBLIC_HOST_PROPERTY, Broker.DEFAULT_PUBLIC_HOST), | |
DagUtil.getIntProperty(Broker.PUBLIC_PORT_PROPERTY, Broker.DEFAULT_PUBLIC_PORT)); | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# | |
#/** | |
# * Copyright 2007 The Apache Software Foundation | |
# * | |
# * Licensed to the Apache Software Foundation (ASF) under one | |
# * or more contributor license agreements. See the NOTICE file | |
# * distributed with this work for additional information | |
# * regarding copyright ownership. The ASF licenses this file | |
# * to you under the Apache License, Version 2.0 (the | |
# * "License"); you may not use this file except in compliance |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<?xml version="1.0"?> | |
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?> | |
<!-- | |
/** | |
* Copyright 2010 The Apache Software Foundation | |
* | |
* Licensed to the Apache Software Foundation (ASF) under one | |
* or more contributor license agreements. See the NOTICE file | |
* distributed with this work for additional information | |
* regarding copyright ownership. The ASF licenses this file |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
public static void oldMakeTable(HTable table, int numRows, int numCols) throws IOException { | |
table.setAutoFlush(false); | |
Random random = new Random(); | |
byte[] family = Bytes.toBytes("family"); | |
for (int i = 0; i < numRows; i++) { | |
byte[] rowBytes = Bytes.toBytes(i); | |
for (int j = 0 ; j < numCols; j++) { | |
table.put(new Put(rowBytes).add(family, Bytes.toBytes(j), Bytes.toBytes(random.nextDouble()))); | |
} | |
System.out.println(String.format("Done ingesting %d rows", i)); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
time = System.currentTimeMillis(); | |
oldMakeTable(table, numRows, numCols); | |
System.out.println(String.format("Ingested in %.2g seconds", (System.currentTimeMillis() - time)/1000.0)); | |
time = System.currentTimeMillis(); | |
oldScanTable(table, numRows, numCols); | |
System.out.println(String.format("Scanned in %.2g seconds", (System.currentTimeMillis() - time)/1000.0)); | |
time = System.currentTimeMillis(); | |
newMakeTable(table, numRows, numCols, segmentSize); | |
System.out.println(String.format("Ingested in %.2g seconds", (System.currentTimeMillis() - time)/1000.0)); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@Test | |
public void InstalledFilterTest() throws IOException, NoSuchUserException { | |
Filter filter = new CompactedRowSingleColumnValueFilter( | |
translator.getFamily(), translator.getQualifier(colSet.getColumn(1)), | |
CompareOp.EQUAL, | |
new LongWritableComparable(2), 1); | |
final List<Long> rowIndices = new ArrayList<Long>(); | |
TableMapFn mapFn = new TableMapFn() { |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
public class changeme extends TableMapFn { | |
HTable table; | |
TableTranslator translator; | |
Map<Integer, Tables.RowSegment.Builder> rowSegments; | |
public changeme(HTable table, TableTranslator translator) { | |
this.table = table; | |
this.translator = translator; | |
} |