Skip to content

Instantly share code, notes, and snippets.

@DNNX
Created November 16, 2011 11:06
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save DNNX/1369840 to your computer and use it in GitHub Desktop.
Save DNNX/1369840 to your computer and use it in GitHub Desktop.
How to test Hadoop Mappers with and without MRUnit
@Test
public void shouldMapDocument() throws IOException, InterruptedException {
new MapDriver(new DocumentMapper())
.withInput(createDocumentForTest())
.withOutput(expected0)
.withOutput(expected1)
.runTest();
}
@Test
public void shouldMapDocument() throws IOException, InterruptedException {
final List<Symbol> mapOutput = new ArrayList<Symbol>(); // this is a closure
Document input = createDocumentForTest();
DocumentMapperTest.Context context = new Context(
new Configuration(),
new TaskAttemptID(),
null, // <- oh, nulls. holy boilerplate!
null,
null,
null,
null
) {
public void write(Text key, Symbol symbol) // look! I'm a wannabe-javascript function.
{
mapOutput .add(symbol); // <- here we use this closure
}
};
DocumentMapperTest mapper = new DocumentMapperTest();
mapper.map(new Text("Q123"), input, context);
assertEquals(2, mapOutput .size());
assertEquals(expected0, mapOutput.get(0));
assertEquals(expected1, mapOutput.get(1));
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment