Skip to content

Instantly share code, notes, and snippets.

@lazyval
Created June 23, 2016 08:42
Show Gist options
  • Save lazyval/55a60dc1d77d5fb70b4d2e373296962e to your computer and use it in GitHub Desktop.
Save lazyval/55a60dc1d77d5fb70b4d2e373296962e to your computer and use it in GitHub Desktop.
Map Reduce 101
/**
* Collects results of map phase, persists it to temporary storage
* results with same key (globally) can be accessed together
*/
interface Emitter<M> {
void emit(String key, M result);
}
import java.util.Iterator;
/**
* Runs locally on each file
*/
interface Mapper<M> {
void map(Iterator<String> lines, Emmiter<M> output);
}
import java.util.Iterator;
/**
* Runs on <some> host, one reducer for each group (group is defined by key at map phase), group is assembled from
* results from all hosts
*/
interface Reducer<R> {
R reduce(Iterator<String> lines, Emmiter<M> output);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment