Skip to content

Instantly share code, notes, and snippets.

@noiano
Created February 3, 2011 17:18
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save noiano/809803 to your computer and use it in GitHub Desktop.
Save noiano/809803 to your computer and use it in GitHub Desktop.
GroupingComparator
public class GroupingComparator implements RawComparator<PairOfStringInt>{
private static final Text.Comparator TEXT_COMPARATOR = new Text.Comparator();
@Override
public int compare(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2) {
return TEXT_COMPARATOR.compare(b1, s1, l1, b2, s2, l2);
/*
int firstL1;;
int firstL2;
try{
firstL1 = WritableUtils.decodeVIntSize(b1[s1]) + WritableComparator.readVInt(b1, s1);
firstL2 = WritableUtils.decodeVIntSize(b2[s2]) + WritableComparator.readVInt(b2, s2);
return TEXT_COMPARATOR.compare(b1, s1, firstL1, b2, s2, firstL2);
}
catch (IOException e) {
throw new RuntimeException(e);
}
*/
}
@Override
public int compare(PairOfStringInt p1, PairOfStringInt p2) {
return p1.getLeftElement().compareTo(p2.getLeftElement());
}
@QwertyManiac
Copy link

Your value grouping comparator seems right for the apples example you gave on the ML, but you need your Partitioner to support it by looking up on the same part of your composite key.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment