Skip to content

Instantly share code, notes, and snippets.

@abramsm
Created April 29, 2013 19:59
Show Gist options
  • Save abramsm/5484296 to your computer and use it in GitHub Desktop.
Save abramsm/5484296 to your computer and use it in GitHub Desktop.
hll+ example with merge
for (int p = 10; p < 18; p++)
{
ICardinality[] hlls = new ICardinality[30];
int ecount = 15000;
int totalCount = ecount * 30;
for (int j = 0; j < 30; j++)
{
hlls[j] = new HyperLogLogPlus(p, 25);
for (int i = 0; i < ecount; i++)
{
hlls[j].offer(UUID.randomUUID());
}
}
HyperLogLogPlus total = new HyperLogLogPlus(p, 25);
long estimate = total.merge(hlls).cardinality();
double se = totalCount * (1.04 / Math.sqrt(Math.pow(2, p)));
long expectedCardinality = totalCount;
double error = (totalCount - estimate) / (double) totalCount;
System.out.println("Expect estimate for p=" + p + " is " + estimate + " with error: " + error + " is between " + (expectedCardinality - (3 * se)) + " and " + (expectedCardinality + (3 * se)) + " mem:" + total.getBytes().length);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment