Skip to content

Instantly share code, notes, and snippets.

@jpe42
Last active June 2, 2016 18:35
Show Gist options
  • Save jpe42/d03482c4b3515f78d1dbf9d7b092711e to your computer and use it in GitHub Desktop.
Save jpe42/d03482c4b3515f78d1dbf9d7b092711e to your computer and use it in GitHub Desktop.
Bulk REPLY OFF Loading w/ Jedipus
private static int runWithJedipus(final IntStream keys) {
int count = 0;
try (final RedisClient client =
RedisClientFactory.startBuilding().create(Node.create("127.0.0.1", 6379))) {
client.sendCmd(Cmds.FLUSHALL);
final long start = System.currentTimeMillis();
client.replyOff();
final byte[] defaultValue = RESP.toBytes("D");
count =
(int) keys.peek(key -> client.sendCmd(Cmds.SETNX, RESP.toBytes(key), defaultValue)).count();
client.replyOn();
System.out.format("Loading %d key/values took %dms.%n", count,
System.currentTimeMillis() - start);
final long dbsize = client.sendCmd(Cmds.DBSIZE.prim());
if (dbsize != count) {
throw new IllegalStateException(
String.format("Expected %d keys, but %d exist.", count, dbsize));
}
}
return count;
}
private static int runWithJedis(final IntStream keys) {
final int[] cnt = {0};
try (final Jedis jedis = new Jedis("127.0.0.1", 6379)) {
jedis.flushAll();
final long start = System.currentTimeMillis();
final byte[] defaultValue = RESP.toBytes("D");
final Pipeline pipeline = jedis.pipelined();
keys.forEach(key -> {
if (++cnt[0] % 1_000 == 0) {
pipeline.sync();
}
pipeline.setnx(RESP.toBytes(key), defaultValue);
});
pipeline.sync();
System.out.format("Loading %d key/values took %dms.%n", cnt[0],
System.currentTimeMillis() - start);
final long dbsize = jedis.dbSize();
if (dbsize != cnt[0]) {
throw new IllegalStateException(
String.format("Expected %d keys, but %d exist.", cnt[0], dbsize));
}
}
return cnt[0];
}
@jpe42
Copy link
Author

jpe42 commented May 27, 2016

@dgomesbr, I've tuned the output buffer size in the latest release. For the examples in this gist I'm seeing a 15-20% improvement over Jedis. The improvement will be even greater when accessing a remote Redis server.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment