Skip to content

Instantly share code, notes, and snippets.

@jdcryans
jdcryans / gist:7997586
Created December 17, 2013 00:07
Adding simple scanner prefetching
diff --git a/src/Scanner.java b/src/Scanner.java
index 7345d34..4ab54b3 100644
--- a/src/Scanner.java
+++ b/src/Scanner.java
@@ -187,6 +187,10 @@ public final class Scanner {
*/
private long scanner_id;
+ private boolean prefetching;
+
@jdcryans
jdcryans / gist:9556663
Created March 14, 2014 20:53
creating a new hbase table based on table in another cluster with the same name
// using a config that's against the old cluster
HBaseAdmin admin = new HBaseAdmin(HBaseConfiguration.create());
byte[] tableName = Bytes.toBytes("tsdb");
HTableDescriptor htd = admin.getTableDescriptor(tableName);
List<HRegionInfo> regions = admin.getTableRegions(tableName);
regions.remove(0); // removing the first region, it has an empty start key
byte[][] startKeys = new byte[regions.size()][];
for (int i = 0; i < regions.size(); i++) {
startKeys[i] = regions.get(i).getStartKey();
@jdcryans
jdcryans / gist:6722492f35edb5302d1b
Created October 30, 2015 17:23
tserver deadlock MM vs DeleteTablet
Thread 418 (Thread 0x7fa9eb29c700 (LWP 13546)):
#0 0x00000035d180e264 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00000035d1809508 in _L_lock_854 () from /lib64/libpthread.so.0
#2 0x00000035d18093d7 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x0000000000876845 in kudu::tablet::TabletMetadata::Flush() ()
#4 0x00000000008794b2 in kudu::tablet::TabletMetadata::UpdateAndFlush(std::tr1::unordered_set<long, std::tr1::hash<long>, std::equal_to<long>, std::allocator<long> > const&, std::vector<std::tr1::shared_ptr<kudu::tablet::RowSetMetadata>, std::allocator<std::tr1::shared_ptr<kudu::tablet::RowSetMetadata> > > const&, long) ()
#5 0x0000000000804325 in kudu::tablet::Tablet::FlushMetadata(std::vector<std::tr1::shared_ptr<kudu::tablet::RowSet>, std::allocator<std::tr1::shared_ptr<kudu::tablet::RowSet> > > const&, std::vector<std::tr1::shared_ptr<kudu::tablet::RowSetMetadata>, std::allocator<std::tr1::shared_ptr<kudu::tablet::RowSetMetadata> > > const&, long) ()
#6 0x000000000080a49f in
Thread 418 (Thread 0x7fa9eb29c700 (LWP 13546)):
#0 0x00000035d180e264 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00000035d1809508 in _L_lock_854 () from /lib64/libpthread.so.0
#2 0x00000035d18093d7 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x0000000000876845 in kudu::tablet::TabletMetadata::Flush() ()
#4 0x00000000008794b2 in kudu::tablet::TabletMetadata::UpdateAndFlush(std::tr1::unordered_set<long, std::tr1::hash<long>, std::equal_to<long>, std::allocator<long> > const&, std::vector<std::tr1::shared_ptr<kudu::tablet::RowSetMetadata>, std::allocator<std::tr1::shared_ptr<kudu::tablet::RowSetMetadata> > > const&, long) ()
#5 0x0000000000804325 in kudu::tablet::Tablet::FlushMetadata(std::vector<std::tr1::shared_ptr<kudu::tablet::RowSet>, std::allocator<std::tr1::shared_ptr<kudu::tablet::RowSet> > > const&, std::vector<std::tr1::shared_ptr<kudu::tablet::RowSetMetadata>, std::allocator<std::tr1::shared_ptr<kudu::tablet::RowSetMetadata> > > const&, long) ()
#6 0x000000000080a49f in
I1102 22:55:10.698065 24069 ts_tablet_manager.cc:726] Registered tablet 1e29b40ce0384e6bb09e4fc09a9ed409
I1102 22:55:10.742638 31857 ts_tablet_manager.cc:599] T 1e29b40ce0384e6bb09e4fc09a9ed409 P 7746b29074c64e56a03dd6c5161fb929: Bootstrapping tablet
I1102 22:55:10.743032 31857 tablet_bootstrap.cc:357] T 1e29b40ce0384e6bb09e4fc09a9ed409 P 7746b29074c64e56a03dd6c5161fb929: Bootstrap starting.
I1102 22:55:10.743201 31857 tablet_bootstrap.cc:520] T 1e29b40ce0384e6bb09e4fc09a9ed409 P 7746b29074c64e56a03dd6c5161fb929: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s
I1102 22:55:10.781862 31857 tablet_bootstrap.cc:460] T 1e29b40ce0384e6bb09e4fc09a9ed409 P 7746b29074c64e56a03dd6c5161fb929: No blocks or log segments found. Creating new log.
I1102 22:55:10.835072 31857 tablet_bootstrap.cc:357] T 1e29b40ce0384e6bb09e4fc09a9ed409 P 7746b29074c64e56a03dd6c5161fb929: No bootstrap required, opened a new log
I1102 22:55:10.835100 31857 ts_tablet_manager.cc:604] T 1e29b40ce0384e6bb09e4fc09a9ed409 P 7746b29074c64
@jdcryans
jdcryans / gist:1df3366b85c8c878ccb766b3caced9b4
Created October 28, 2016 17:27
YCSB perf with and without tracing
Before tracing:
Run #1
[OVERALL], RunTime(ms), 274163.0
[OVERALL], Throughput(ops/sec), 54711.97791095079
[TOTAL_GCS_PS_Scavenge], Count, 2174.0
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 2689.0
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.9808033906836443
[TOTAL_GCS_PS_MarkSweep], Count, 0.0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0.0