Skip to content

Instantly share code, notes, and snippets.

@brfrn169
Last active September 12, 2015 14:12
Show Gist options
  • Save brfrn169/cb4f2c157129330cd932 to your computer and use it in GitHub Desktop.
Save brfrn169/cb4f2c157129330cd932 to your computer and use it in GitHub Desktop.
Performance degradation between CDH5.3.1(HBase0.98.6) and CDH5.4.5(HBase1.0.0)
RegionServer status for slave5.example.local,60020,1441184528814 as of Wed Sep 02 19:06:22 JST 2015
Version Info:
===========================================================
HBase 1.0.0-cdh5.4.5
Source code repository file:///data/jenkins/workspace/generic-package-rhel64-6-0/topdir/BUILD/hbase-1.0.0-cdh5.4.5 revision=Unknown
Compiled by jenkins on Wed Aug 12 14:17:02 PDT 2015
From source with checksum 83861a8328fb22c0999b0559c683e396
Hadoop 2.6.0-cdh5.4.5
Source code repository http://github.com/cloudera/hadoop revision=ab14c89fe25e9fb3f9de4fb852c21365b7c5608b
Compiled by jenkins on 2015-08-12T21:12Z
Tasks:
===========================================================
Task: RpcServer.reader=1,bindAddress=0.0.0.0,port=60020
Status: WAITING:Waiting for a call
Running for 3809s
Task: RpcServer.reader=3,bindAddress=0.0.0.0,port=60020
Status: RUNNING:Servicing call from 127.0.0.83:35271: Get
Running for 3809s
Task: RpcServer.reader=4,bindAddress=0.0.0.0,port=60020
Status: WAITING:Waiting for a call
Running for 3809s
Task: RpcServer.reader=2,bindAddress=0.0.0.0,port=60020
Status: WAITING:Waiting for a call
Running for 3809s
Task: RpcServer.reader=5,bindAddress=0.0.0.0,port=60020
Status: WAITING:Waiting for a call
Running for 3809s
Task: RpcServer.reader=6,bindAddress=0.0.0.0,port=60020
Status: RUNNING:Servicing call from 127.0.0.145:56432: Get
Running for 3809s
Task: RpcServer.reader=7,bindAddress=0.0.0.0,port=60020
Status: WAITING:Waiting for a call
Running for 3809s
Task: RpcServer.reader=8,bindAddress=0.0.0.0,port=60020
Status: WAITING:Waiting for a call
Running for 3809s
Task: RpcServer.reader=0,bindAddress=0.0.0.0,port=60020
Status: WAITING:Waiting for a call
Running for 3809s
Task: RpcServer.reader=9,bindAddress=0.0.0.0,port=60020
Status: WAITING:Waiting for a call
Running for 3809s
Executors:
===========================================================
Status for executor: Executor-4-RS_CLOSE_META-slave5:60020
=======================================
0 events queued, 0 running
Status for executor: Executor-2-RS_OPEN_META-slave5:60020
=======================================
0 events queued, 0 running
Status for executor: Executor-3-RS_CLOSE_REGION-slave5:60020
=======================================
0 events queued, 0 running
Status for executor: Executor-1-RS_OPEN_REGION-slave5:60020
=======================================
0 events queued, 0 running
Status for executor: Executor-5-RS_LOG_REPLAY_OPS-slave5:60020
=======================================
0 events queued, 0 running
Process Thread Dump:
403 active threads
Thread 14265 (hedgedRead-139):
State: TIMED_WAITING
Blocked count: 0
Waited count: 1003
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 14168 (ResponseProcessor for block BP-664460170-127.0.0.35-1423743007662:blk_1074330235_589828):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)
org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:118)
java.io.FilterInputStream.read(FilterInputStream.java:83)
java.io.FilterInputStream.read(FilterInputStream.java:83)
org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2237)
org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:176)
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer$ResponseProcessor.run(DFSOutputStream.java:879)
Thread 13660 (RMI TCP Connection(idle)):
State: TIMED_WAITING
Blocked count: 0
Waited count: 4
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 13659 (hedgedRead-138):
State: TIMED_WAITING
Blocked count: 0
Waited count: 7130
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 13625 (1199421195@qtp-1113829504-25):
State: RUNNABLE
Blocked count: 32
Waited count: 32
Stack:
sun.management.ThreadImpl.getThreadInfo1(Native Method)
sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:174)
sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:139)
org.apache.hadoop.util.ReflectionUtils.printThreadInfo(ReflectionUtils.java:165)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:497)
org.apache.hadoop.hbase.util.Threads.printThreadInfo(Threads.java:302)
org.apache.hadoop.hbase.regionserver.RSDumpServlet.doGet(RSDumpServlet.java:79)
javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:113)
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1351)
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:49)
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
Thread 13536 (934384282@qtp-1113829504-23):
State: TIMED_WAITING
Blocked count: 29
Waited count: 31
Stack:
java.lang.Object.wait(Native Method)
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
Thread 13446 (DataStreamer for file /hbase/WALs/slave5.example.local,60020,1441184528814/slave5.example.local%2C60020%2C1441184528814.null0.1441188168410 block BP-664460170-127.0.0.35-1423743007662:blk_1074330235_589828):
State: TIMED_WAITING
Blocked count: 5010
Waited count: 5013
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:576)
Thread 13432 (hedgedRead-137):
State: TIMED_WAITING
Blocked count: 1
Waited count: 8190
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 13166 (hedgedRead-136):
State: TIMED_WAITING
Blocked count: 0
Waited count: 11726
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 12820 (RMI TCP Connection(281)-127.0.0.36):
State: RUNNABLE
Blocked count: 0
Waited count: 12
Stack:
java.net.SocketInputStream.socketRead0(Native Method)
java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
java.net.SocketInputStream.read(SocketInputStream.java:170)
java.net.SocketInputStream.read(SocketInputStream.java:141)
java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
java.io.BufferedInputStream.read(BufferedInputStream.java:265)
java.io.FilterInputStream.read(FilterInputStream.java:83)
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:550)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$79(TCPTransport.java:683)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$9/962345647.run(Unknown Source)
java.security.AccessController.doPrivileged(Native Method)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 11667 (hedgedRead-131):
State: TIMED_WAITING
Blocked count: 1
Waited count: 24678
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 11411 (hedgedRead-129):
State: TIMED_WAITING
Blocked count: 7
Waited count: 37105
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 11069 (hedgedRead-125):
State: TIMED_WAITING
Blocked count: 10
Waited count: 45934
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 9963 (RMI TCP Connection(283)-127.0.0.36):
State: RUNNABLE
Blocked count: 0
Waited count: 32
Stack:
java.net.SocketInputStream.socketRead0(Native Method)
java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
java.net.SocketInputStream.read(SocketInputStream.java:170)
java.net.SocketInputStream.read(SocketInputStream.java:141)
java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
java.io.BufferedInputStream.read(BufferedInputStream.java:265)
java.io.FilterInputStream.read(FilterInputStream.java:83)
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:550)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$79(TCPTransport.java:683)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$9/962345647.run(Unknown Source)
java.security.AccessController.doPrivileged(Native Method)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 3203 (IPC Parameter Sending Thread #2):
State: TIMED_WAITING
Blocked count: 9
Waited count: 34668
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 613 (ShortCircuitCache_SlotReleaser):
State: WAITING
Blocked count: 2
Waited count: 34575
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@59c73767
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 602 (IPC Client (1241480588) connection to slave2.example.local/127.0.0.216:60020 from hbase):
State: TIMED_WAITING
Blocked count: 2453
Waited count: 5929
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.waitForWork(RpcClientImpl.java:538)
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.run(RpcClientImpl.java:553)
Thread 569 (IPC Client (1241480588) connection to slave1.example.local/127.0.0.214:60020 from hbase):
State: TIMED_WAITING
Blocked count: 2535
Waited count: 6016
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.waitForWork(RpcClientImpl.java:538)
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.run(RpcClientImpl.java:553)
Thread 562 (RS_OPEN_REGION-slave5:60020-2):
State: WAITING
Blocked count: 44
Waited count: 73
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6a1d69ba
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 559 (RMI Scheduler(0)):
State: TIMED_WAITING
Blocked count: 0
Waited count: 245
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 556 (IPC Client (1241480588) connection to slave4.example.local/127.0.0.217:60020 from hbase):
State: TIMED_WAITING
Blocked count: 2518
Waited count: 6000
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.waitForWork(RpcClientImpl.java:538)
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.run(RpcClientImpl.java:553)
Thread 547 (RS_OPEN_REGION-slave5:60020-1):
State: WAITING
Blocked count: 68
Waited count: 111
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6a1d69ba
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 545 (IPC Client (1241480588) connection to slave3.example.local/127.0.0.215:60020 from hbase):
State: TIMED_WAITING
Blocked count: 2437
Waited count: 5920
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.waitForWork(RpcClientImpl.java:538)
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.run(RpcClientImpl.java:553)
Thread 463 (regionserver/slave5.example.local/127.0.0.221:60020-shortCompactions-1441184573936):
State: WAITING
Blocked count: 379
Waited count: 615
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@42dece99
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 459 (ShortCircuitCache_Cleaner):
State: TIMED_WAITING
Blocked count: 0
Waited count: 51
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 455 (RS_OPEN_REGION-slave5:60020-0):
State: WAITING
Blocked count: 66
Waited count: 110
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6a1d69ba
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 413 (org.apache.hadoop.hdfs.PeerCache@7807e29c):
State: TIMED_WAITING
Blocked count: 0
Waited count: 1283
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:255)
org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46)
org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124)
java.lang.Thread.run(Thread.java:745)
Thread 411 (slave5.example.local,60020,1441184528814-HeapMemoryTunerChore):
State: TIMED_WAITING
Blocked count: 0
Waited count: 65
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:92)
org.apache.hadoop.hbase.Chore.run(Chore.java:95)
java.lang.Thread.run(Thread.java:745)
Thread 410 (SplitLogWorker-slave5:60020):
State: TIMED_WAITING
Blocked count: 2
Waited count: 773
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.coordination.ZkSplitLogWorkerCoordination.taskLoop(ZkSplitLogWorkerCoordination.java:444)
org.apache.hadoop.hbase.regionserver.SplitLogWorker.run(SplitLogWorker.java:142)
java.lang.Thread.run(Thread.java:745)
Thread 409 (slave5:60020Replication Statistics #0):
State: TIMED_WAITING
Blocked count: 0
Waited count: 13
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 407 (regionserver/slave5.example.local/127.0.0.221:60020-EventThread):
State: WAITING
Blocked count: 0
Waited count: 3
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7efadb3a
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
Thread 406 (regionserver/slave5.example.local/127.0.0.221:60020-SendThread(master2.example.local:2181)):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
Thread 404 (regionserver/slave5.example.local/127.0.0.221:60020.replicationSource,1-EventThread):
State: WAITING
Blocked count: 0
Waited count: 3
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5b8e786
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
Thread 403 (regionserver/slave5.example.local/127.0.0.221:60020.replicationSource,1-SendThread(master2.example.local:2181)):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
Thread 399 (regionserver/slave5.example.local/127.0.0.221:60020.replicationSource,1):
State: TIMED_WAITING
Blocked count: 62181
Waited count: 106164
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.sleepForRetries(ReplicationSource.java:634)
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:404)
Thread 402 (regionserver/slave5.example.local/127.0.0.221:60020-EventThread):
State: WAITING
Blocked count: 0
Waited count: 3
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@21bcaca
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
Thread 401 (regionserver/slave5.example.local/127.0.0.221:60020-SendThread(master1.example.local:2181)):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
Thread 377 (regionserver/slave5.example.local/127.0.0.221:60020.leaseChecker):
State: TIMED_WAITING
Blocked count: 0
Waited count: 386
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.hbase.regionserver.Leases.run(Leases.java:94)
java.lang.Thread.run(Thread.java:745)
Thread 379 (regionserver/slave5.example.local/127.0.0.221:60020.nonceCleaner):
State: TIMED_WAITING
Blocked count: 0
Waited count: 11
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:92)
org.apache.hadoop.hbase.Chore.run(Chore.java:95)
java.lang.Thread.run(Thread.java:745)
Thread 376 (regionserver/slave5.example.local/127.0.0.221:60020.periodicFlusher):
State: TIMED_WAITING
Blocked count: 0
Waited count: 386
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:92)
org.apache.hadoop.hbase.Chore.run(Chore.java:95)
java.lang.Thread.run(Thread.java:745)
Thread 375 (regionserver/slave5.example.local/127.0.0.221:60020.compactionChecker):
State: TIMED_WAITING
Blocked count: 0
Waited count: 386
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:92)
org.apache.hadoop.hbase.Chore.run(Chore.java:95)
java.lang.Thread.run(Thread.java:745)
Thread 397 (MemStoreFlusher.1):
State: TIMED_WAITING
Blocked count: 99
Waited count: 576
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.DelayQueue.poll(DelayQueue.java:259)
java.util.concurrent.DelayQueue.poll(DelayQueue.java:70)
org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:209)
java.lang.Thread.run(Thread.java:745)
Thread 395 (MemStoreFlusher.0):
State: TIMED_WAITING
Blocked count: 99
Waited count: 575
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.DelayQueue.poll(DelayQueue.java:259)
java.util.concurrent.DelayQueue.poll(DelayQueue.java:70)
org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:209)
java.lang.Thread.run(Thread.java:745)
Thread 370 (regionserver/slave5.example.local/127.0.0.221:60020.logRoller):
State: TIMED_WAITING
Blocked count: 44
Waited count: 440
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.LogRoller.run(LogRoller.java:116)
java.lang.Thread.run(Thread.java:745)
Thread 394 (HBase-Metrics2-1):
State: TIMED_WAITING
Blocked count: 0
Waited count: 1443
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 392 (sync.4):
State: WAITING
Blocked count: 14
Waited count: 22
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4a121f0f
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:1311)
java.lang.Thread.run(Thread.java:745)
Thread 391 (sync.3):
State: WAITING
Blocked count: 15
Waited count: 24
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@72d599e7
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:1311)
java.lang.Thread.run(Thread.java:745)
Thread 390 (sync.2):
State: WAITING
Blocked count: 14
Waited count: 22
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@433df39b
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:1311)
java.lang.Thread.run(Thread.java:745)
Thread 389 (sync.1):
State: WAITING
Blocked count: 15
Waited count: 24
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6fd72182
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:1311)
java.lang.Thread.run(Thread.java:745)
Thread 388 (sync.0):
State: WAITING
Blocked count: 15
Waited count: 24
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@35e7b110
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:1311)
java.lang.Thread.run(Thread.java:745)
Thread 393 (regionserver/slave5.example.local/127.0.0.221:60020.append-pool1-t1):
State: WAITING
Blocked count: 1
Waited count: 5509648
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ba30cea
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
com.lmax.disruptor.BlockingWaitStrategy.waitFor(BlockingWaitStrategy.java:45)
com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:55)
com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:123)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 386 (LeaseRenewer:hbase@left):
State: TIMED_WAITING
Blocked count: 127
Waited count: 4098
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:438)
org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71)
org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:298)
java.lang.Thread.run(Thread.java:745)
Thread 384 (IPC Client (1241480588) connection to master1.example.local/127.0.0.35:8020 from hbase):
State: TIMED_WAITING
Blocked count: 29789
Waited count: 29772
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:920)
org.apache.hadoop.ipc.Client$Connection.run(Client.java:965)
Thread 381 (IPC Client (1241480588) connection to master2.example.local/127.0.0.37:60000 from hbase):
State: TIMED_WAITING
Blocked count: 967
Waited count: 4406
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.waitForWork(RpcClientImpl.java:538)
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.run(RpcClientImpl.java:553)
Thread 374 (JvmPauseMonitor):
State: TIMED_WAITING
Blocked count: 0
Waited count: 7670
Stack:
java.lang.Thread.sleep(Native Method)
org.apache.hadoop.hbase.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:159)
java.lang.Thread.run(Thread.java:745)
Thread 373 (regionserver/slave5.example.local/127.0.0.221:60020-EventThread):
State: WAITING
Blocked count: 0
Waited count: 3
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@59b7af78
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
Thread 372 (regionserver/slave5.example.local/127.0.0.221:60020-SendThread(master1.example.local:2181)):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
Thread 29 (regionserver/slave5.example.local/127.0.0.221:60020):
State: TIMED_WAITING
Blocked count: 1346
Waited count: 2630
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:92)
org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:56)
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:897)
java.lang.Thread.run(Thread.java:745)
Thread 369 (Timer-0):
State: TIMED_WAITING
Blocked count: 0
Waited count: 129
Stack:
java.lang.Object.wait(Native Method)
java.util.TimerThread.mainLoop(Timer.java:552)
java.util.TimerThread.run(Timer.java:505)
Thread 368 (100708535@qtp-1113829504-1 - Acceptor0 SelectChannelConnector@0.0.0.0:60030):
State: RUNNABLE
Blocked count: 81
Waited count: 1
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:511)
org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:193)
org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Thread 365 (ReplicationRpcServer.handler=2,queue=0,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cf1f2f0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 364 (ReplicationRpcServer.handler=1,queue=0,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cf1f2f0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 363 (ReplicationRpcServer.handler=0,queue=0,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cf1f2f0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 362 (PriorityRpcServer.handler=9,queue=1,port=60020):
State: WAITING
Blocked count: 3
Waited count: 7
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@78d95279
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 361 (PriorityRpcServer.handler=8,queue=0,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@dcbdf0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 360 (PriorityRpcServer.handler=7,queue=1,port=60020):
State: WAITING
Blocked count: 3
Waited count: 7
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@78d95279
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 359 (PriorityRpcServer.handler=6,queue=0,port=60020):
State: WAITING
Blocked count: 4
Waited count: 10
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@dcbdf0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 358 (PriorityRpcServer.handler=5,queue=1,port=60020):
State: WAITING
Blocked count: 3
Waited count: 7
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@78d95279
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 357 (PriorityRpcServer.handler=4,queue=0,port=60020):
State: WAITING
Blocked count: 3
Waited count: 9
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@dcbdf0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 356 (PriorityRpcServer.handler=3,queue=1,port=60020):
State: WAITING
Blocked count: 6
Waited count: 14
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@78d95279
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 355 (PriorityRpcServer.handler=2,queue=0,port=60020):
State: WAITING
Blocked count: 3
Waited count: 7
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@dcbdf0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 354 (PriorityRpcServer.handler=1,queue=1,port=60020):
State: WAITING
Blocked count: 6
Waited count: 13
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@78d95279
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 353 (PriorityRpcServer.handler=0,queue=0,port=60020):
State: WAITING
Blocked count: 3
Waited count: 7
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@dcbdf0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 352 (RW.default.scanRpcServer.handler=299,queue=29,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7dc9b0cb
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 351 (RW.default.scanRpcServer.handler=298,queue=28,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6e3cbbf0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 350 (RW.default.scanRpcServer.handler=297,queue=27,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5c4466d4
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 349 (RW.default.scanRpcServer.handler=296,queue=26,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@39126391
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 348 (RW.default.scanRpcServer.handler=295,queue=25,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@9edc390
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 347 (RW.default.scanRpcServer.handler=294,queue=24,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2850a66
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 346 (RW.default.scanRpcServer.handler=293,queue=23,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7d88be24
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 345 (RW.default.scanRpcServer.handler=292,queue=22,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@572c54ef
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 344 (RW.default.scanRpcServer.handler=291,queue=21,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@303afcd9
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 343 (RW.default.scanRpcServer.handler=290,queue=20,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1ac61682
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 342 (RW.default.scanRpcServer.handler=289,queue=29,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7dc9b0cb
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 341 (RW.default.scanRpcServer.handler=288,queue=28,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6e3cbbf0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 340 (RW.default.scanRpcServer.handler=287,queue=27,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5c4466d4
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 339 (RW.default.scanRpcServer.handler=286,queue=26,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@39126391
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 338 (RW.default.scanRpcServer.handler=285,queue=25,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@9edc390
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 337 (RW.default.scanRpcServer.handler=284,queue=24,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2850a66
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 336 (RW.default.scanRpcServer.handler=283,queue=23,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7d88be24
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 335 (RW.default.scanRpcServer.handler=282,queue=22,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@572c54ef
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 334 (RW.default.scanRpcServer.handler=281,queue=21,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@303afcd9
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 333 (RW.default.scanRpcServer.handler=280,queue=20,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1ac61682
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 332 (RW.default.scanRpcServer.handler=279,queue=29,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7dc9b0cb
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 331 (RW.default.scanRpcServer.handler=278,queue=28,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6e3cbbf0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 330 (RW.default.scanRpcServer.handler=277,queue=27,port=60020):
State: WAITING
Blocked count: 0
Waited count: 3
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5c4466d4
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 329 (RW.default.scanRpcServer.handler=276,queue=26,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@39126391
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 328 (RW.default.scanRpcServer.handler=275,queue=25,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@9edc390
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 327 (RW.default.scanRpcServer.handler=274,queue=24,port=60020):
State: WAITING
Blocked count: 0
Waited count: 3
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2850a66
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 326 (RW.default.scanRpcServer.handler=273,queue=23,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7d88be24
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 325 (RW.default.scanRpcServer.handler=272,queue=22,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@572c54ef
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 324 (RW.default.scanRpcServer.handler=271,queue=21,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@303afcd9
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 323 (RW.default.scanRpcServer.handler=270,queue=20,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1ac61682
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 322 (RW.default.scanRpcServer.handler=269,queue=29,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7dc9b0cb
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 321 (RW.default.scanRpcServer.handler=268,queue=28,port=60020):
State: WAITING
Blocked count: 0
Waited count: 3
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6e3cbbf0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 320 (RW.default.scanRpcServer.handler=267,queue=27,port=60020):
State: WAITING
Blocked count: 0
Waited count: 3
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5c4466d4
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 319 (RW.default.scanRpcServer.handler=266,queue=26,port=60020):
State: WAITING
Blocked count: 0
Waited count: 3
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@39126391
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 318 (RW.default.scanRpcServer.handler=265,queue=25,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@9edc390
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 317 (RW.default.scanRpcServer.handler=264,queue=24,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2850a66
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 316 (RW.default.scanRpcServer.handler=263,queue=23,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7d88be24
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 315 (RW.default.scanRpcServer.handler=262,queue=22,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@572c54ef
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 314 (RW.default.scanRpcServer.handler=261,queue=21,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@303afcd9
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 313 (RW.default.scanRpcServer.handler=260,queue=20,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1ac61682
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 312 (RW.default.scanRpcServer.handler=259,queue=29,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7dc9b0cb
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 311 (RW.default.scanRpcServer.handler=258,queue=28,port=60020):
State: WAITING
Blocked count: 0
Waited count: 3
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6e3cbbf0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 310 (RW.default.scanRpcServer.handler=257,queue=27,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5c4466d4
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 309 (RW.default.scanRpcServer.handler=256,queue=26,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@39126391
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 308 (RW.default.scanRpcServer.handler=255,queue=25,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@9edc390
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 307 (RW.default.scanRpcServer.handler=254,queue=24,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2850a66
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 306 (RW.default.scanRpcServer.handler=253,queue=23,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7d88be24
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 305 (RW.default.scanRpcServer.handler=252,queue=22,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@572c54ef
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 304 (RW.default.scanRpcServer.handler=251,queue=21,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@303afcd9
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 303 (RW.default.scanRpcServer.handler=250,queue=20,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1ac61682
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 302 (RW.default.scanRpcServer.handler=249,queue=29,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7dc9b0cb
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 301 (RW.default.scanRpcServer.handler=248,queue=28,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6e3cbbf0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 300 (RW.default.scanRpcServer.handler=247,queue=27,port=60020):
State: WAITING
Blocked count: 0
Waited count: 3
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5c4466d4
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 299 (RW.default.scanRpcServer.handler=246,queue=26,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@39126391
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 298 (RW.default.scanRpcServer.handler=245,queue=25,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@9edc390
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 297 (RW.default.scanRpcServer.handler=244,queue=24,port=60020):
State: WAITING
Blocked count: 0
Waited count: 3
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2850a66
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 296 (RW.default.scanRpcServer.handler=243,queue=23,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7d88be24
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 295 (RW.default.scanRpcServer.handler=242,queue=22,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@572c54ef
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 294 (RW.default.scanRpcServer.handler=241,queue=21,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@303afcd9
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 293 (RW.default.scanRpcServer.handler=240,queue=20,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1ac61682
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 292 (RW.default.scanRpcServer.handler=239,queue=29,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7dc9b0cb
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 291 (RW.default.scanRpcServer.handler=238,queue=28,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6e3cbbf0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 290 (RW.default.scanRpcServer.handler=237,queue=27,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5c4466d4
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 289 (RW.default.scanRpcServer.handler=236,queue=26,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@39126391
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 288 (RW.default.scanRpcServer.handler=235,queue=25,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@9edc390
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 287 (RW.default.scanRpcServer.handler=234,queue=24,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2850a66
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 286 (RW.default.scanRpcServer.handler=233,queue=23,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7d88be24
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 285 (RW.default.scanRpcServer.handler=232,queue=22,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@572c54ef
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 284 (RW.default.scanRpcServer.handler=231,queue=21,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@303afcd9
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 283 (RW.default.scanRpcServer.handler=230,queue=20,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1ac61682
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 282 (RW.default.scanRpcServer.handler=229,queue=29,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7dc9b0cb
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 281 (RW.default.scanRpcServer.handler=228,queue=28,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6e3cbbf0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 280 (RW.default.scanRpcServer.handler=227,queue=27,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5c4466d4
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 279 (RW.default.scanRpcServer.handler=226,queue=26,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@39126391
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 278 (RW.default.scanRpcServer.handler=225,queue=25,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@9edc390
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 277 (RW.default.scanRpcServer.handler=224,queue=24,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2850a66
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 276 (RW.default.scanRpcServer.handler=223,queue=23,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7d88be24
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 275 (RW.default.scanRpcServer.handler=222,queue=22,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@572c54ef
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 274 (RW.default.scanRpcServer.handler=221,queue=21,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@303afcd9
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 273 (RW.default.scanRpcServer.handler=220,queue=20,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1ac61682
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 272 (RW.default.scanRpcServer.handler=219,queue=29,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7dc9b0cb
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 271 (RW.default.scanRpcServer.handler=218,queue=28,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6e3cbbf0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 270 (RW.default.scanRpcServer.handler=217,queue=27,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5c4466d4
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 269 (RW.default.scanRpcServer.handler=216,queue=26,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@39126391
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 268 (RW.default.scanRpcServer.handler=215,queue=25,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@9edc390
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 267 (RW.default.scanRpcServer.handler=214,queue=24,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2850a66
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 266 (RW.default.scanRpcServer.handler=213,queue=23,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7d88be24
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 265 (RW.default.scanRpcServer.handler=212,queue=22,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@572c54ef
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 264 (RW.default.scanRpcServer.handler=211,queue=21,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@303afcd9
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 263 (RW.default.scanRpcServer.handler=210,queue=20,port=60020):
State: WAITING
Blocked count: 0
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1ac61682
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 262 (RW.default.scanRpcServer.handler=209,queue=29,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7dc9b0cb
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 261 (RW.default.scanRpcServer.handler=208,queue=28,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6e3cbbf0
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 260 (RW.default.scanRpcServer.handler=207,queue=27,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5c4466d4
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 259 (RW.default.scanRpcServer.handler=206,queue=26,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@39126391
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 258 (RW.default.scanRpcServer.handler=205,queue=25,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@9edc390
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 257 (RW.default.scanRpcServer.handler=204,queue=24,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2850a66
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 256 (RW.default.scanRpcServer.handler=203,queue=23,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7d88be24
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 255 (RW.default.scanRpcServer.handler=202,queue=22,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@572c54ef
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 254 (RW.default.scanRpcServer.handler=201,queue=21,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@303afcd9
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 253 (RW.default.scanRpcServer.handler=200,queue=20,port=60020):
State: WAITING
Blocked count: 0
Waited count: 2
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1ac61682
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
org.apache.hadoop.hbase.util.BoundedPriorityBlockingQueue.take(BoundedPriorityBlockingQueue.java:214)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 252 (RW.default.readRpcServer.handler=199,queue=10,port=60020):
State: BLOCKED
Blocked count: 7104
Waited count: 824928
Blocked on java.util.concurrent.ConcurrentHashMap@570a4a28
Blocked by 199 (RW.default.readRpcServer.handler=146,queue=17,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 251 (RW.default.readRpcServer.handler=198,queue=19,port=60020):
State: BLOCKED
Blocked count: 8119
Waited count: 904631
Blocked on java.util.concurrent.ConcurrentHashMap@570a4a28
Blocked by 199 (RW.default.readRpcServer.handler=146,queue=17,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 250 (RW.default.readRpcServer.handler=197,queue=18,port=60020):
State: BLOCKED
Blocked count: 7905
Waited count: 904121
Blocked on java.util.Properties@2b39f3bd
Blocked by 247 (RW.default.readRpcServer.handler=194,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 249 (RW.default.readRpcServer.handler=196,queue=17,port=60020):
State: BLOCKED
Blocked count: 7920
Waited count: 904870
Blocked on java.util.concurrent.ConcurrentHashMap@3ec40ac8
Blocked by 196 (RW.default.readRpcServer.handler=143,queue=14,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:542)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 248 (RW.default.readRpcServer.handler=195,queue=16,port=60020):
State: BLOCKED
Blocked count: 7783
Waited count: 904746
Blocked on java.util.concurrent.ConcurrentHashMap@3ec40ac8
Blocked by 196 (RW.default.readRpcServer.handler=143,queue=14,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 247 (RW.default.readRpcServer.handler=194,queue=15,port=60020):
State: RUNNABLE
Blocked count: 7742
Waited count: 903448
Stack:
java.lang.Object.hashCode(Native Method)
java.util.concurrent.ConcurrentHashMap.replaceNode(ConcurrentHashMap.java:1106)
java.util.concurrent.ConcurrentHashMap.remove(ConcurrentHashMap.java:1097)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.close(HRegion.java:5422)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6109)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 246 (RW.default.readRpcServer.handler=193,queue=14,port=60020):
State: BLOCKED
Blocked count: 7867
Waited count: 903705
Blocked on java.util.concurrent.ConcurrentHashMap@3ec40ac8
Blocked by 196 (RW.default.readRpcServer.handler=143,queue=14,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 245 (RW.default.readRpcServer.handler=192,queue=13,port=60020):
State: BLOCKED
Blocked count: 7987
Waited count: 905059
Blocked on java.util.concurrent.ConcurrentHashMap@570a4a28
Blocked by 220 (RW.default.readRpcServer.handler=167,queue=18,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 244 (RW.default.readRpcServer.handler=191,queue=12,port=60020):
State: BLOCKED
Blocked count: 7874
Waited count: 904223
Blocked on java.util.concurrent.ConcurrentHashMap@3ec40ac8
Blocked by 196 (RW.default.readRpcServer.handler=143,queue=14,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:542)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 243 (RW.default.readRpcServer.handler=190,queue=11,port=60020):
State: BLOCKED
Blocked count: 7785
Waited count: 903750
Blocked on java.util.Properties@2b39f3bd
Blocked by 252 (RW.default.readRpcServer.handler=199,queue=10,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 242 (RW.default.readRpcServer.handler=189,queue=10,port=60020):
State: RUNNABLE
Blocked count: 7341
Waited count: 824946
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 241 (RW.default.readRpcServer.handler=188,queue=19,port=60020):
State: BLOCKED
Blocked count: 7909
Waited count: 903727
Blocked on java.util.Properties@2b39f3bd
Blocked by 252 (RW.default.readRpcServer.handler=199,queue=10,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 240 (RW.default.readRpcServer.handler=187,queue=18,port=60020):
State: BLOCKED
Blocked count: 7939
Waited count: 903993
Blocked on java.util.concurrent.ConcurrentHashMap@570a4a28
Blocked by 215 (RW.default.readRpcServer.handler=162,queue=13,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 239 (RW.default.readRpcServer.handler=186,queue=17,port=60020):
State: BLOCKED
Blocked count: 8063
Waited count: 905274
Blocked on java.util.concurrent.ConcurrentHashMap@3ec40ac8
Blocked by 153 (RW.default.readRpcServer.handler=100,queue=11,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 238 (RW.default.readRpcServer.handler=185,queue=16,port=60020):
State: BLOCKED
Blocked count: 7944
Waited count: 905038
Blocked on java.util.concurrent.ConcurrentHashMap@3ec40ac8
Blocked by 153 (RW.default.readRpcServer.handler=100,queue=11,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 237 (RW.default.readRpcServer.handler=184,queue=15,port=60020):
State: BLOCKED
Blocked count: 7883
Waited count: 903952
Blocked on java.util.concurrent.ConcurrentHashMap@570a4a28
Blocked by 229 (RW.default.readRpcServer.handler=176,queue=17,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 236 (RW.default.readRpcServer.handler=183,queue=14,port=60020):
State: BLOCKED
Blocked count: 8028
Waited count: 903630
Blocked on java.util.concurrent.ConcurrentHashMap@3ec40ac8
Blocked by 252 (RW.default.readRpcServer.handler=199,queue=10,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 235 (RW.default.readRpcServer.handler=182,queue=13,port=60020):
State: BLOCKED
Blocked count: 7939
Waited count: 904519
Blocked on java.util.Properties@2b39f3bd
Blocked by 250 (RW.default.readRpcServer.handler=197,queue=18,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 234 (RW.default.readRpcServer.handler=181,queue=12,port=60020):
State: BLOCKED
Blocked count: 7761
Waited count: 903896
Blocked on java.util.Properties@2b39f3bd
Blocked by 250 (RW.default.readRpcServer.handler=197,queue=18,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 233 (RW.default.readRpcServer.handler=180,queue=11,port=60020):
State: BLOCKED
Blocked count: 7789
Waited count: 903531
Blocked on java.util.Properties@2b39f3bd
Blocked by 250 (RW.default.readRpcServer.handler=197,queue=18,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 232 (RW.default.readRpcServer.handler=179,queue=10,port=60020):
State: BLOCKED
Blocked count: 7300
Waited count: 824712
Blocked on java.util.Properties@2b39f3bd
Blocked by 250 (RW.default.readRpcServer.handler=197,queue=18,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 231 (RW.default.readRpcServer.handler=178,queue=19,port=60020):
State: BLOCKED
Blocked count: 7732
Waited count: 904142
Blocked on java.util.Properties@2b39f3bd
Blocked by 250 (RW.default.readRpcServer.handler=197,queue=18,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 230 (RW.default.readRpcServer.handler=177,queue=18,port=60020):
State: BLOCKED
Blocked count: 7767
Waited count: 904181
Blocked on java.util.Properties@2b39f3bd
Blocked by 161 (RW.default.readRpcServer.handler=108,queue=19,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 229 (RW.default.readRpcServer.handler=176,queue=17,port=60020):
State: BLOCKED
Blocked count: 8047
Waited count: 905078
Blocked on java.util.Properties@2b39f3bd
Blocked by 161 (RW.default.readRpcServer.handler=108,queue=19,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 228 (RW.default.readRpcServer.handler=175,queue=16,port=60020):
State: BLOCKED
Blocked count: 7974
Waited count: 904855
Blocked on java.util.concurrent.ConcurrentHashMap@3ec40ac8
Blocked by 213 (RW.default.readRpcServer.handler=160,queue=11,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:542)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 227 (RW.default.readRpcServer.handler=174,queue=15,port=60020):
State: BLOCKED
Blocked count: 7896
Waited count: 903605
Blocked on java.util.concurrent.ConcurrentHashMap@570a4a28
Blocked by 186 (RW.default.readRpcServer.handler=133,queue=14,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 226 (RW.default.readRpcServer.handler=173,queue=14,port=60020):
State: BLOCKED
Blocked count: 7929
Waited count: 903816
Blocked on java.util.Properties@2b39f3bd
Blocked by 191 (RW.default.readRpcServer.handler=138,queue=19,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 225 (RW.default.readRpcServer.handler=172,queue=13,port=60020):
State: BLOCKED
Blocked count: 7871
Waited count: 903726
Blocked on java.util.concurrent.ConcurrentHashMap@570a4a28
Blocked by 186 (RW.default.readRpcServer.handler=133,queue=14,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 224 (RW.default.readRpcServer.handler=171,queue=12,port=60020):
State: BLOCKED
Blocked count: 7816
Waited count: 904042
Blocked on java.util.Properties@2b39f3bd
Blocked by 243 (RW.default.readRpcServer.handler=190,queue=11,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 223 (RW.default.readRpcServer.handler=170,queue=11,port=60020):
State: BLOCKED
Blocked count: 8078
Waited count: 904162
Blocked on java.util.Properties@2b39f3bd
Blocked by 243 (RW.default.readRpcServer.handler=190,queue=11,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:542)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 222 (RW.default.readRpcServer.handler=169,queue=10,port=60020):
State: BLOCKED
Blocked count: 7316
Waited count: 825664
Blocked on java.util.Properties@2b39f3bd
Blocked by 243 (RW.default.readRpcServer.handler=190,queue=11,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 221 (RW.default.readRpcServer.handler=168,queue=19,port=60020):
State: BLOCKED
Blocked count: 7857
Waited count: 903652
Blocked on java.util.concurrent.ConcurrentHashMap@3ec40ac8
Blocked by 155 (RW.default.readRpcServer.handler=102,queue=13,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 220 (RW.default.readRpcServer.handler=167,queue=18,port=60020):
State: BLOCKED
Blocked count: 8426
Waited count: 903611
Blocked on java.util.Properties@2b39f3bd
Blocked by 243 (RW.default.readRpcServer.handler=190,queue=11,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 219 (RW.default.readRpcServer.handler=166,queue=17,port=60020):
State: BLOCKED
Blocked count: 7872
Waited count: 905304
Blocked on java.util.Properties@2b39f3bd
Blocked by 243 (RW.default.readRpcServer.handler=190,queue=11,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 218 (RW.default.readRpcServer.handler=165,queue=16,port=60020):
State: BLOCKED
Blocked count: 7809
Waited count: 904644
Blocked on java.util.Properties@2b39f3bd
Blocked by 243 (RW.default.readRpcServer.handler=190,queue=11,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 217 (RW.default.readRpcServer.handler=164,queue=15,port=60020):
State: BLOCKED
Blocked count: 7773
Waited count: 905518
Blocked on java.util.Properties@2b39f3bd
Blocked by 243 (RW.default.readRpcServer.handler=190,queue=11,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 216 (RW.default.readRpcServer.handler=163,queue=14,port=60020):
State: BLOCKED
Blocked count: 7879
Waited count: 904002
Blocked on java.util.Properties@2b39f3bd
Blocked by 177 (RW.default.readRpcServer.handler=124,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 215 (RW.default.readRpcServer.handler=162,queue=13,port=60020):
State: BLOCKED
Blocked count: 7858
Waited count: 903790
Blocked on java.util.Properties@2b39f3bd
Blocked by 177 (RW.default.readRpcServer.handler=124,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 214 (RW.default.readRpcServer.handler=161,queue=12,port=60020):
State: BLOCKED
Blocked count: 7949
Waited count: 904567
Blocked on java.util.Properties@2b39f3bd
Blocked by 177 (RW.default.readRpcServer.handler=124,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 213 (RW.default.readRpcServer.handler=160,queue=11,port=60020):
State: RUNNABLE
Blocked count: 7981
Waited count: 904261
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 212 (RW.default.readRpcServer.handler=159,queue=10,port=60020):
State: BLOCKED
Blocked count: 7219
Waited count: 824673
Blocked on java.util.Properties@2b39f3bd
Blocked by 192 (RW.default.readRpcServer.handler=139,queue=10,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 211 (RW.default.readRpcServer.handler=158,queue=19,port=60020):
State: BLOCKED
Blocked count: 7957
Waited count: 903722
Blocked on java.util.Properties@2b39f3bd
Blocked by 192 (RW.default.readRpcServer.handler=139,queue=10,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 210 (RW.default.readRpcServer.handler=157,queue=18,port=60020):
State: BLOCKED
Blocked count: 8029
Waited count: 904094
Blocked on java.util.Properties@2b39f3bd
Blocked by 177 (RW.default.readRpcServer.handler=124,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 209 (RW.default.readRpcServer.handler=156,queue=17,port=60020):
State: BLOCKED
Blocked count: 8011
Waited count: 905027
Blocked on java.util.Properties@2b39f3bd
Blocked by 177 (RW.default.readRpcServer.handler=124,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 208 (RW.default.readRpcServer.handler=155,queue=16,port=60020):
State: BLOCKED
Blocked count: 8051
Waited count: 904869
Blocked on java.util.Properties@2b39f3bd
Blocked by 177 (RW.default.readRpcServer.handler=124,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 207 (RW.default.readRpcServer.handler=154,queue=15,port=60020):
State: BLOCKED
Blocked count: 7924
Waited count: 903961
Blocked on java.util.Properties@2b39f3bd
Blocked by 177 (RW.default.readRpcServer.handler=124,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 206 (RW.default.readRpcServer.handler=153,queue=14,port=60020):
State: BLOCKED
Blocked count: 7785
Waited count: 903828
Blocked on java.util.Properties@2b39f3bd
Blocked by 177 (RW.default.readRpcServer.handler=124,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:542)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 205 (RW.default.readRpcServer.handler=152,queue=13,port=60020):
State: BLOCKED
Blocked count: 7882
Waited count: 904398
Blocked on java.util.Properties@2b39f3bd
Blocked by 190 (RW.default.readRpcServer.handler=137,queue=18,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:542)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 204 (RW.default.readRpcServer.handler=151,queue=12,port=60020):
State: BLOCKED
Blocked count: 7979
Waited count: 904433
Blocked on java.util.Properties@2b39f3bd
Blocked by 190 (RW.default.readRpcServer.handler=137,queue=18,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 203 (RW.default.readRpcServer.handler=150,queue=11,port=60020):
State: BLOCKED
Blocked count: 8080
Waited count: 904190
Blocked on java.util.Properties@2b39f3bd
Blocked by 235 (RW.default.readRpcServer.handler=182,queue=13,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 202 (RW.default.readRpcServer.handler=149,queue=10,port=60020):
State: BLOCKED
Blocked count: 7088
Waited count: 824465
Blocked on java.util.Properties@2b39f3bd
Blocked by 235 (RW.default.readRpcServer.handler=182,queue=13,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 201 (RW.default.readRpcServer.handler=148,queue=19,port=60020):
State: BLOCKED
Blocked count: 7870
Waited count: 904351
Blocked on java.util.Properties@2b39f3bd
Blocked by 235 (RW.default.readRpcServer.handler=182,queue=13,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 200 (RW.default.readRpcServer.handler=147,queue=18,port=60020):
State: BLOCKED
Blocked count: 7821
Waited count: 903675
Blocked on java.util.Properties@2b39f3bd
Blocked by 184 (RW.default.readRpcServer.handler=131,queue=12,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 199 (RW.default.readRpcServer.handler=146,queue=17,port=60020):
State: BLOCKED
Blocked count: 7906
Waited count: 904901
Blocked on java.util.Properties@2b39f3bd
Blocked by 184 (RW.default.readRpcServer.handler=131,queue=12,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 198 (RW.default.readRpcServer.handler=145,queue=16,port=60020):
State: BLOCKED
Blocked count: 8107
Waited count: 904545
Blocked on java.util.Properties@2b39f3bd
Blocked by 184 (RW.default.readRpcServer.handler=131,queue=12,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 197 (RW.default.readRpcServer.handler=144,queue=15,port=60020):
State: BLOCKED
Blocked count: 7823
Waited count: 903958
Blocked on java.util.Properties@2b39f3bd
Blocked by 184 (RW.default.readRpcServer.handler=131,queue=12,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 196 (RW.default.readRpcServer.handler=143,queue=14,port=60020):
State: BLOCKED
Blocked count: 7792
Waited count: 904103
Blocked on java.util.Properties@2b39f3bd
Blocked by 184 (RW.default.readRpcServer.handler=131,queue=12,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 195 (RW.default.readRpcServer.handler=142,queue=13,port=60020):
State: BLOCKED
Blocked count: 8086
Waited count: 904116
Blocked on java.util.Properties@2b39f3bd
Blocked by 131 (RW.default.writeRpcServer.handler=78,queue=8,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 194 (RW.default.readRpcServer.handler=141,queue=12,port=60020):
State: BLOCKED
Blocked count: 7838
Waited count: 903688
Blocked on java.util.Properties@2b39f3bd
Blocked by 131 (RW.default.writeRpcServer.handler=78,queue=8,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 193 (RW.default.readRpcServer.handler=140,queue=11,port=60020):
State: BLOCKED
Blocked count: 7955
Waited count: 903978
Blocked on java.util.Properties@2b39f3bd
Blocked by 131 (RW.default.writeRpcServer.handler=78,queue=8,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 192 (RW.default.readRpcServer.handler=139,queue=10,port=60020):
State: BLOCKED
Blocked count: 7156
Waited count: 825065
Blocked on java.util.Properties@2b39f3bd
Blocked by 155 (RW.default.readRpcServer.handler=102,queue=13,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 191 (RW.default.readRpcServer.handler=138,queue=19,port=60020):
State: BLOCKED
Blocked count: 7864
Waited count: 904834
Blocked on java.util.Properties@2b39f3bd
Blocked by 155 (RW.default.readRpcServer.handler=102,queue=13,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 190 (RW.default.readRpcServer.handler=137,queue=18,port=60020):
State: BLOCKED
Blocked count: 8012
Waited count: 903812
Blocked on java.util.Properties@2b39f3bd
Blocked by 155 (RW.default.readRpcServer.handler=102,queue=13,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 189 (RW.default.readRpcServer.handler=136,queue=17,port=60020):
State: BLOCKED
Blocked count: 7718
Waited count: 904429
Blocked on java.util.Properties@2b39f3bd
Blocked by 155 (RW.default.readRpcServer.handler=102,queue=13,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 188 (RW.default.readRpcServer.handler=135,queue=16,port=60020):
State: BLOCKED
Blocked count: 7936
Waited count: 904642
Blocked on java.util.Properties@2b39f3bd
Blocked by 183 (RW.default.readRpcServer.handler=130,queue=11,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 187 (RW.default.readRpcServer.handler=134,queue=15,port=60020):
State: BLOCKED
Blocked count: 7847
Waited count: 904039
Blocked on java.util.Properties@2b39f3bd
Blocked by 183 (RW.default.readRpcServer.handler=130,queue=11,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 186 (RW.default.readRpcServer.handler=133,queue=14,port=60020):
State: BLOCKED
Blocked count: 7755
Waited count: 904330
Blocked on java.util.Properties@2b39f3bd
Blocked by 183 (RW.default.readRpcServer.handler=130,queue=11,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 185 (RW.default.readRpcServer.handler=132,queue=13,port=60020):
State: BLOCKED
Blocked count: 7886
Waited count: 904128
Blocked on java.util.Properties@2b39f3bd
Blocked by 183 (RW.default.readRpcServer.handler=130,queue=11,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:542)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 184 (RW.default.readRpcServer.handler=131,queue=12,port=60020):
State: RUNNABLE
Blocked count: 7842
Waited count: 904204
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 183 (RW.default.readRpcServer.handler=130,queue=11,port=60020):
State: RUNNABLE
Blocked count: 7849
Waited count: 904219
Stack:
java.lang.Object.hashCode(Native Method)
java.util.concurrent.ConcurrentHashMap.replaceNode(ConcurrentHashMap.java:1106)
java.util.concurrent.ConcurrentHashMap.remove(ConcurrentHashMap.java:1097)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.close(HRegion.java:5422)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6109)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 182 (RW.default.readRpcServer.handler=129,queue=10,port=60020):
State: BLOCKED
Blocked count: 7462
Waited count: 824709
Blocked on java.util.Properties@2b39f3bd
Blocked by 186 (RW.default.readRpcServer.handler=133,queue=14,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:542)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 181 (RW.default.readRpcServer.handler=128,queue=19,port=60020):
State: BLOCKED
Blocked count: 7999
Waited count: 903592
Blocked on java.util.Properties@2b39f3bd
Blocked by 186 (RW.default.readRpcServer.handler=133,queue=14,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 180 (RW.default.readRpcServer.handler=127,queue=18,port=60020):
State: BLOCKED
Blocked count: 7933
Waited count: 903716
Blocked on java.util.Properties@2b39f3bd
Blocked by 186 (RW.default.readRpcServer.handler=133,queue=14,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 179 (RW.default.readRpcServer.handler=126,queue=17,port=60020):
State: BLOCKED
Blocked count: 8020
Waited count: 904655
Blocked on java.util.Properties@2b39f3bd
Blocked by 238 (RW.default.readRpcServer.handler=185,queue=16,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 178 (RW.default.readRpcServer.handler=125,queue=16,port=60020):
State: BLOCKED
Blocked count: 7964
Waited count: 904067
Blocked on java.util.Properties@2b39f3bd
Blocked by 238 (RW.default.readRpcServer.handler=185,queue=16,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 177 (RW.default.readRpcServer.handler=124,queue=15,port=60020):
State: BLOCKED
Blocked count: 7907
Waited count: 903591
Blocked on java.util.Properties@2b39f3bd
Blocked by 238 (RW.default.readRpcServer.handler=185,queue=16,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 176 (RW.default.readRpcServer.handler=123,queue=14,port=60020):
State: BLOCKED
Blocked count: 8200
Waited count: 903792
Blocked on java.util.Properties@2b39f3bd
Blocked by 238 (RW.default.readRpcServer.handler=185,queue=16,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 175 (RW.default.readRpcServer.handler=122,queue=13,port=60020):
State: BLOCKED
Blocked count: 7940
Waited count: 904523
Blocked on java.util.Properties@2b39f3bd
Blocked by 228 (RW.default.readRpcServer.handler=175,queue=16,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 174 (RW.default.readRpcServer.handler=121,queue=12,port=60020):
State: BLOCKED
Blocked count: 8027
Waited count: 904070
Blocked on java.util.Properties@2b39f3bd
Blocked by 237 (RW.default.readRpcServer.handler=184,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:542)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 173 (RW.default.readRpcServer.handler=120,queue=11,port=60020):
State: BLOCKED
Blocked count: 7875
Waited count: 904050
Blocked on java.util.Properties@2b39f3bd
Blocked by 237 (RW.default.readRpcServer.handler=184,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 172 (RW.default.readRpcServer.handler=119,queue=10,port=60020):
State: BLOCKED
Blocked count: 7268
Waited count: 824972
Blocked on java.util.Properties@2b39f3bd
Blocked by 237 (RW.default.readRpcServer.handler=184,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 171 (RW.default.readRpcServer.handler=118,queue=19,port=60020):
State: BLOCKED
Blocked count: 7922
Waited count: 903944
Blocked on java.util.Properties@2b39f3bd
Blocked by 237 (RW.default.readRpcServer.handler=184,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 170 (RW.default.readRpcServer.handler=117,queue=18,port=60020):
State: BLOCKED
Blocked count: 7779
Waited count: 904213
Blocked on java.util.Properties@2b39f3bd
Blocked by 212 (RW.default.readRpcServer.handler=159,queue=10,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 169 (RW.default.readRpcServer.handler=116,queue=17,port=60020):
State: BLOCKED
Blocked count: 8061
Waited count: 905207
Blocked on java.util.Properties@2b39f3bd
Blocked by 212 (RW.default.readRpcServer.handler=159,queue=10,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 168 (RW.default.readRpcServer.handler=115,queue=16,port=60020):
State: BLOCKED
Blocked count: 7884
Waited count: 904775
Blocked on java.util.Properties@2b39f3bd
Blocked by 223 (RW.default.readRpcServer.handler=170,queue=11,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 167 (RW.default.readRpcServer.handler=114,queue=15,port=60020):
State: BLOCKED
Blocked count: 7831
Waited count: 903693
Blocked on java.util.Properties@2b39f3bd
Blocked by 223 (RW.default.readRpcServer.handler=170,queue=11,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 166 (RW.default.readRpcServer.handler=113,queue=14,port=60020):
State: BLOCKED
Blocked count: 7819
Waited count: 903401
Blocked on java.util.Properties@2b39f3bd
Blocked by 194 (RW.default.readRpcServer.handler=141,queue=12,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 165 (RW.default.readRpcServer.handler=112,queue=13,port=60020):
State: BLOCKED
Blocked count: 8251
Waited count: 904160
Blocked on java.util.Properties@2b39f3bd
Blocked by 194 (RW.default.readRpcServer.handler=141,queue=12,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 164 (RW.default.readRpcServer.handler=111,queue=12,port=60020):
State: BLOCKED
Blocked count: 7748
Waited count: 903684
Blocked on java.util.Properties@2b39f3bd
Blocked by 194 (RW.default.readRpcServer.handler=141,queue=12,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 163 (RW.default.readRpcServer.handler=110,queue=11,port=60020):
State: BLOCKED
Blocked count: 7891
Waited count: 904237
Blocked on java.util.Properties@2b39f3bd
Blocked by 194 (RW.default.readRpcServer.handler=141,queue=12,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 162 (RW.default.readRpcServer.handler=109,queue=10,port=60020):
State: BLOCKED
Blocked count: 7272
Waited count: 825304
Blocked on java.util.concurrent.ConcurrentHashMap@5e23dfe2
Blocked by 222 (RW.default.readRpcServer.handler=169,queue=10,port=60020)
Stack:
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5077)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 161 (RW.default.readRpcServer.handler=108,queue=19,port=60020):
State: BLOCKED
Blocked count: 7822
Waited count: 903947
Blocked on java.util.Properties@2b39f3bd
Blocked by 175 (RW.default.readRpcServer.handler=122,queue=13,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 160 (RW.default.readRpcServer.handler=107,queue=18,port=60020):
State: BLOCKED
Blocked count: 8005
Waited count: 903641
Blocked on java.util.Properties@2b39f3bd
Blocked by 175 (RW.default.readRpcServer.handler=122,queue=13,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 159 (RW.default.readRpcServer.handler=106,queue=17,port=60020):
State: BLOCKED
Blocked count: 7904
Waited count: 904928
Blocked on java.util.Properties@2b39f3bd
Blocked by 175 (RW.default.readRpcServer.handler=122,queue=13,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 158 (RW.default.readRpcServer.handler=105,queue=16,port=60020):
State: BLOCKED
Blocked count: 7807
Waited count: 904588
Blocked on org.apache.hadoop.conf.Configuration@54c562f7
Blocked by 214 (RW.default.readRpcServer.handler=161,queue=12,port=60020)
Stack:
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2315)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
Thread 157 (RW.default.readRpcServer.handler=104,queue=15,port=60020):
State: BLOCKED
Blocked count: 7864
Waited count: 904415
Blocked on org.apache.hadoop.conf.Configuration@54c562f7
Blocked by 214 (RW.default.readRpcServer.handler=161,queue=12,port=60020)
Stack:
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2315)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
Thread 156 (RW.default.readRpcServer.handler=103,queue=14,port=60020):
State: BLOCKED
Blocked count: 7883
Waited count: 903483
Blocked on org.apache.hadoop.conf.Configuration@54c562f7
Blocked by 214 (RW.default.readRpcServer.handler=161,queue=12,port=60020)
Stack:
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2315)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
Thread 155 (RW.default.readRpcServer.handler=102,queue=13,port=60020):
State: BLOCKED
Blocked count: 7867
Waited count: 904319
Blocked on java.util.Properties@2b39f3bd
Blocked by 187 (RW.default.readRpcServer.handler=134,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 154 (RW.default.readRpcServer.handler=101,queue=12,port=60020):
State: BLOCKED
Blocked count: 7865
Waited count: 903800
Blocked on java.util.Properties@2b39f3bd
Blocked by 187 (RW.default.readRpcServer.handler=134,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 153 (RW.default.readRpcServer.handler=100,queue=11,port=60020):
State: BLOCKED
Blocked count: 7783
Waited count: 903531
Blocked on java.util.Properties@2b39f3bd
Blocked by 187 (RW.default.readRpcServer.handler=134,queue=15,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2121)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31443)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 152 (RW.default.readRpcServer.handler=99,queue=10,port=60020):
State: BLOCKED
Blocked count: 7185
Waited count: 824314
Blocked on java.util.Properties@2b39f3bd
Blocked by 189 (RW.default.readRpcServer.handler=136,queue=17,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6079)
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1792)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31439)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
Thread 151 (RW.default.writeRpcServer.handler=98,queue=8,port=60020):
State: WAITING
Blocked count: 759004
Waited count: 855434
Waiting on java.util.LinkedList@39c59765
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 150 (RW.default.writeRpcServer.handler=97,queue=7,port=60020):
State: WAITING
Blocked count: 767234
Waited count: 861630
Waiting on java.util.LinkedList@36df20ed
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 149 (RW.default.writeRpcServer.handler=96,queue=6,port=60020):
State: RUNNABLE
Blocked count: 753513
Waited count: 849492
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 148 (RW.default.writeRpcServer.handler=95,queue=5,port=60020):
State: WAITING
Blocked count: 756256
Waited count: 850687
Waiting on java.util.LinkedList@36df20ed
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 147 (RW.default.writeRpcServer.handler=94,queue=4,port=60020):
State: WAITING
Blocked count: 758699
Waited count: 852455
Waiting on java.util.LinkedList@243e1221
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 146 (RW.default.writeRpcServer.handler=93,queue=3,port=60020):
State: BLOCKED
Blocked count: 762010
Waited count: 859263
Blocked on java.util.LinkedList@778a8c58
Blocked by 72 (RW.default.writeRpcServer.handler=19,queue=9,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 145 (RW.default.writeRpcServer.handler=92,queue=2,port=60020):
State: BLOCKED
Blocked count: 757333
Waited count: 852454
Blocked on java.util.Properties@2b39f3bd
Blocked by 241 (RW.default.readRpcServer.handler=188,queue=19,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6741)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
Thread 144 (RW.default.writeRpcServer.handler=91,queue=1,port=60020):
State: WAITING
Blocked count: 752385
Waited count: 847506
Waiting on java.util.LinkedList@39c59765
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 143 (RW.default.writeRpcServer.handler=90,queue=0,port=60020):
State: WAITING
Blocked count: 764758
Waited count: 859265
Waiting on java.util.LinkedList@39c59765
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 142 (RW.default.writeRpcServer.handler=89,queue=9,port=60020):
State: WAITING
Blocked count: 822756
Waited count: 924657
Waiting on java.util.LinkedList@36df20ed
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 141 (RW.default.writeRpcServer.handler=88,queue=8,port=60020):
State: BLOCKED
Blocked count: 762961
Waited count: 858295
Blocked on java.util.LinkedList@778a8c58
Blocked by 105 (RW.default.writeRpcServer.handler=52,queue=2,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 140 (RW.default.writeRpcServer.handler=87,queue=7,port=60020):
State: WAITING
Blocked count: 770143
Waited count: 866102
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@168d009d
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 139 (RW.default.writeRpcServer.handler=86,queue=6,port=60020):
State: WAITING
Blocked count: 757633
Waited count: 851863
Waiting on java.util.LinkedList@39c59765
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 138 (RW.default.writeRpcServer.handler=85,queue=5,port=60020):
State: BLOCKED
Blocked count: 754056
Waited count: 850004
Blocked on java.util.LinkedList@778a8c58
Blocked by 105 (RW.default.writeRpcServer.handler=52,queue=2,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 137 (RW.default.writeRpcServer.handler=84,queue=4,port=60020):
State: WAITING
Blocked count: 752147
Waited count: 845993
Waiting on java.util.LinkedList@36df20ed
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 136 (RW.default.writeRpcServer.handler=83,queue=3,port=60020):
State: BLOCKED
Blocked count: 770301
Waited count: 863621
Blocked on java.util.LinkedList@778a8c58
Blocked by 141 (RW.default.writeRpcServer.handler=88,queue=8,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 135 (RW.default.writeRpcServer.handler=82,queue=2,port=60020):
State: WAITING
Blocked count: 756383
Waited count: 852210
Waiting on java.util.LinkedList@778a8c58
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 134 (RW.default.writeRpcServer.handler=81,queue=1,port=60020):
State: RUNNABLE
Blocked count: 757887
Waited count: 853471
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 133 (RW.default.writeRpcServer.handler=80,queue=0,port=60020):
State: RUNNABLE
Blocked count: 768453
Waited count: 862322
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 132 (RW.default.writeRpcServer.handler=79,queue=9,port=60020):
State: WAITING
Blocked count: 824373
Waited count: 928722
Waiting on java.util.LinkedList@243e1221
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 131 (RW.default.writeRpcServer.handler=78,queue=8,port=60020):
State: WAITING
Blocked count: 771119
Waited count: 865384
Waiting on java.util.LinkedList@778a8c58
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 130 (RW.default.writeRpcServer.handler=77,queue=7,port=60020):
State: WAITING
Blocked count: 772811
Waited count: 867205
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@168d009d
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 129 (RW.default.writeRpcServer.handler=76,queue=6,port=60020):
State: WAITING
Blocked count: 769525
Waited count: 864686
Waiting on java.util.LinkedList@778a8c58
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 128 (RW.default.writeRpcServer.handler=75,queue=5,port=60020):
State: BLOCKED
Blocked count: 760495
Waited count: 854235
Blocked on java.util.LinkedList@39c59765
Blocked by 74 (RW.default.writeRpcServer.handler=21,queue=1,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 127 (RW.default.writeRpcServer.handler=74,queue=4,port=60020):
State: BLOCKED
Blocked count: 760324
Waited count: 854415
Blocked on java.util.LinkedList@243e1221
Blocked by 119 (RW.default.writeRpcServer.handler=66,queue=6,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 126 (RW.default.writeRpcServer.handler=73,queue=3,port=60020):
State: BLOCKED
Blocked count: 765053
Waited count: 860697
Blocked on java.util.LinkedList@39c59765
Blocked by 74 (RW.default.writeRpcServer.handler=21,queue=1,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 125 (RW.default.writeRpcServer.handler=72,queue=2,port=60020):
State: WAITING
Blocked count: 747385
Waited count: 842725
Waiting on java.util.LinkedList@243e1221
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 124 (RW.default.writeRpcServer.handler=71,queue=1,port=60020):
State: RUNNABLE
Blocked count: 769744
Waited count: 864479
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 123 (RW.default.writeRpcServer.handler=70,queue=0,port=60020):
State: WAITING
Blocked count: 772107
Waited count: 867274
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5f054e9
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 122 (RW.default.writeRpcServer.handler=69,queue=9,port=60020):
State: RUNNABLE
Blocked count: 826286
Waited count: 931243
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 121 (RW.default.writeRpcServer.handler=68,queue=8,port=60020):
State: RUNNABLE
Blocked count: 772776
Waited count: 866375
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 120 (RW.default.writeRpcServer.handler=67,queue=7,port=60020):
State: RUNNABLE
Blocked count: 761711
Waited count: 857569
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 119 (RW.default.writeRpcServer.handler=66,queue=6,port=60020):
State: RUNNABLE
Blocked count: 763591
Waited count: 859181
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 118 (RW.default.writeRpcServer.handler=65,queue=5,port=60020):
State: BLOCKED
Blocked count: 752689
Waited count: 846639
Blocked on org.apache.hadoop.conf.Configuration@54c562f7
Blocked by 169 (RW.default.readRpcServer.handler=116,queue=17,port=60020)
Stack:
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2315)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1346)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:154)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6741)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 117 (RW.default.writeRpcServer.handler=64,queue=4,port=60020):
State: RUNNABLE
Blocked count: 754441
Waited count: 850416
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 116 (RW.default.writeRpcServer.handler=63,queue=3,port=60020):
State: RUNNABLE
Blocked count: 757418
Waited count: 853398
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 115 (RW.default.writeRpcServer.handler=62,queue=2,port=60020):
State: BLOCKED
Blocked count: 754819
Waited count: 849309
Blocked on org.apache.hadoop.conf.Configuration@54c562f7
Blocked by 166 (RW.default.readRpcServer.handler=113,queue=14,port=60020)
Stack:
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2315)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6741)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 114 (RW.default.writeRpcServer.handler=61,queue=1,port=60020):
State: BLOCKED
Blocked count: 748257
Waited count: 841376
Blocked on java.util.LinkedList@778a8c58
Blocked by 75 (RW.default.writeRpcServer.handler=22,queue=2,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 113 (RW.default.writeRpcServer.handler=60,queue=0,port=60020):
State: WAITING
Blocked count: 758153
Waited count: 853186
Waiting on java.util.LinkedList@39c59765
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 112 (RW.default.writeRpcServer.handler=59,queue=9,port=60020):
State: BLOCKED
Blocked count: 811310
Waited count: 915744
Blocked on java.util.LinkedList@778a8c58
Blocked by 65 (RW.default.writeRpcServer.handler=12,queue=2,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 111 (RW.default.writeRpcServer.handler=58,queue=8,port=60020):
State: BLOCKED
Blocked count: 757008
Waited count: 852940
Blocked on org.apache.hadoop.conf.Configuration@54c562f7
Blocked by 168 (RW.default.readRpcServer.handler=115,queue=16,port=60020)
Stack:
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2315)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1346)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:154)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6741)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 110 (RW.default.writeRpcServer.handler=57,queue=7,port=60020):
State: RUNNABLE
Blocked count: 764760
Waited count: 858891
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 109 (RW.default.writeRpcServer.handler=56,queue=6,port=60020):
State: RUNNABLE
Blocked count: 766641
Waited count: 859740
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 108 (RW.default.writeRpcServer.handler=55,queue=5,port=60020):
State: BLOCKED
Blocked count: 767457
Waited count: 861978
Blocked on org.apache.hadoop.conf.Configuration@54c562f7
Blocked by 158 (RW.default.readRpcServer.handler=105,queue=16,port=60020)
Stack:
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2315)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6741)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 107 (RW.default.writeRpcServer.handler=54,queue=4,port=60020):
State: BLOCKED
Blocked count: 744242
Waited count: 839158
Blocked on java.util.LinkedList@39c59765
Blocked by 55 (RW.default.writeRpcServer.handler=2,queue=2,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 106 (RW.default.writeRpcServer.handler=53,queue=3,port=60020):
State: WAITING
Blocked count: 759751
Waited count: 854443
Waiting on java.util.LinkedList@243e1221
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 105 (RW.default.writeRpcServer.handler=52,queue=2,port=60020):
State: BLOCKED
Blocked count: 753321
Waited count: 849192
Blocked on java.util.LinkedList@778a8c58
Blocked by 145 (RW.default.writeRpcServer.handler=92,queue=2,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 104 (RW.default.writeRpcServer.handler=51,queue=1,port=60020):
State: BLOCKED
Blocked count: 757000
Waited count: 852212
Blocked on java.util.LinkedList@39c59765
Blocked by 151 (RW.default.writeRpcServer.handler=98,queue=8,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 103 (RW.default.writeRpcServer.handler=50,queue=0,port=60020):
State: BLOCKED
Blocked count: 762351
Waited count: 857104
Blocked on org.apache.hadoop.conf.Configuration@54c562f7
Blocked by 177 (RW.default.readRpcServer.handler=124,queue=15,port=60020)
Stack:
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2315)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6741)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 102 (RW.default.writeRpcServer.handler=49,queue=9,port=60020):
State: BLOCKED
Blocked count: 823340
Waited count: 927478
Blocked on java.util.LinkedList@243e1221
Blocked by 125 (RW.default.writeRpcServer.handler=72,queue=2,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 101 (RW.default.writeRpcServer.handler=48,queue=8,port=60020):
State: RUNNABLE
Blocked count: 757411
Waited count: 851925
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 100 (RW.default.writeRpcServer.handler=47,queue=7,port=60020):
State: WAITING
Blocked count: 763505
Waited count: 858800
Waiting on java.util.LinkedList@410c4b88
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 99 (RW.default.writeRpcServer.handler=46,queue=6,port=60020):
State: BLOCKED
Blocked count: 761623
Waited count: 857286
Blocked on org.apache.hadoop.conf.Configuration@54c562f7
Blocked by 243 (RW.default.readRpcServer.handler=190,queue=11,port=60020)
Stack:
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2315)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1346)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:136)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6741)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 98 (RW.default.writeRpcServer.handler=45,queue=5,port=60020):
State: WAITING
Blocked count: 767702
Waited count: 864471
Waiting on java.util.LinkedList@410c4b88
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 97 (RW.default.writeRpcServer.handler=44,queue=4,port=60020):
State: BLOCKED
Blocked count: 750385
Waited count: 844938
Blocked on java.util.LinkedList@243e1221
Blocked by 64 (RW.default.writeRpcServer.handler=11,queue=1,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 96 (RW.default.writeRpcServer.handler=43,queue=3,port=60020):
State: BLOCKED
Blocked count: 761557
Waited count: 855839
Blocked on org.apache.hadoop.conf.Configuration@54c562f7
Blocked by 179 (RW.default.readRpcServer.handler=126,queue=17,port=60020)
Stack:
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2315)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6741)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 95 (RW.default.writeRpcServer.handler=42,queue=2,port=60020):
State: WAITING
Blocked count: 760346
Waited count: 854352
Waiting on java.util.LinkedList@410c4b88
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 94 (RW.default.writeRpcServer.handler=41,queue=1,port=60020):
State: WAITING
Blocked count: 752101
Waited count: 845390
Waiting on java.util.LinkedList@39c59765
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 93 (RW.default.writeRpcServer.handler=40,queue=0,port=60020):
State: WAITING
Blocked count: 777280
Waited count: 870617
Waiting on java.util.LinkedList@778a8c58
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 92 (RW.default.writeRpcServer.handler=39,queue=9,port=60020):
State: BLOCKED
Blocked count: 809904
Waited count: 914294
Blocked on java.util.LinkedList@39c59765
Blocked by 121 (RW.default.writeRpcServer.handler=68,queue=8,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 91 (RW.default.writeRpcServer.handler=38,queue=8,port=60020):
State: RUNNABLE
Blocked count: 752411
Waited count: 847174
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 90 (RW.default.writeRpcServer.handler=37,queue=7,port=60020):
State: WAITING
Blocked count: 762993
Waited count: 856972
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@168d009d
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 89 (RW.default.writeRpcServer.handler=36,queue=6,port=60020):
State: WAITING
Blocked count: 759629
Waited count: 855691
Waiting on java.util.LinkedList@410c4b88
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 88 (RW.default.writeRpcServer.handler=35,queue=5,port=60020):
State: TIMED_WAITING
Blocked count: 761011
Waited count: 854919
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
java.util.concurrent.ExecutorCompletionService.poll(ExecutorCompletionService.java:202)
org.apache.hadoop.hdfs.DFSInputStream.hedgedFetchBlockByteRange(DFSInputStream.java:1205)
org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:1379)
org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:89)
org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader.readAtOffset(HFileBlock.java:1377)
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1588)
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1467)
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:430)
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:259)
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:625)
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:605)
org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:267)
org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:181)
org.apache.hadoop.hbase.regionserver.StoreFileScanner.enforceSeek(StoreFileScanner.java:387)
org.apache.hadoop.hbase.regionserver.KeyValueHeap.pollRealKV(KeyValueHeap.java:346)
org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:303)
Thread 87 (RW.default.writeRpcServer.handler=34,queue=4,port=60020):
State: RUNNABLE
Blocked count: 757864
Waited count: 852835
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 86 (RW.default.writeRpcServer.handler=33,queue=3,port=60020):
State: WAITING
Blocked count: 756458
Waited count: 852196
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5c1148f4
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 85 (RW.default.writeRpcServer.handler=32,queue=2,port=60020):
State: RUNNABLE
Blocked count: 755454
Waited count: 849452
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 84 (RW.default.writeRpcServer.handler=31,queue=1,port=60020):
State: WAITING
Blocked count: 756236
Waited count: 850611
Waiting on java.util.LinkedList@39c59765
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 83 (RW.default.writeRpcServer.handler=30,queue=0,port=60020):
State: WAITING
Blocked count: 759971
Waited count: 854576
Waiting on java.util.LinkedList@778a8c58
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 82 (RW.default.writeRpcServer.handler=29,queue=9,port=60020):
State: BLOCKED
Blocked count: 828713
Waited count: 933633
Blocked on java.util.Properties@2b39f3bd
Blocked by 200 (RW.default.readRpcServer.handler=147,queue=18,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6741)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
Thread 81 (RW.default.writeRpcServer.handler=28,queue=8,port=60020):
State: WAITING
Blocked count: 761711
Waited count: 857046
Waiting on java.util.LinkedList@410c4b88
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 80 (RW.default.writeRpcServer.handler=27,queue=7,port=60020):
State: RUNNABLE
Blocked count: 769813
Waited count: 865793
Stack:
java.lang.Object.hashCode(Native Method)
java.util.concurrent.ConcurrentHashMap.replaceNode(ConcurrentHashMap.java:1106)
java.util.concurrent.ConcurrentHashMap.remove(ConcurrentHashMap.java:1097)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.close(HRegion.java:5422)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6109)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6741)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 79 (RW.default.writeRpcServer.handler=26,queue=6,port=60020):
State: RUNNABLE
Blocked count: 763205
Waited count: 858034
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 78 (RW.default.writeRpcServer.handler=25,queue=5,port=60020):
State: WAITING
Blocked count: 762289
Waited count: 856373
Waiting on java.util.LinkedList@36df20ed
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 77 (RW.default.writeRpcServer.handler=24,queue=4,port=60020):
State: RUNNABLE
Blocked count: 753915
Waited count: 848494
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 76 (RW.default.writeRpcServer.handler=23,queue=3,port=60020):
State: RUNNABLE
Blocked count: 759783
Waited count: 854008
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 75 (RW.default.writeRpcServer.handler=22,queue=2,port=60020):
State: BLOCKED
Blocked count: 767695
Waited count: 861431
Blocked on java.util.LinkedList@778a8c58
Blocked by 131 (RW.default.writeRpcServer.handler=78,queue=8,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 74 (RW.default.writeRpcServer.handler=21,queue=1,port=60020):
State: WAITING
Blocked count: 752309
Waited count: 845496
Waiting on java.util.LinkedList@39c59765
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 73 (RW.default.writeRpcServer.handler=20,queue=0,port=60020):
State: WAITING
Blocked count: 772975
Waited count: 867435
Waiting on java.util.LinkedList@36df20ed
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 72 (RW.default.writeRpcServer.handler=19,queue=9,port=60020):
State: WAITING
Blocked count: 813756
Waited count: 916507
Waiting on java.util.LinkedList@778a8c58
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 71 (RW.default.writeRpcServer.handler=18,queue=8,port=60020):
State: BLOCKED
Blocked count: 763910
Waited count: 858422
Blocked on java.util.LinkedList@778a8c58
Blocked by 146 (RW.default.writeRpcServer.handler=93,queue=3,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 70 (RW.default.writeRpcServer.handler=17,queue=7,port=60020):
State: WAITING
Blocked count: 764604
Waited count: 859782
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@168d009d
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 69 (RW.default.writeRpcServer.handler=16,queue=6,port=60020):
State: WAITING
Blocked count: 773034
Waited count: 867261
Waiting on java.util.LinkedList@243e1221
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 68 (RW.default.writeRpcServer.handler=15,queue=5,port=60020):
State: WAITING
Blocked count: 748930
Waited count: 844055
Waiting on java.util.LinkedList@778a8c58
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 67 (RW.default.writeRpcServer.handler=14,queue=4,port=60020):
State: WAITING
Blocked count: 750720
Waited count: 846255
Waiting on java.util.LinkedList@36df20ed
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 66 (RW.default.writeRpcServer.handler=13,queue=3,port=60020):
State: RUNNABLE
Blocked count: 762840
Waited count: 856650
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 65 (RW.default.writeRpcServer.handler=12,queue=2,port=60020):
State: WAITING
Blocked count: 750829
Waited count: 846521
Waiting on java.util.LinkedList@778a8c58
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 64 (RW.default.writeRpcServer.handler=11,queue=1,port=60020):
State: BLOCKED
Blocked count: 752122
Waited count: 845854
Blocked on java.util.Properties@2b39f3bd
Blocked by 171 (RW.default.readRpcServer.handler=118,queue=19,port=60020)
Stack:
java.util.Hashtable.get(Hashtable.java:362)
java.util.Properties.getProperty(Properties.java:969)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6741)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
Thread 63 (RW.default.writeRpcServer.handler=10,queue=0,port=60020):
State: WAITING
Blocked count: 776713
Waited count: 873050
Waiting on java.util.LinkedList@36df20ed
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 62 (RW.default.writeRpcServer.handler=9,queue=9,port=60020):
State: BLOCKED
Blocked count: 822831
Waited count: 926401
Blocked on org.apache.hadoop.conf.Configuration@54c562f7
Blocked by 155 (RW.default.readRpcServer.handler=102,queue=13,port=60020)
Stack:
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2315)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1346)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:136)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6741)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 61 (RW.default.writeRpcServer.handler=8,queue=8,port=60020):
State: WAITING
Blocked count: 766971
Waited count: 861598
Waiting on java.util.LinkedList@39c59765
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 60 (RW.default.writeRpcServer.handler=7,queue=7,port=60020):
State: WAITING
Blocked count: 745726
Waited count: 839008
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@168d009d
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 59 (RW.default.writeRpcServer.handler=6,queue=6,port=60020):
State: WAITING
Blocked count: 760833
Waited count: 856001
Waiting on java.util.LinkedList@36df20ed
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 58 (RW.default.writeRpcServer.handler=5,queue=5,port=60020):
State: BLOCKED
Blocked count: 765343
Waited count: 858443
Blocked on org.apache.hadoop.conf.Configuration@54c562f7
Blocked by 238 (RW.default.readRpcServer.handler=185,queue=16,port=60020)
Stack:
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2315)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1346)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:154)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6741)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 57 (RW.default.writeRpcServer.handler=4,queue=4,port=60020):
State: WAITING
Blocked count: 762568
Waited count: 856022
Waiting on java.util.LinkedList@39c59765
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 56 (RW.default.writeRpcServer.handler=3,queue=3,port=60020):
State: RUNNABLE
Blocked count: 745697
Waited count: 840532
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.completeMemstoreInsertWithSeqNum(MultiVersionConsistencyControl.java:127)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6893)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 55 (RW.default.writeRpcServer.handler=2,queue=2,port=60020):
State: BLOCKED
Blocked count: 757064
Waited count: 852466
Blocked on org.apache.hadoop.conf.Configuration@54c562f7
Blocked by 216 (RW.default.readRpcServer.handler=163,queue=14,port=60020)
Stack:
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2315)
org.apache.hadoop.conf.Configuration.get(Configuration.java:895)
org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:945)
org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1218)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:134)
org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:173)
org.apache.hadoop.hbase.regionserver.HStore.createScanner(HStore.java:2030)
org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:2020)
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5093)
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2402)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2388)
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2365)
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6105)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6741)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
Thread 54 (RW.default.writeRpcServer.handler=1,queue=1,port=60020):
State: BLOCKED
Blocked count: 744750
Waited count: 838590
Blocked on java.util.LinkedList@39c59765
Blocked by 124 (RW.default.writeRpcServer.handler=71,queue=1,port=60020)
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:224)
org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl.waitForPreviousTransactionsComplete(MultiVersionConsistencyControl.java:203)
org.apache.hadoop.hbase.regionserver.HRegion.increment(HRegion.java:6712)
org.apache.hadoop.hbase.regionserver.RSRpcServices.increment(RSRpcServices.java:501)
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:570)
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1901)
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31451)
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 53 (RW.default.writeRpcServer.handler=0,queue=0,port=60020):
State: WAITING
Blocked count: 773656
Waited count: 868823
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5f054e9
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:127)
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
java.lang.Thread.run(Thread.java:745)
Thread 30 (RpcServer.listener,port=60020):
State: RUNNABLE
Blocked count: 3610
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:678)
Thread 42 (RpcServer.responder):
State: RUNNABLE
Blocked count: 76
Waited count: 146
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:917)
org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:864)
Thread 52 (main-EventThread):
State: WAITING
Blocked count: 0
Waited count: 3
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@21d9540f
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
Thread 51 (main-SendThread(master2.example.local:2181)):
State: RUNNABLE
Blocked count: 444
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
Thread 48 (Thread-6):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
org.apache.hadoop.net.unix.DomainSocketWatcher.doPoll0(Native Method)
org.apache.hadoop.net.unix.DomainSocketWatcher.access$900(DomainSocketWatcher.java:52)
org.apache.hadoop.net.unix.DomainSocketWatcher$1.run(DomainSocketWatcher.java:474)
java.lang.Thread.run(Thread.java:745)
Thread 47 (MobFileCache #0):
State: TIMED_WAITING
Blocked count: 0
Waited count: 2
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 45 (LruBlockCacheStatsExecutor):
State: TIMED_WAITING
Blocked count: 0
Waited count: 13
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 43 (main.LruBlockCache.EvictionThread):
State: TIMED_WAITING
Blocked count: 0
Waited count: 385
Stack:
java.lang.Object.wait(Native Method)
org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread.run(LruBlockCache.java:791)
java.lang.Thread.run(Thread.java:745)
Thread 41 (Timer for 'HBase' metrics system):
State: TIMED_WAITING
Blocked count: 0
Waited count: 386
Stack:
java.lang.Object.wait(Native Method)
java.util.TimerThread.mainLoop(Timer.java:552)
java.util.TimerThread.run(Timer.java:505)
Thread 40 (RpcServer.reader=9,bindAddress=0.0.0.0,port=60020):
State: RUNNABLE
Blocked count: 4503
Waited count: 25353
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:575)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:562)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 39 (RpcServer.reader=8,bindAddress=0.0.0.0,port=60020):
State: RUNNABLE
Blocked count: 4519
Waited count: 25165
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:575)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:562)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 38 (RpcServer.reader=7,bindAddress=0.0.0.0,port=60020):
State: RUNNABLE
Blocked count: 4576
Waited count: 25443
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:575)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:562)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 37 (RpcServer.reader=6,bindAddress=0.0.0.0,port=60020):
State: RUNNABLE
Blocked count: 4464
Waited count: 25580
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:575)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:562)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 36 (RpcServer.reader=5,bindAddress=0.0.0.0,port=60020):
State: RUNNABLE
Blocked count: 4533
Waited count: 25420
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:575)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:562)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 35 (RpcServer.reader=4,bindAddress=0.0.0.0,port=60020):
State: RUNNABLE
Blocked count: 4520
Waited count: 24829
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:575)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:562)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 34 (RpcServer.reader=3,bindAddress=0.0.0.0,port=60020):
State: RUNNABLE
Blocked count: 4495
Waited count: 25744
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:575)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:562)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 33 (RpcServer.reader=2,bindAddress=0.0.0.0,port=60020):
State: RUNNABLE
Blocked count: 4552
Waited count: 26057
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:575)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:562)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 32 (RpcServer.reader=1,bindAddress=0.0.0.0,port=60020):
State: RUNNABLE
Blocked count: 4547
Waited count: 25621
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:575)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:562)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 31 (RpcServer.reader=0,bindAddress=0.0.0.0,port=60020):
State: RUNNABLE
Blocked count: 4634
Waited count: 25935
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:575)
org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:562)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 28 (New I/O worker #1):
State: RUNNABLE
Blocked count: 7
Waited count: 0
Stack:
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
com.cloudera.cmf.event.shaded.org.jboss.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:52)
com.cloudera.cmf.event.shaded.org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:208)
com.cloudera.cmf.event.shaded.org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
com.cloudera.cmf.event.shaded.org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)
com.cloudera.cmf.event.shaded.org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 27 (AvroEventStorePublisher-NettyThread-boss_1):
State: WAITING
Blocked count: 1
Waited count: 1
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3cce57c7
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 24 (EventStorePublisherWithRetry-0):
State: WAITING
Blocked count: 12
Waited count: 179
Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@33f676f6
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Thread 23 (event filterer):
State: TIMED_WAITING
Blocked count: 2
Waited count: 6277
Stack:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
com.cloudera.cmf.eventcatcher.client.logs.LogEventProcessor.runFiltering(LogEventProcessor.java:132)
com.cloudera.cmf.eventcatcher.client.logs.LogEventProcessor.access$000(LogEventProcessor.java:28)
com.cloudera.cmf.eventcatcher.client.logs.LogEventProcessor$1.run(LogEventProcessor.java:81)
Thread 22 (RMI TCP Accept-0):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
java.net.PlainSocketImpl.socketAccept(Native Method)
java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
java.net.ServerSocket.implAccept(ServerSocket.java:545)
java.net.ServerSocket.accept(ServerSocket.java:513)
sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept(LocalRMIServerSocketFactory.java:52)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:400)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:372)
java.lang.Thread.run(Thread.java:745)
Thread 21 (RMI TCP Accept-10102):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
java.net.PlainSocketImpl.socketAccept(Native Method)
java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
java.net.ServerSocket.implAccept(ServerSocket.java:545)
java.net.ServerSocket.accept(ServerSocket.java:513)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:400)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:372)
java.lang.Thread.run(Thread.java:745)
Thread 20 (RMI TCP Accept-0):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
java.net.PlainSocketImpl.socketAccept(Native Method)
java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
java.net.ServerSocket.implAccept(ServerSocket.java:545)
java.net.ServerSocket.accept(ServerSocket.java:513)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:400)
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:372)
java.lang.Thread.run(Thread.java:745)
Thread 5 (Signal Dispatcher):
State: RUNNABLE
Blocked count: 0
Waited count: 0
Stack:
Thread 3 (Finalizer):
State: WAITING
Blocked count: 3805
Waited count: 3351
Waiting on java.lang.ref.ReferenceQueue$Lock@264f218
Stack:
java.lang.Object.wait(Native Method)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
Thread 2 (Reference Handler):
State: WAITING
Blocked count: 2392
Waited count: 2263
Waiting on java.lang.ref.Reference$Lock@3576ddc2
Stack:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
Thread 1 (main):
State: WAITING
Blocked count: 13
Waited count: 6
Waiting on java.lang.Thread@57958426
Stack:
java.lang.Object.wait(Native Method)
java.lang.Thread.join(Thread.java:1245)
java.lang.Thread.join(Thread.java:1319)
org.apache.hadoop.hbase.util.HasThread.join(HasThread.java:89)
org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:66)
org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2540)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment