Skip to content

Instantly share code, notes, and snippets.

@fanf
Created April 14, 2023 06:55
Show Gist options
  • Save fanf/c16e9cd26c70fed50f8cf7930fba3394 to your computer and use it in GitHub Desktop.
Save fanf/c16e9cd26c70fed50f8cf7930fba3394 to your computer and use it in GitHub Desktop.
ZIO deadlock on multiple unsafe.run and foreachPar
2023-04-14 08:49:40
Full thread dump OpenJDK 64-Bit Server VM (11.0.18+10 mixed mode):
Threads class SMR info:
_java_thread_list=0x00007fd8b00016d0, length=56, elements={
0x00007fda0802c000, 0x00007fda09237800, 0x00007fda0923c000, 0x00007fda09251800,
0x00007fda09253800, 0x00007fda09255800, 0x00007fda09257800, 0x00007fda0925a000,
0x00007fda09746800, 0x00007fda09bb5800, 0x00007fd9b8001000, 0x00007fda09eb5800,
0x00007fda09fdb000, 0x00007fda09fdc000, 0x00007fda09fdd800, 0x00007fda09fdf000,
0x00007fda09fe1000, 0x00007fda09fe2800, 0x00007fda09fe4800, 0x00007fda09fe6000,
0x00007fda0a349800, 0x00007fda0a46e800, 0x00007fda0a768000, 0x00007fda0aba3800,
0x00007fda0aba5000, 0x00007fda0aba6000, 0x00007fda0aba7800, 0x00007fda0aba9800,
0x00007fda0abab000, 0x00007fda0abad000, 0x00007fda0abaf000, 0x00007fda0abb0800,
0x00007fda0abb2800, 0x00007fda0abb4000, 0x00007fda0abb6000, 0x00007fda0abbe800,
0x00007fda0abc0800, 0x00007fda0abc2000, 0x00007fda0abc4000, 0x00007fda0abc6000,
0x00007fda0acf8800, 0x00007fda0acff000, 0x00007fda0ad0e800, 0x00007fda0af86800,
0x00007fda0b1d1800, 0x00007fda0b1c9800, 0x00007fd90c002000, 0x00007fd8d4005000,
0x00007fd8d400a800, 0x00007fd8d400f800, 0x00007fd8d400c000, 0x00007fd978006800,
0x00007fd90c004000, 0x00007fd8a0001800, 0x00007fd978008800, 0x00007fd894001800
}
"main" #1 prio=5 os_prio=0 cpu=12489,09ms elapsed=35325,24s tid=0x00007fda0802c000 nid=0x26d00 in Object.wait() [0x00007fda0cffc000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(java.base@11.0.18/Native Method)
- waiting on <0x00000007ef247ed8> (a zio.internal.OneShot)
at java.lang.Object.wait(java.base@11.0.18/Object.java:328)
at zio.internal.OneShot.get(OneShot.scala:83)
- waiting to re-lock in wait() <0x00000007ef247ed8> (a zio.internal.OneShot)
at zio.Runtime$UnsafeAPIV1.run(Runtime.scala:139)
at com.normation.zio$ZioRuntime$.$anonfun$unsafeRun$1(ZioCommons.scala:458)
at com.normation.zio$ZioRuntime$$$Lambda$403/0x0000000800483040.apply(Unknown Source)
at zio.Unsafe$.unsafe(Unsafe.scala:37)
at com.normation.zio$ZioRuntime$.unsafeRun(ZioCommons.scala:458)
at bootstrap.liftweb.RudderConfig$.nodeFactRepository$lzycompute(RudderConfig.scala:1512)
- locked <0x00000007f805d090> (a bootstrap.liftweb.InitLock$)
- locked <0x00000007f6e42d28> (a bootstrap.liftweb.RudderConfig$)
at bootstrap.liftweb.RudderConfig$.nodeFactRepository(RudderConfig.scala:1461)
at bootstrap.liftweb.RudderConfig$.woFactNodeRepository$lzycompute(RudderConfig.scala:2334)
- locked <0x00000007f6e42d28> (a bootstrap.liftweb.RudderConfig$)
at bootstrap.liftweb.RudderConfig$.woFactNodeRepository(RudderConfig.scala:2334)
at bootstrap.liftweb.RudderConfig$.<clinit>(RudderConfig.scala:975)
at bootstrap.liftweb.LiftInitContextListener.contextInitialized(LiftInitContextListener.scala:134)
at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1002)
at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:553)
at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:931)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:764)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:360)
at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1445)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1409)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:844)
at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:275)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
- locked <0x000000068004ab58> (a java.lang.Object)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
- locked <0x000000068005c7b0> (a java.lang.Object)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
- locked <0x000000068005aee8> (a java.lang.Object)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.server.Server.start(Server.java:408)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at org.eclipse.jetty.server.Server.doStart(Server.java:372)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
- locked <0x000000068004ae90> (a java.lang.Object)
at org.eclipse.jetty.runner.Runner.run(Runner.java:520)
at org.eclipse.jetty.runner.Runner.main(Runner.java:565)
"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=2,01ms elapsed=35325,17s tid=0x00007fda09237800 nid=0x26d08 waiting on condition [0x00007fd9d16fc000]
java.lang.Thread.State: RUNNABLE
at java.lang.ref.Reference.waitForReferencePendingList(java.base@11.0.18/Native Method)
at java.lang.ref.Reference.processPendingReferences(java.base@11.0.18/Reference.java:241)
at java.lang.ref.Reference$ReferenceHandler.run(java.base@11.0.18/Reference.java:213)
"Finalizer" #3 daemon prio=8 os_prio=0 cpu=0,94ms elapsed=35325,17s tid=0x00007fda0923c000 nid=0x26d09 in Object.wait() [0x00007fd9d15fc000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(java.base@11.0.18/Native Method)
- waiting on <0x0000000680009f00> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(java.base@11.0.18/ReferenceQueue.java:155)
- waiting to re-lock in wait() <0x0000000680009f00> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(java.base@11.0.18/ReferenceQueue.java:176)
at java.lang.ref.Finalizer$FinalizerThread.run(java.base@11.0.18/Finalizer.java:170)
"Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=1,16ms elapsed=35325,16s tid=0x00007fda09251800 nid=0x26d0a runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Service Thread" #5 daemon prio=9 os_prio=0 cpu=0,18ms elapsed=35325,16s tid=0x00007fda09253800 nid=0x26d0b runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" #6 daemon prio=9 os_prio=0 cpu=7669,65ms elapsed=35325,16s tid=0x00007fda09255800 nid=0x26d0c waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
No compile task
"C1 CompilerThread0" #14 daemon prio=9 os_prio=0 cpu=14347,74ms elapsed=35325,16s tid=0x00007fda09257800 nid=0x26d0d waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
No compile task
"Sweeper thread" #18 daemon prio=9 os_prio=0 cpu=378,23ms elapsed=35325,16s tid=0x00007fda0925a000 nid=0x26d0e runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Common-Cleaner" #19 daemon prio=8 os_prio=0 cpu=4,02ms elapsed=35325,06s tid=0x00007fda09746800 nid=0x26d12 in Object.wait() [0x00007fd9d08fe000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(java.base@11.0.18/Native Method)
- waiting on <no object reference available>
at java.lang.ref.ReferenceQueue.remove(java.base@11.0.18/ReferenceQueue.java:155)
- waiting to re-lock in wait() <0x0000000680028500> (a java.lang.ref.ReferenceQueue$Lock)
at jdk.internal.ref.CleanerImpl.run(java.base@11.0.18/CleanerImpl.java:148)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
at jdk.internal.misc.InnocuousThread.run(java.base@11.0.18/InnocuousThread.java:161)
"Monitor Ctrl-Break" #20 daemon prio=5 os_prio=0 cpu=20,17ms elapsed=35324,71s tid=0x00007fda09bb5800 nid=0x26d15 runnable [0x00007fd9d03fe000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(java.base@11.0.18/Native Method)
at java.net.SocketInputStream.socketRead(java.base@11.0.18/SocketInputStream.java:115)
at java.net.SocketInputStream.read(java.base@11.0.18/SocketInputStream.java:168)
at java.net.SocketInputStream.read(java.base@11.0.18/SocketInputStream.java:140)
at sun.nio.cs.StreamDecoder.readBytes(java.base@11.0.18/StreamDecoder.java:284)
at sun.nio.cs.StreamDecoder.implRead(java.base@11.0.18/StreamDecoder.java:326)
at sun.nio.cs.StreamDecoder.read(java.base@11.0.18/StreamDecoder.java:178)
- locked <0x000000068000a3e0> (a java.io.InputStreamReader)
at java.io.InputStreamReader.read(java.base@11.0.18/InputStreamReader.java:181)
at java.io.BufferedReader.fill(java.base@11.0.18/BufferedReader.java:161)
at java.io.BufferedReader.readLine(java.base@11.0.18/BufferedReader.java:326)
- locked <0x000000068000a3e0> (a java.io.InputStreamReader)
at java.io.BufferedReader.readLine(java.base@11.0.18/BufferedReader.java:392)
at com.intellij.rt.execution.application.AppMainV2$1.run(AppMainV2.java:53)
"Attach Listener" #21 daemon prio=9 os_prio=0 cpu=58,45ms elapsed=35324,19s tid=0x00007fd9b8001000 nid=0x26d17 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"logback-1" #22 daemon prio=5 os_prio=0 cpu=7,21ms elapsed=35323,65s tid=0x00007fda09eb5800 nid=0x26d1b waiting on condition [0x00007fd9935fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680016960> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"qtp1239183618-24" #24 prio=5 os_prio=0 cpu=759,49ms elapsed=35322,83s tid=0x00007fda09fdb000 nid=0x26d1c waiting on condition [0x00007fd993ffd000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680000328> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"qtp1239183618-25" #25 prio=5 os_prio=0 cpu=1,94ms elapsed=35322,83s tid=0x00007fda09fdc000 nid=0x26d1d waiting on condition [0x00007fd993bfd000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680000328> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"qtp1239183618-26" #26 prio=5 os_prio=0 cpu=1,34ms elapsed=35322,83s tid=0x00007fda09fdd800 nid=0x26d1e waiting on condition [0x00007fd992ffd000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680000328> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"qtp1239183618-27" #27 prio=5 os_prio=0 cpu=1,33ms elapsed=35322,83s tid=0x00007fda09fdf000 nid=0x26d1f waiting on condition [0x00007fd9929fd000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680000328> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"qtp1239183618-28" #28 prio=5 os_prio=0 cpu=0,92ms elapsed=35322,83s tid=0x00007fda09fe1000 nid=0x26d20 waiting on condition [0x00007fd9928fd000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680000328> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"qtp1239183618-29" #29 prio=5 os_prio=0 cpu=0,82ms elapsed=35322,83s tid=0x00007fda09fe2800 nid=0x26d21 waiting on condition [0x00007fd9927fd000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680000328> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"qtp1239183618-30" #30 prio=5 os_prio=0 cpu=1,51ms elapsed=35322,83s tid=0x00007fda09fe4800 nid=0x26d22 waiting on condition [0x00007fd9926fd000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680000328> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"qtp1239183618-31" #31 prio=5 os_prio=0 cpu=0,98ms elapsed=35322,83s tid=0x00007fda09fe6000 nid=0x26d23 waiting on condition [0x00007fd9925fd000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680000328> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"Session-HouseKeeper-781f10f2-1" #32 prio=5 os_prio=0 cpu=0,65ms elapsed=35320,88s tid=0x00007fda0a349800 nid=0x26d34 waiting on condition [0x00007fd990ee1000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x00000007f766e420> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1182)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"BC Entropy Daemon" #34 daemon prio=5 os_prio=0 cpu=5,35ms elapsed=35320,10s tid=0x00007fda0a46e800 nid=0x26d35 waiting on condition [0x00007fd990be2000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(java.base@11.0.18/Native Method)
at org.bouncycastle.jcajce.provider.drbg.DRBG$EntropyDaemon.run(Unknown Source)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"logback-2" #35 daemon prio=5 os_prio=0 cpu=23,83ms elapsed=35319,46s tid=0x00007fda0a768000 nid=0x26d36 waiting on condition [0x00007fd9901fd000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680016960> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1182)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"ZScheduler-Worker-0" #36 daemon prio=5 os_prio=0 cpu=4,36ms elapsed=35318,68s tid=0x00007fda0aba3800 nid=0x26d37 in Object.wait() [0x00007fd9924fd000]
java.lang.Thread.State: RUNNABLE
at bootstrap.liftweb.RudderConfig$$$Lambda$1395/0x00000008008b1040.apply(Unknown Source)
at zio.ZIO$.$anonfun$foreachPar$7(ZIO.scala:5843)
at zio.ZIO$$$Lambda$1403/0x00000008008b5040.apply(Unknown Source)
at zio.ZIO$.$anonfun$foreachParDiscard$4(ZIO.scala:5861)
at zio.ZIO$$$Lambda$1427/0x00000008008c8840.apply(Unknown Source)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:1024)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967)
at zio.internal.FiberRuntime.evaluateEffect(FiberRuntime.scala:381)
at zio.internal.FiberRuntime.evaluateMessageWhileSuspended(FiberRuntime.scala:504)
at zio.internal.FiberRuntime.drainQueueOnCurrentThread(FiberRuntime.scala:220)
at zio.internal.FiberRuntime.run(FiberRuntime.scala:139)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:476)
"ZScheduler-Worker-1" #37 daemon prio=5 os_prio=0 cpu=2,59ms elapsed=35318,68s tid=0x00007fda0aba5000 nid=0x26d38 in Object.wait() [0x00007fd9923fd000]
java.lang.Thread.State: RUNNABLE
at bootstrap.liftweb.RudderConfig$$$Lambda$1395/0x00000008008b1040.apply(Unknown Source)
at zio.ZIO$.$anonfun$foreachPar$7(ZIO.scala:5843)
at zio.ZIO$$$Lambda$1403/0x00000008008b5040.apply(Unknown Source)
at zio.ZIO$.$anonfun$foreachParDiscard$4(ZIO.scala:5861)
at zio.ZIO$$$Lambda$1427/0x00000008008c8840.apply(Unknown Source)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:1024)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967)
at zio.internal.FiberRuntime.evaluateEffect(FiberRuntime.scala:381)
at zio.internal.FiberRuntime.evaluateMessageWhileSuspended(FiberRuntime.scala:504)
at zio.internal.FiberRuntime.drainQueueOnCurrentThread(FiberRuntime.scala:220)
at zio.internal.FiberRuntime.run(FiberRuntime.scala:139)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:476)
"ZScheduler-Worker-2" #38 daemon prio=5 os_prio=0 cpu=2,00ms elapsed=35318,68s tid=0x00007fda0aba6000 nid=0x26d39 in Object.wait() [0x00007fd9900fd000]
java.lang.Thread.State: RUNNABLE
at bootstrap.liftweb.RudderConfig$$$Lambda$1395/0x00000008008b1040.apply(Unknown Source)
at zio.ZIO$.$anonfun$foreachPar$7(ZIO.scala:5843)
at zio.ZIO$$$Lambda$1403/0x00000008008b5040.apply(Unknown Source)
at zio.ZIO$.$anonfun$foreachParDiscard$4(ZIO.scala:5861)
at zio.ZIO$$$Lambda$1427/0x00000008008c8840.apply(Unknown Source)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:1024)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967)
at zio.internal.FiberRuntime.evaluateEffect(FiberRuntime.scala:381)
at zio.internal.FiberRuntime.evaluateMessageWhileSuspended(FiberRuntime.scala:504)
at zio.internal.FiberRuntime.drainQueueOnCurrentThread(FiberRuntime.scala:220)
at zio.internal.FiberRuntime.run(FiberRuntime.scala:139)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:476)
"ZScheduler-Worker-3" #39 daemon prio=5 os_prio=0 cpu=1,98ms elapsed=35318,68s tid=0x00007fda0aba7800 nid=0x26d3a in Object.wait() [0x00007fd9135fd000]
java.lang.Thread.State: RUNNABLE
at bootstrap.liftweb.RudderConfig$$$Lambda$1395/0x00000008008b1040.apply(Unknown Source)
at zio.ZIO$.$anonfun$foreachPar$7(ZIO.scala:5843)
at zio.ZIO$$$Lambda$1403/0x00000008008b5040.apply(Unknown Source)
at zio.ZIO$.$anonfun$foreachParDiscard$4(ZIO.scala:5861)
at zio.ZIO$$$Lambda$1427/0x00000008008c8840.apply(Unknown Source)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:1024)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967)
at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967)
at zio.internal.FiberRuntime.evaluateEffect(FiberRuntime.scala:381)
at zio.internal.FiberRuntime.evaluateMessageWhileSuspended(FiberRuntime.scala:504)
at zio.internal.FiberRuntime.drainQueueOnCurrentThread(FiberRuntime.scala:220)
at zio.internal.FiberRuntime.run(FiberRuntime.scala:139)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:476)
"ZScheduler-Worker-4" #40 daemon prio=5 os_prio=0 cpu=0,49ms elapsed=35318,68s tid=0x00007fda0aba9800 nid=0x26d3b waiting on condition [0x00007fd9134fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"ZScheduler-Worker-5" #41 daemon prio=5 os_prio=0 cpu=2,08ms elapsed=35318,68s tid=0x00007fda0abab000 nid=0x26d3c waiting on condition [0x00007fd9133fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"ZScheduler-Worker-6" #42 daemon prio=5 os_prio=0 cpu=0,36ms elapsed=35318,68s tid=0x00007fda0abad000 nid=0x26d3d waiting on condition [0x00007fd9132fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"ZScheduler-Worker-7" #43 daemon prio=5 os_prio=0 cpu=0,22ms elapsed=35318,68s tid=0x00007fda0abaf000 nid=0x26d3e waiting on condition [0x00007fd9131fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"ZScheduler-Worker-8" #44 daemon prio=5 os_prio=0 cpu=0,38ms elapsed=35318,68s tid=0x00007fda0abb0800 nid=0x26d3f waiting on condition [0x00007fd9130fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"ZScheduler-Worker-9" #45 daemon prio=5 os_prio=0 cpu=0,38ms elapsed=35318,68s tid=0x00007fda0abb2800 nid=0x26d40 waiting on condition [0x00007fd912ffe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"ZScheduler-Worker-10" #46 daemon prio=5 os_prio=0 cpu=0,23ms elapsed=35318,68s tid=0x00007fda0abb4000 nid=0x26d41 waiting on condition [0x00007fd912efe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"ZScheduler-Worker-11" #47 daemon prio=5 os_prio=0 cpu=0,28ms elapsed=35318,68s tid=0x00007fda0abb6000 nid=0x26d42 waiting on condition [0x00007fd912dfe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"ZScheduler-Worker-12" #48 daemon prio=5 os_prio=0 cpu=0,31ms elapsed=35318,68s tid=0x00007fda0abbe800 nid=0x26d43 waiting on condition [0x00007fd912cfe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"ZScheduler-Worker-13" #49 daemon prio=5 os_prio=0 cpu=0,28ms elapsed=35318,68s tid=0x00007fda0abc0800 nid=0x26d44 waiting on condition [0x00007fd912bfe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"ZScheduler-Worker-14" #50 daemon prio=5 os_prio=0 cpu=0,27ms elapsed=35318,68s tid=0x00007fda0abc2000 nid=0x26d45 waiting on condition [0x00007fd912afe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"ZScheduler-Worker-15" #51 daemon prio=5 os_prio=0 cpu=0,33ms elapsed=35318,68s tid=0x00007fda0abc4000 nid=0x26d46 waiting on condition [0x00007fd9129fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"ZScheduler-Supervisor" #52 daemon prio=5 os_prio=0 cpu=291,48ms elapsed=35318,68s tid=0x00007fda0abc6000 nid=0x26d47 waiting on condition [0x00007fd9128fe000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.parkUntil(java.base@11.0.18/LockSupport.java:391)
at zio.internal.ZScheduler$$anon$3.run(ZScheduler.scala:316)
"Write Timeout Handler Timer" #53 daemon prio=5 os_prio=0 cpu=511,03ms elapsed=35318,55s tid=0x00007fda0acf8800 nid=0x26d48 in Object.wait() [0x00007fd9125fe000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(java.base@11.0.18/Native Method)
- waiting on <no object reference available>
at java.util.TimerThread.mainLoop(java.base@11.0.18/Timer.java:553)
- waiting to re-lock in wait() <0x00000007f7713e80> (a java.util.TaskQueue)
at java.util.TimerThread.run(java.base@11.0.18/Timer.java:506)
"Connection reader for connection 0 in pool 'rudder-authenticated-ro' to localhost:1389" #55 daemon prio=5 os_prio=0 cpu=426,76ms elapsed=35318,55s tid=0x00007fda0acff000 nid=0x26d4a runnable [0x00007fd9124fe000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(java.base@11.0.18/Native Method)
at java.net.SocketInputStream.socketRead(java.base@11.0.18/SocketInputStream.java:115)
at java.net.SocketInputStream.read(java.base@11.0.18/SocketInputStream.java:168)
at java.net.SocketInputStream.read(java.base@11.0.18/SocketInputStream.java:140)
at java.io.BufferedInputStream.fill(java.base@11.0.18/BufferedInputStream.java:252)
at java.io.BufferedInputStream.read(java.base@11.0.18/BufferedInputStream.java:271)
- locked <0x00000007f76dd9d8> (a java.io.BufferedInputStream)
at com.unboundid.asn1.ASN1StreamReader.read(ASN1StreamReader.java:1199)
at com.unboundid.asn1.ASN1StreamReader.readType(ASN1StreamReader.java:360)
at com.unboundid.asn1.ASN1StreamReader.beginSequence(ASN1StreamReader.java:1118)
at com.unboundid.ldap.protocol.LDAPMessage.readLDAPResponseFrom(LDAPMessage.java:1198)
at com.unboundid.ldap.sdk.LDAPConnectionReader.run(LDAPConnectionReader.java:242)
"Health Check Thread for LDAPConnectionPool(serverSet=SingleServerSet(server=localhost:1389, includesAuthentication=false, includesPostConnectProcessing=false), maxConnections=2)" #56 daemon prio=5 os_prio=0 cpu=3,07ms elapsed=35318,50s tid=0x00007fda0ad0e800 nid=0x26d4b waiting on condition [0x00007fd9123fd000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x00000007f77efe98> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
at java.util.concurrent.LinkedBlockingQueue.poll(java.base@11.0.18/LinkedBlockingQueue.java:458)
at com.unboundid.ldap.sdk.LDAPConnectionPoolHealthCheckThread.run(LDAPConnectionPoolHealthCheckThread.java:137)
"HikariPool-1 housekeeper" #57 daemon prio=5 os_prio=0 cpu=4,91ms elapsed=35317,90s tid=0x00007fda0af86800 nid=0x26d4c waiting on condition [0x00007fd911cfd000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x00000007f77b8d40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1182)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"Connection reader for connection 1 in pool 'rudder-authenticated-rw' to localhost:1389" #64 daemon prio=5 os_prio=0 cpu=8,46ms elapsed=35317,19s tid=0x00007fda0b1d1800 nid=0x26d56 runnable [0x00007fd9115fe000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(java.base@11.0.18/Native Method)
at java.net.SocketInputStream.socketRead(java.base@11.0.18/SocketInputStream.java:115)
at java.net.SocketInputStream.read(java.base@11.0.18/SocketInputStream.java:168)
at java.net.SocketInputStream.read(java.base@11.0.18/SocketInputStream.java:140)
at java.io.BufferedInputStream.fill(java.base@11.0.18/BufferedInputStream.java:252)
at java.io.BufferedInputStream.read(java.base@11.0.18/BufferedInputStream.java:271)
- locked <0x00000007f7783240> (a java.io.BufferedInputStream)
at com.unboundid.asn1.ASN1StreamReader.read(ASN1StreamReader.java:1199)
at com.unboundid.asn1.ASN1StreamReader.readType(ASN1StreamReader.java:360)
at com.unboundid.asn1.ASN1StreamReader.beginSequence(ASN1StreamReader.java:1118)
at com.unboundid.ldap.protocol.LDAPMessage.readLDAPResponseFrom(LDAPMessage.java:1198)
at com.unboundid.ldap.sdk.LDAPConnectionReader.run(LDAPConnectionReader.java:242)
"Health Check Thread for LDAPConnectionPool(serverSet=SingleServerSet(server=localhost:1389, includesAuthentication=false, includesPostConnectProcessing=false), maxConnections=2)" #65 daemon prio=5 os_prio=0 cpu=2,35ms elapsed=35317,19s tid=0x00007fda0b1c9800 nid=0x26d57 waiting on condition [0x00007fd9112fd000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x00000007f77140d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
at java.util.concurrent.LinkedBlockingQueue.poll(java.base@11.0.18/LinkedBlockingQueue.java:458)
at com.unboundid.ldap.sdk.LDAPConnectionPoolHealthCheckThread.run(LDAPConnectionPoolHealthCheckThread.java:137)
"logback-3" #67 daemon prio=5 os_prio=0 cpu=1,73ms elapsed=35314,46s tid=0x00007fd90c002000 nid=0x26d5e waiting on condition [0x00007fd910afe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680016960> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"ZScheduler-0" #68 daemon prio=5 os_prio=0 cpu=0,29ms elapsed=35309,98s tid=0x00007fd8d4005000 nid=0x26d61 waiting on condition [0x00007fd911bfe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"ZScheduler-1" #69 daemon prio=5 os_prio=0 cpu=0,29ms elapsed=35309,98s tid=0x00007fd8d400a800 nid=0x26d62 waiting on condition [0x00007fd9104fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"ZScheduler-2" #70 daemon prio=5 os_prio=0 cpu=0,31ms elapsed=35309,98s tid=0x00007fd8d400f800 nid=0x26d63 waiting on condition [0x00007fd9103fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"ZScheduler-3" #71 daemon prio=5 os_prio=0 cpu=0,28ms elapsed=35309,98s tid=0x00007fd8d400c000 nid=0x26d64 waiting on condition [0x00007fd9102fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)
"logback-4" #72 daemon prio=5 os_prio=0 cpu=1,81ms elapsed=35309,46s tid=0x00007fd978006800 nid=0x26d65 waiting on condition [0x00007fd9101fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680016960> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"logback-5" #73 daemon prio=5 os_prio=0 cpu=1,38ms elapsed=35304,46s tid=0x00007fd90c004000 nid=0x26d67 waiting on condition [0x00007fd9d01fc000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680016960> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"logback-6" #74 daemon prio=5 os_prio=0 cpu=1,29ms elapsed=35299,46s tid=0x00007fd8a0001800 nid=0x26d68 waiting on condition [0x00007fd9d0bfe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680016960> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"logback-7" #75 daemon prio=5 os_prio=0 cpu=1,53ms elapsed=35294,46s tid=0x00007fd978008800 nid=0x26d6c waiting on condition [0x00007fd9109fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680016960> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"logback-8" #76 daemon prio=5 os_prio=0 cpu=1,15ms elapsed=35289,46s tid=0x00007fd894001800 nid=0x26d6d waiting on condition [0x00007fd9116fe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
- parking to wait for <0x0000000680016960> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
"VM Thread" os_prio=0 cpu=59,93ms elapsed=35325,18s tid=0x00007fda0922f800 nid=0x26d07 runnable
"GC Thread#0" os_prio=0 cpu=76,16ms elapsed=35325,24s tid=0x00007fda08045800 nid=0x26d01 runnable
"GC Thread#1" os_prio=0 cpu=65,89ms elapsed=35320,94s tid=0x00007fd9bc071000 nid=0x26d26 runnable
"GC Thread#2" os_prio=0 cpu=59,53ms elapsed=35320,94s tid=0x00007fd9bc072800 nid=0x26d27 runnable
"GC Thread#3" os_prio=0 cpu=49,84ms elapsed=35320,94s tid=0x00007fd9bc074000 nid=0x26d28 runnable
"GC Thread#4" os_prio=0 cpu=49,98ms elapsed=35320,94s tid=0x00007fd9bc075800 nid=0x26d29 runnable
"GC Thread#5" os_prio=0 cpu=68,39ms elapsed=35320,94s tid=0x00007fd9bc077800 nid=0x26d2a runnable
"GC Thread#6" os_prio=0 cpu=46,51ms elapsed=35320,94s tid=0x00007fd9bc079000 nid=0x26d2b runnable
"GC Thread#7" os_prio=0 cpu=48,79ms elapsed=35320,94s tid=0x00007fd9bc07a800 nid=0x26d2c runnable
"GC Thread#8" os_prio=0 cpu=63,71ms elapsed=35320,94s tid=0x00007fd9bc07c000 nid=0x26d2d runnable
"GC Thread#9" os_prio=0 cpu=62,03ms elapsed=35320,94s tid=0x00007fd9bc07d800 nid=0x26d2e runnable
"GC Thread#10" os_prio=0 cpu=48,83ms elapsed=35320,94s tid=0x00007fd9bc07f000 nid=0x26d2f runnable
"GC Thread#11" os_prio=0 cpu=62,77ms elapsed=35320,94s tid=0x00007fd9bc080800 nid=0x26d30 runnable
"GC Thread#12" os_prio=0 cpu=63,02ms elapsed=35320,93s tid=0x00007fd9bc082000 nid=0x26d31 runnable
"G1 Main Marker" os_prio=0 cpu=3,63ms elapsed=35325,24s tid=0x00007fda080a4000 nid=0x26d02 runnable
"G1 Conc#0" os_prio=0 cpu=58,81ms elapsed=35325,24s tid=0x00007fda080a6000 nid=0x26d03 runnable
"G1 Conc#1" os_prio=0 cpu=56,04ms elapsed=35320,91s tid=0x00007fd9d4001000 nid=0x26d32 runnable
"G1 Conc#2" os_prio=0 cpu=58,71ms elapsed=35320,91s tid=0x00007fd9d4002800 nid=0x26d33 runnable
"G1 Refine#0" os_prio=0 cpu=8,95ms elapsed=35325,21s tid=0x00007fda0915e000 nid=0x26d04 runnable
"G1 Young RemSet Sampling" os_prio=0 cpu=147,88ms elapsed=35325,20s tid=0x00007fda09160000 nid=0x26d05 runnable
"StrDedup" os_prio=0 cpu=40,53ms elapsed=35325,20s tid=0x00007fda09168000 nid=0x26d06 runnable
"VM Periodic Task Thread" os_prio=0 cpu=372,16ms elapsed=35324,71s tid=0x00007fda09cc0000 nid=0x26d16 waiting on condition
JNI global refs: 25, weak refs: 0
@fanf
Copy link
Author

fanf commented Apr 14, 2023

Interestingly (I guess?) if I change unsafeRun to add the blocking tag, I get the same deadlock on the very first call of unsafeRun, with a similar thread dump (but simpler):
image

The main difference is the presence of zio-default-blocking related to LDAPConnectionProvider.scala:82
image

2023-04-14 09:34:49
Full thread dump OpenJDK 64-Bit Server VM (11.0.18+10 mixed mode):

Threads class SMR info:
_java_thread_list=0x00007fd58aa12040, length=50, elements={
0x00007fd58802c000, 0x00007fd589237800, 0x00007fd58923c000, 0x00007fd589251800,
0x00007fd589253800, 0x00007fd589255800, 0x00007fd589257800, 0x00007fd58925a000,
0x00007fd589557000, 0x00007fd589b32000, 0x00007fd589cbd000, 0x00007fd589ea1000,
0x00007fd589ea2800, 0x00007fd589ea4800, 0x00007fd589ea6800, 0x00007fd589ea8800,
0x00007fd589eaa000, 0x00007fd589eac000, 0x00007fd589eae000, 0x00007fd538001000,
0x00007fd58a0be000, 0x00007fd58a1e7800, 0x00007fd58a316000, 0x00007fd58a8d0800,
0x00007fd58a8d2800, 0x00007fd58a8d4000, 0x00007fd58a8d6000, 0x00007fd58a8d7800,
0x00007fd58a8d9800, 0x00007fd58a8db800, 0x00007fd58a8dd000, 0x00007fd58a8df000,
0x00007fd58a8e0800, 0x00007fd58a8e2800, 0x00007fd58a8e8000, 0x00007fd58a8ea000,
0x00007fd58a8eb800, 0x00007fd58a8ed800, 0x00007fd58a8ef800, 0x00007fd58a8f1800,
0x00007fd440002000, 0x00007fd444048000, 0x00007fd444056800, 0x00007fd44406d000,
0x00007fd500002800, 0x00007fd48c001800, 0x00007fd500003800, 0x00007fd52087c000,
0x00007fd48c002800, 0x00007fd51cb0d000
}

"main" #1 prio=5 os_prio=0 cpu=2249,51ms elapsed=115,88s tid=0x00007fd58802c000 nid=0x279c9 in Object.wait()  [0x00007fd58d314000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.18/Native Method)
	- waiting on <0x00000007fe2cd4d0> (a zio.internal.OneShot)
	at java.lang.Object.wait(java.base@11.0.18/Object.java:328)
	at zio.internal.OneShot.get(OneShot.scala:83)
	- waiting to re-lock in wait() <0x00000007fe2cd4d0> (a zio.internal.OneShot)
	at zio.Runtime$UnsafeAPIV1.run(Runtime.scala:139)
	at com.normation.zio$ZioRuntime$.$anonfun$unsafeRun$1(ZioCommons.scala:458)
	at com.normation.zio$ZioRuntime$$$Lambda$403/0x0000000800483040.apply(Unknown Source)
	at zio.Unsafe$.unsafe(Unsafe.scala:37)
	at com.normation.zio$ZioRuntime$.unsafeRun(ZioCommons.scala:458)
	at bootstrap.liftweb.RudderConfig$.<clinit>(RudderConfig.scala:966)
	at bootstrap.liftweb.LiftInitContextListener.contextInitialized(LiftInitContextListener.scala:134)
	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1002)
	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:553)
	at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:931)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:764)
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:360)
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1445)
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1409)
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:844)
	at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:275)
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
	- locked <0x00000007fac5c6b8> (a java.lang.Object)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
	- locked <0x00000007fac6b140> (a java.lang.Object)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
	- locked <0x00000007fac6aac0> (a java.lang.Object)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
	at org.eclipse.jetty.server.Server.start(Server.java:408)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
	at org.eclipse.jetty.server.Server.doStart(Server.java:372)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
	- locked <0x00000007fac5ca38> (a java.lang.Object)
	at org.eclipse.jetty.runner.Runner.run(Runner.java:520)
	at org.eclipse.jetty.runner.Runner.main(Runner.java:565)

"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=0,96ms elapsed=115,85s tid=0x00007fd589237800 nid=0x279d1 waiting on condition  [0x00007fd55c1b8000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ref.Reference.waitForReferencePendingList(java.base@11.0.18/Native Method)
	at java.lang.ref.Reference.processPendingReferences(java.base@11.0.18/Reference.java:241)
	at java.lang.ref.Reference$ReferenceHandler.run(java.base@11.0.18/Reference.java:213)

"Finalizer" #3 daemon prio=8 os_prio=0 cpu=0,54ms elapsed=115,85s tid=0x00007fd58923c000 nid=0x279d2 in Object.wait()  [0x00007fd5557fe000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.18/Native Method)
	- waiting on <0x00000007fac3ac68> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@11.0.18/ReferenceQueue.java:155)
	- waiting to re-lock in wait() <0x00000007fac3ac68> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@11.0.18/ReferenceQueue.java:176)
	at java.lang.ref.Finalizer$FinalizerThread.run(java.base@11.0.18/Finalizer.java:170)

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0,79ms elapsed=115,84s tid=0x00007fd589251800 nid=0x279d3 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Service Thread" #5 daemon prio=9 os_prio=0 cpu=0,10ms elapsed=115,84s tid=0x00007fd589253800 nid=0x279d4 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #6 daemon prio=9 os_prio=0 cpu=1460,44ms elapsed=115,84s tid=0x00007fd589255800 nid=0x279d5 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C1 CompilerThread0" #14 daemon prio=9 os_prio=0 cpu=3052,56ms elapsed=115,84s tid=0x00007fd589257800 nid=0x279d6 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"Sweeper thread" #18 daemon prio=9 os_prio=0 cpu=41,37ms elapsed=115,84s tid=0x00007fd58925a000 nid=0x279d7 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" #19 daemon prio=8 os_prio=0 cpu=1,01ms elapsed=115,80s tid=0x00007fd589557000 nid=0x279db in Object.wait()  [0x00007fd554afe000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.18/Native Method)
	- waiting on <no object reference available>
	at java.lang.ref.ReferenceQueue.remove(java.base@11.0.18/ReferenceQueue.java:155)
	- waiting to re-lock in wait() <0x00000007fac58678> (a java.lang.ref.ReferenceQueue$Lock)
	at jdk.internal.ref.CleanerImpl.run(java.base@11.0.18/CleanerImpl.java:148)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
	at jdk.internal.misc.InnocuousThread.run(java.base@11.0.18/InnocuousThread.java:161)

"Monitor Ctrl-Break" #20 daemon prio=5 os_prio=0 cpu=11,98ms elapsed=115,66s tid=0x00007fd589b32000 nid=0x279de runnable  [0x00007fd5547fe000]
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(java.base@11.0.18/Native Method)
	at java.net.SocketInputStream.socketRead(java.base@11.0.18/SocketInputStream.java:115)
	at java.net.SocketInputStream.read(java.base@11.0.18/SocketInputStream.java:168)
	at java.net.SocketInputStream.read(java.base@11.0.18/SocketInputStream.java:140)
	at sun.nio.cs.StreamDecoder.readBytes(java.base@11.0.18/StreamDecoder.java:284)
	at sun.nio.cs.StreamDecoder.implRead(java.base@11.0.18/StreamDecoder.java:326)
	at sun.nio.cs.StreamDecoder.read(java.base@11.0.18/StreamDecoder.java:178)
	- locked <0x00000007fac23d00> (a java.io.InputStreamReader)
	at java.io.InputStreamReader.read(java.base@11.0.18/InputStreamReader.java:181)
	at java.io.BufferedReader.fill(java.base@11.0.18/BufferedReader.java:161)
	at java.io.BufferedReader.readLine(java.base@11.0.18/BufferedReader.java:326)
	- locked <0x00000007fac23d00> (a java.io.InputStreamReader)
	at java.io.BufferedReader.readLine(java.base@11.0.18/BufferedReader.java:392)
	at com.intellij.rt.execution.application.AppMainV2$1.run(AppMainV2.java:53)

"logback-1" #21 daemon prio=5 os_prio=0 cpu=7,01ms elapsed=115,44s tid=0x00007fd589cbd000 nid=0x279e0 waiting on condition  [0x00007fd50bffe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007facb0820> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"qtp574268151-23" #23 prio=5 os_prio=0 cpu=395,04ms elapsed=115,19s tid=0x00007fd589ea1000 nid=0x279e1 waiting on condition  [0x00007fd50bbfe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007fac75ee8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"qtp574268151-24" #24 prio=5 os_prio=0 cpu=0,87ms elapsed=115,19s tid=0x00007fd589ea2800 nid=0x279e2 waiting on condition  [0x00007fd50b5fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007fac75ee8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"qtp574268151-25" #25 prio=5 os_prio=0 cpu=0,62ms elapsed=115,19s tid=0x00007fd589ea4800 nid=0x279e3 waiting on condition  [0x00007fd50b1fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007fac75ee8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"qtp574268151-26" #26 prio=5 os_prio=0 cpu=0,64ms elapsed=115,19s tid=0x00007fd589ea6800 nid=0x279e4 waiting on condition  [0x00007fd50abfe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007fac75ee8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"qtp574268151-27" #27 prio=5 os_prio=0 cpu=0,47ms elapsed=115,19s tid=0x00007fd589ea8800 nid=0x279e5 waiting on condition  [0x00007fd50aafe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007fac75ee8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"qtp574268151-28" #28 prio=5 os_prio=0 cpu=0,51ms elapsed=115,19s tid=0x00007fd589eaa000 nid=0x279e6 waiting on condition  [0x00007fd50a9fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007fac75ee8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"qtp574268151-29" #29 prio=5 os_prio=0 cpu=0,29ms elapsed=115,19s tid=0x00007fd589eac000 nid=0x279e7 waiting on condition  [0x00007fd50a8fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007fac75ee8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"qtp574268151-30" #30 prio=5 os_prio=0 cpu=0,27ms elapsed=115,19s tid=0x00007fd589eae000 nid=0x279e8 waiting on condition  [0x00007fd50a7fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007fac75ee8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"Attach Listener" #31 daemon prio=9 os_prio=0 cpu=3,47ms elapsed=114,85s tid=0x00007fd538001000 nid=0x279ea waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Session-HouseKeeper-e98770d-1" #32 prio=5 os_prio=0 cpu=0,18ms elapsed=114,21s tid=0x00007fd58a0be000 nid=0x279fa waiting on condition  [0x00007fd5090e2000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007fac76110> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1182)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"BC Entropy Daemon" #34 daemon prio=5 os_prio=0 cpu=0,99ms elapsed=113,87s tid=0x00007fd58a1e7800 nid=0x279fb waiting on condition  [0x00007fd508de2000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(java.base@11.0.18/Native Method)
	at org.bouncycastle.jcajce.provider.drbg.DRBG$EntropyDaemon.run(Unknown Source)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"logback-2" #35 daemon prio=5 os_prio=0 cpu=1,33ms elapsed=113,57s tid=0x00007fd58a316000 nid=0x279fc waiting on condition  [0x00007fd5083fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007facb0820> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"ZScheduler-Worker-0" #36 daemon prio=5 os_prio=0 cpu=0,63ms elapsed=113,16s tid=0x00007fd58a8d0800 nid=0x279fd waiting on condition  [0x00007fd5080fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Worker-1" #37 daemon prio=5 os_prio=0 cpu=0,49ms elapsed=113,16s tid=0x00007fd58a8d2800 nid=0x279fe waiting on condition  [0x00007fd4937fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Worker-2" #38 daemon prio=5 os_prio=0 cpu=0,26ms elapsed=113,16s tid=0x00007fd58a8d4000 nid=0x279ff waiting on condition  [0x00007fd4936fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Worker-3" #39 daemon prio=5 os_prio=0 cpu=0,30ms elapsed=113,16s tid=0x00007fd58a8d6000 nid=0x27a00 waiting on condition  [0x00007fd4935fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Worker-4" #40 daemon prio=5 os_prio=0 cpu=0,39ms elapsed=113,16s tid=0x00007fd58a8d7800 nid=0x27a01 waiting on condition  [0x00007fd4934fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Worker-5" #41 daemon prio=5 os_prio=0 cpu=0,40ms elapsed=113,16s tid=0x00007fd58a8d9800 nid=0x27a02 waiting on condition  [0x00007fd4933fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Worker-6" #42 daemon prio=5 os_prio=0 cpu=0,29ms elapsed=113,16s tid=0x00007fd58a8db800 nid=0x27a03 waiting on condition  [0x00007fd4932fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Worker-7" #43 daemon prio=5 os_prio=0 cpu=0,26ms elapsed=113,15s tid=0x00007fd58a8dd000 nid=0x27a04 waiting on condition  [0x00007fd4931fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Worker-8" #44 daemon prio=5 os_prio=0 cpu=0,17ms elapsed=113,15s tid=0x00007fd58a8df000 nid=0x27a05 waiting on condition  [0x00007fd4930fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Worker-9" #45 daemon prio=5 os_prio=0 cpu=0,22ms elapsed=113,15s tid=0x00007fd58a8e0800 nid=0x27a06 waiting on condition  [0x00007fd492ffe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Worker-10" #46 daemon prio=5 os_prio=0 cpu=0,23ms elapsed=113,16s tid=0x00007fd58a8e2800 nid=0x27a07 waiting on condition  [0x00007fd492efe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Worker-11" #47 daemon prio=5 os_prio=0 cpu=0,17ms elapsed=113,16s tid=0x00007fd58a8e8000 nid=0x27a08 waiting on condition  [0x00007fd492dfe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Worker-12" #48 daemon prio=5 os_prio=0 cpu=0,11ms elapsed=113,16s tid=0x00007fd58a8ea000 nid=0x27a09 waiting on condition  [0x00007fd492cfe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Worker-13" #49 daemon prio=5 os_prio=0 cpu=0,19ms elapsed=113,16s tid=0x00007fd58a8eb800 nid=0x27a0a waiting on condition  [0x00007fd492bfe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Worker-14" #50 daemon prio=5 os_prio=0 cpu=0,22ms elapsed=113,16s tid=0x00007fd58a8ed800 nid=0x27a0b waiting on condition  [0x00007fd492afe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Worker-15" #51 daemon prio=5 os_prio=0 cpu=0,12ms elapsed=113,15s tid=0x00007fd58a8ef800 nid=0x27a0c waiting on condition  [0x00007fd4929fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)

"ZScheduler-Supervisor" #52 daemon prio=5 os_prio=0 cpu=47,86ms elapsed=113,15s tid=0x00007fd58a8f1800 nid=0x27a0d waiting on condition  [0x00007fd4928fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.parkUntil(java.base@11.0.18/LockSupport.java:391)
	at zio.internal.ZScheduler$$anon$3.run(ZScheduler.scala:316)

"zio-default-blocking-2" #54 daemon prio=5 os_prio=0 cpu=33,07ms elapsed=113,08s tid=0x00007fd440002000 nid=0x27a0f in Object.wait()  [0x00007fd4924fc000]
   java.lang.Thread.State: RUNNABLE
	at bootstrap.liftweb.RudderConfig$$$Lambda$419/0x000000080048c840.apply(Unknown Source)
	at com.normation.ldap.sdk.LDAPConnectionProvider.$anonfun$map$2(LDAPConnectionProvider.scala:82)
	at com.normation.ldap.sdk.LDAPConnectionProvider$$Lambda$462/0x00000008004c7440.apply(Unknown Source)
	at zio.ZIOCompanionVersionSpecific.$anonfun$attempt$1(ZIOCompanionVersionSpecific.scala:100)
	at zio.ZIOCompanionVersionSpecific$$Lambda$454/0x00000008004c2440.apply(Unknown Source)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:904)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:1024)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:1024)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:1024)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890)
	at zio.internal.FiberRuntime.evaluateEffect(FiberRuntime.scala:381)
	at zio.internal.FiberRuntime.evaluateMessageWhileSuspended(FiberRuntime.scala:504)
	at zio.internal.FiberRuntime.drainQueueOnCurrentThread(FiberRuntime.scala:220)
	at zio.internal.FiberRuntime.run(FiberRuntime.scala:139)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1128)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"Write Timeout Handler Timer" #55 daemon prio=5 os_prio=0 cpu=47,95ms elapsed=113,06s tid=0x00007fd444048000 nid=0x27a10 in Object.wait()  [0x00007fd4923fe000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.18/Native Method)
	- waiting on <no object reference available>
	at java.util.TimerThread.mainLoop(java.base@11.0.18/Timer.java:553)
	- waiting to re-lock in wait() <0x00000007fe1713a8> (a java.util.TaskQueue)
	at java.util.TimerThread.run(java.base@11.0.18/Timer.java:506)

"Connection reader for connection 0 in pool 'rudder-authenticated-ro' to localhost:1389" #57 daemon prio=5 os_prio=0 cpu=26,49ms elapsed=113,06s tid=0x00007fd444056800 nid=0x27a12 runnable  [0x00007fd4922fe000]
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(java.base@11.0.18/Native Method)
	at java.net.SocketInputStream.socketRead(java.base@11.0.18/SocketInputStream.java:115)
	at java.net.SocketInputStream.read(java.base@11.0.18/SocketInputStream.java:168)
	at java.net.SocketInputStream.read(java.base@11.0.18/SocketInputStream.java:140)
	at java.io.BufferedInputStream.fill(java.base@11.0.18/BufferedInputStream.java:252)
	at java.io.BufferedInputStream.read(java.base@11.0.18/BufferedInputStream.java:271)
	- locked <0x00000007fe18ecd8> (a java.io.BufferedInputStream)
	at com.unboundid.asn1.ASN1StreamReader.read(ASN1StreamReader.java:1199)
	at com.unboundid.asn1.ASN1StreamReader.readType(ASN1StreamReader.java:360)
	at com.unboundid.asn1.ASN1StreamReader.beginSequence(ASN1StreamReader.java:1118)
	at com.unboundid.ldap.protocol.LDAPMessage.readLDAPResponseFrom(LDAPMessage.java:1198)
	at com.unboundid.ldap.sdk.LDAPConnectionReader.run(LDAPConnectionReader.java:242)

"Health Check Thread for LDAPConnectionPool(serverSet=SingleServerSet(server=localhost:1389, includesAuthentication=false, includesPostConnectProcessing=false), maxConnections=2)" #58 daemon prio=5 os_prio=0 cpu=1,24ms elapsed=113,03s tid=0x00007fd44406d000 nid=0x27a13 waiting on condition  [0x00007fd4921fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007fdefa630> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at java.util.concurrent.LinkedBlockingQueue.poll(java.base@11.0.18/LinkedBlockingQueue.java:458)
	at com.unboundid.ldap.sdk.LDAPConnectionPoolHealthCheckThread.run(LDAPConnectionPoolHealthCheckThread.java:137)

"logback-3" #59 daemon prio=5 os_prio=0 cpu=0,94ms elapsed=108,57s tid=0x00007fd500002800 nid=0x27a16 waiting on condition  [0x00007fd554cfe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007facb0820> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"logback-4" #60 daemon prio=5 os_prio=0 cpu=0,79ms elapsed=103,57s tid=0x00007fd48c001800 nid=0x27a17 waiting on condition  [0x00007fd554dfe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007facb0820> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"logback-5" #61 daemon prio=5 os_prio=0 cpu=0,38ms elapsed=98,57s tid=0x00007fd500003800 nid=0x27a1a waiting on condition  [0x00007fd5549fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007facb0820> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"logback-6" #62 daemon prio=5 os_prio=0 cpu=0,39ms elapsed=93,57s tid=0x00007fd52087c000 nid=0x27a31 waiting on condition  [0x00007fd5548fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007facb0820> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"logback-7" #63 daemon prio=5 os_prio=0 cpu=0,41ms elapsed=88,57s tid=0x00007fd48c002800 nid=0x27a33 waiting on condition  [0x00007fd554bfe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007facb0820> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"logback-8" #64 daemon prio=5 os_prio=0 cpu=2,23ms elapsed=83,57s tid=0x00007fd51cb0d000 nid=0x27a47 waiting on condition  [0x00007fd50a4fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007facb0820> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1182)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)

"VM Thread" os_prio=0 cpu=17,99ms elapsed=115,85s tid=0x00007fd58922f800 nid=0x279d0 runnable  

"GC Thread#0" os_prio=0 cpu=24,53ms elapsed=115,88s tid=0x00007fd588045800 nid=0x279ca runnable  

"GC Thread#1" os_prio=0 cpu=15,39ms elapsed=114,24s tid=0x00007fd53c001000 nid=0x279ec runnable  

"GC Thread#2" os_prio=0 cpu=25,44ms elapsed=114,24s tid=0x00007fd53c002800 nid=0x279ed runnable  

"GC Thread#3" os_prio=0 cpu=15,97ms elapsed=114,24s tid=0x00007fd53c004000 nid=0x279ee runnable  

"GC Thread#4" os_prio=0 cpu=15,85ms elapsed=114,24s tid=0x00007fd53c006000 nid=0x279ef runnable  

"GC Thread#5" os_prio=0 cpu=16,23ms elapsed=114,24s tid=0x00007fd53c007800 nid=0x279f0 runnable  

"GC Thread#6" os_prio=0 cpu=17,76ms elapsed=114,24s tid=0x00007fd53c009000 nid=0x279f1 runnable  

"GC Thread#7" os_prio=0 cpu=16,04ms elapsed=114,24s tid=0x00007fd53c00a800 nid=0x279f2 runnable  

"GC Thread#8" os_prio=0 cpu=17,67ms elapsed=114,24s tid=0x00007fd53c00c000 nid=0x279f3 runnable  

"GC Thread#9" os_prio=0 cpu=18,49ms elapsed=114,24s tid=0x00007fd53c00d800 nid=0x279f4 runnable  

"GC Thread#10" os_prio=0 cpu=16,00ms elapsed=114,24s tid=0x00007fd53c00f000 nid=0x279f5 runnable  

"GC Thread#11" os_prio=0 cpu=19,17ms elapsed=114,24s tid=0x00007fd53c010800 nid=0x279f6 runnable  

"GC Thread#12" os_prio=0 cpu=22,14ms elapsed=114,24s tid=0x00007fd53c012000 nid=0x279f7 runnable  

"G1 Main Marker" os_prio=0 cpu=0,92ms elapsed=115,88s tid=0x00007fd5880a4000 nid=0x279cb runnable  

"G1 Conc#0" os_prio=0 cpu=25,28ms elapsed=115,88s tid=0x00007fd5880a6000 nid=0x279cc runnable  

"G1 Conc#1" os_prio=0 cpu=24,99ms elapsed=114,22s tid=0x00007fd558001000 nid=0x279f8 runnable  

"G1 Conc#2" os_prio=0 cpu=24,89ms elapsed=114,22s tid=0x00007fd558002800 nid=0x279f9 runnable  

"G1 Refine#0" os_prio=0 cpu=0,19ms elapsed=115,86s tid=0x00007fd58915e000 nid=0x279cd runnable  

"G1 Young RemSet Sampling" os_prio=0 cpu=14,80ms elapsed=115,86s tid=0x00007fd589160000 nid=0x279ce runnable  
"StrDedup" os_prio=0 cpu=0,18ms elapsed=115,86s tid=0x00007fd589168000 nid=0x279cf runnable  

"VM Periodic Task Thread" os_prio=0 cpu=53,66ms elapsed=115,66s tid=0x00007fd589b34800 nid=0x279df waiting on condition  

JNI global refs: 24, weak refs: 0

@fanf
Copy link
Author

fanf commented Apr 14, 2023

Some more tests: it does not seems to be an init problem with nodeFactRepository class because :

  • I put out of the init the getFacts method (the one with the faulty foreachPar
  • I cherry picked only the need services and force their init early
    -I checked that I can call methods on them and unsafe run and it returns
  • I built by end inputs for getFacts
  • I call getFacts just after the needed classes init
  • it locks.
  • amazingly, if the input size is one, it runs, but as low as 2 (even with parallelism at 4), it locks;

image

@fanf
Copy link
Author

fanf commented Apr 14, 2023

Even simple foreachPar blocks:
image

Threadump for it:

Full thread dump

"main" #1 prio=5 os_prio=0 cpu=2279,61ms elapsed=117,41s tid=0x00007f29b802c000 nid=0x33726 in Object.wait()  [0x00007f29bcd15000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.18/Native Method)
	- waiting on <0x00000007fbeead78> (a zio.internal.OneShot)
	at java.lang.Object.wait(java.base@11.0.18/Object.java:328)
	at zio.internal.OneShot.get(OneShot.scala:83)
	- waiting to re-lock in wait() <0x00000007fbeead78> (a zio.internal.OneShot)
	at zio.Runtime$UnsafeAPIV1.run(Runtime.scala:139)
	at bootstrap.liftweb.RudderConfig$.$anonfun$new$4(RudderConfig.scala:390)
	at bootstrap.liftweb.RudderConfig$$$Lambda$208/0x0000000800358840.apply(Unknown Source)
	at zio.Unsafe$.unsafe(Unsafe.scala:37)
	at bootstrap.liftweb.RudderConfig$.<clinit>(RudderConfig.scala:388)
	at bootstrap.liftweb.LiftInitContextListener.contextInitialized(LiftInitContextListener.scala:134)
	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1002)
	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:553)
	at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:931)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:764)
	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:360)
	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1445)
	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1409)
	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:844)
	at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:275)
	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
	- locked <0x00000007f4000558> (a java.lang.Object)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
	- locked <0x00000007f30091c8> (a java.lang.Object)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
	- locked <0x00000007f3009240> (a java.lang.Object)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
	at org.eclipse.jetty.server.Server.start(Server.java:408)
	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
	at org.eclipse.jetty.server.Server.doStart(Server.java:372)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
	- locked <0x00000007f3009350> (a java.lang.Object)
	at org.eclipse.jetty.runner.Runner.run(Runner.java:520)
	at org.eclipse.jetty.runner.Runner.main(Runner.java:565)


"zio-default-blocking-2" #54 daemon prio=5 os_prio=0 cpu=12,44ms elapsed=114,57s tid=0x00007f287c002000 nid=0x3376d in Object.wait()  [0x00007f28c6afd000]
   java.lang.Thread.State: RUNNABLE
	at bootstrap.liftweb.RudderConfig$$$Lambda$242/0x000000080037d040.apply$mcI$sp(Unknown Source)
	at zio.ZIO$.$anonfun$withParallelism$1(ZIO.scala:4981)
	at zio.ZIO$$$Lambda$243/0x000000080037d440.apply(Unknown Source)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:904)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:1024)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967)
	at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890)
	at zio.internal.FiberRuntime.evaluateEffect(FiberRuntime.scala:381)
	at zio.internal.FiberRuntime.evaluateMessageWhileSuspended(FiberRuntime.scala:504)
	at zio.internal.FiberRuntime.drainQueueOnCurrentThread(FiberRuntime.scala:220)
	at zio.internal.FiberRuntime.run(FiberRuntime.scala:139)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1128)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"qtp574268151-23" #23 prio=5 os_prio=0 cpu=429,68ms elapsed=116,56s tid=0x00007f29b9ffa800 nid=0x3373e waiting on condition  [0x00007f2943bfe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f4005b30> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"qtp574268151-24" #24 prio=5 os_prio=0 cpu=0,79ms elapsed=116,56s tid=0x00007f29b9ffc000 nid=0x3373f waiting on condition  [0x00007f29437fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f4005b30> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"qtp574268151-25" #25 prio=5 os_prio=0 cpu=0,52ms elapsed=116,56s tid=0x00007f29b9ffe000 nid=0x33740 waiting on condition  [0x00007f2942ffe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f4005b30> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"qtp574268151-26" #26 prio=5 os_prio=0 cpu=0,62ms elapsed=116,56s tid=0x00007f29b9fff800 nid=0x33741 waiting on condition  [0x00007f2942bfe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f4005b30> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"qtp574268151-27" #27 prio=5 os_prio=0 cpu=0,27ms elapsed=116,56s tid=0x00007f29ba001800 nid=0x33742 waiting on condition  [0x00007f29425fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f4005b30> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"qtp574268151-28" #28 prio=5 os_prio=0 cpu=0,18ms elapsed=116,56s tid=0x00007f29ba003000 nid=0x33743 waiting on condition  [0x00007f29424fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f4005b30> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"qtp574268151-29" #29 prio=5 os_prio=0 cpu=0,41ms elapsed=116,56s tid=0x00007f29ba005000 nid=0x33744 waiting on condition  [0x00007f29423fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f4005b30> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"qtp574268151-30" #30 prio=5 os_prio=0 cpu=0,11ms elapsed=116,56s tid=0x00007f29ba007000 nid=0x33745 waiting on condition  [0x00007f29422fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f4005b30> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:875)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:925)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"Common-Cleaner" #19 daemon prio=8 os_prio=0 cpu=1,04ms elapsed=117,30s tid=0x00007f29b95c6800 nid=0x33737 in Object.wait()  [0x00007f29806fe000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.18/Native Method)
	- waiting on <0x00000007f400a058> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@11.0.18/ReferenceQueue.java:155)
	- waiting to re-lock in wait() <0x00000007f400a058> (a java.lang.ref.ReferenceQueue$Lock)
	at jdk.internal.ref.CleanerImpl.run(java.base@11.0.18/CleanerImpl.java:148)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)
	at jdk.internal.misc.InnocuousThread.run(java.base@11.0.18/InnocuousThread.java:161)


"ZScheduler-Worker-0" #36 daemon prio=5 os_prio=0 cpu=0,65ms elapsed=114,67s tid=0x00007f29ba66d800 nid=0x3375b waiting on condition  [0x00007f28c7efe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Worker-1" #37 daemon prio=5 os_prio=0 cpu=0,43ms elapsed=114,67s tid=0x00007f29ba66f000 nid=0x3375c waiting on condition  [0x00007f28c7dfe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Worker-2" #38 daemon prio=5 os_prio=0 cpu=0,39ms elapsed=114,67s tid=0x00007f29ba671000 nid=0x3375d waiting on condition  [0x00007f28c7cfe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Worker-3" #39 daemon prio=5 os_prio=0 cpu=0,35ms elapsed=114,67s tid=0x00007f29ba672800 nid=0x3375e waiting on condition  [0x00007f28c7bfe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Worker-4" #40 daemon prio=5 os_prio=0 cpu=0,31ms elapsed=114,67s tid=0x00007f29ba674800 nid=0x3375f waiting on condition  [0x00007f28c7afe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Worker-5" #41 daemon prio=5 os_prio=0 cpu=0,23ms elapsed=114,67s tid=0x00007f29ba676800 nid=0x33760 waiting on condition  [0x00007f28c79fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Worker-6" #42 daemon prio=5 os_prio=0 cpu=0,31ms elapsed=114,67s tid=0x00007f29ba678000 nid=0x33761 waiting on condition  [0x00007f28c78fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Worker-7" #43 daemon prio=5 os_prio=0 cpu=0,31ms elapsed=114,67s tid=0x00007f29ba67a000 nid=0x33762 waiting on condition  [0x00007f28c77fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Worker-8" #44 daemon prio=5 os_prio=0 cpu=0,23ms elapsed=114,67s tid=0x00007f29ba67b800 nid=0x33763 waiting on condition  [0x00007f28c76fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Worker-9" #45 daemon prio=5 os_prio=0 cpu=0,18ms elapsed=114,67s tid=0x00007f29ba67d800 nid=0x33764 waiting on condition  [0x00007f28c75fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Worker-10" #46 daemon prio=5 os_prio=0 cpu=0,12ms elapsed=114,67s tid=0x00007f29ba67f800 nid=0x33765 waiting on condition  [0x00007f28c74fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Worker-11" #47 daemon prio=5 os_prio=0 cpu=0,18ms elapsed=114,67s tid=0x00007f29ba681000 nid=0x33766 waiting on condition  [0x00007f28c73fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Worker-12" #48 daemon prio=5 os_prio=0 cpu=0,11ms elapsed=114,67s tid=0x00007f29ba683000 nid=0x33767 waiting on condition  [0x00007f28c72fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Worker-13" #49 daemon prio=5 os_prio=0 cpu=0,24ms elapsed=114,67s tid=0x00007f29ba685000 nid=0x33768 waiting on condition  [0x00007f28c71fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Worker-14" #50 daemon prio=5 os_prio=0 cpu=0,17ms elapsed=114,67s tid=0x00007f29ba687000 nid=0x33769 waiting on condition  [0x00007f28c70fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Worker-15" #51 daemon prio=5 os_prio=0 cpu=0,13ms elapsed=114,67s tid=0x00007f29ba689000 nid=0x3376a waiting on condition  [0x00007f28c6ffe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:323)
	at zio.internal.ZScheduler$$anon$4.run(ZScheduler.scala:451)


"ZScheduler-Supervisor" #52 daemon prio=5 os_prio=0 cpu=58,13ms elapsed=114,67s tid=0x00007f29ba68b000 nid=0x3376b waiting on condition  [0x00007f28c6efe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	at java.util.concurrent.locks.LockSupport.parkUntil(java.base@11.0.18/LockSupport.java:391)
	at zio.internal.ZScheduler$$anon$3.run(ZScheduler.scala:316)


"Monitor Ctrl-Break" #20 daemon prio=5 os_prio=0 cpu=8,26ms elapsed=117,13s tid=0x00007f29b9980800 nid=0x3373a runnable  [0x00007f29803fe000]
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(java.base@11.0.18/Native Method)
	at java.net.SocketInputStream.socketRead(java.base@11.0.18/SocketInputStream.java:115)
	at java.net.SocketInputStream.read(java.base@11.0.18/SocketInputStream.java:168)
	at java.net.SocketInputStream.read(java.base@11.0.18/SocketInputStream.java:140)
	at sun.nio.cs.StreamDecoder.readBytes(java.base@11.0.18/StreamDecoder.java:284)
	at sun.nio.cs.StreamDecoder.implRead(java.base@11.0.18/StreamDecoder.java:326)
	at sun.nio.cs.StreamDecoder.read(java.base@11.0.18/StreamDecoder.java:178)
	- locked <0x00000007f4006488> (a java.io.InputStreamReader)
	at java.io.InputStreamReader.read(java.base@11.0.18/InputStreamReader.java:181)
	at java.io.BufferedReader.fill(java.base@11.0.18/BufferedReader.java:161)
	at java.io.BufferedReader.readLine(java.base@11.0.18/BufferedReader.java:326)
	- locked <0x00000007f4006488> (a java.io.InputStreamReader)
	at java.io.BufferedReader.readLine(java.base@11.0.18/BufferedReader.java:392)
	at com.intellij.rt.execution.application.AppMainV2$1.run(AppMainV2.java:53)


"logback-1" #21 daemon prio=5 os_prio=0 cpu=8,25ms elapsed=116,81s tid=0x00007f29b9d88800 nid=0x3373d waiting on condition  [0x00007f2943ffe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f40022a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1182)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"Session-HouseKeeper-e98770d-1" #32 prio=5 os_prio=0 cpu=0,19ms elapsed=115,51s tid=0x00007f29ba284000 nid=0x33757 waiting on condition  [0x00007f29801fc000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007ffe7a980> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.18/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.18/AbstractQueuedSynchronizer.java:2123)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1182)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"BC Entropy Daemon" #34 daemon prio=5 os_prio=0 cpu=1,08ms elapsed=115,09s tid=0x00007f29ba3a8000 nid=0x33759 waiting on condition  [0x00007f29420fe000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(java.base@11.0.18/Native Method)
	at org.bouncycastle.jcajce.provider.drbg.DRBG$EntropyDaemon.run(Unknown Source)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"logback-2" #35 daemon prio=5 os_prio=0 cpu=1,73ms elapsed=114,71s tid=0x00007f29ba650000 nid=0x3375a waiting on condition  [0x00007f28c7ffe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f40022a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"logback-3" #55 daemon prio=5 os_prio=0 cpu=0,90ms elapsed=109,71s tid=0x00007f2928002800 nid=0x3376e waiting on condition  [0x00007f29805fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f40022a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"logback-4" #56 daemon prio=5 os_prio=0 cpu=0,90ms elapsed=104,71s tid=0x00007f28c0001000 nid=0x33776 waiting on condition  [0x00007f29807fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f40022a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"logback-5" #57 daemon prio=5 os_prio=0 cpu=0,41ms elapsed=99,71s tid=0x00007f2928004000 nid=0x33779 waiting on condition  [0x00007f29804fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f40022a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"logback-6" #58 daemon prio=5 os_prio=0 cpu=0,49ms elapsed=94,71s tid=0x00007f2948847000 nid=0x33781 waiting on condition  [0x00007f29808fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f40022a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"logback-7" #59 daemon prio=5 os_prio=0 cpu=0,42ms elapsed=89,71s tid=0x00007f28c0002800 nid=0x33787 waiting on condition  [0x00007f29421fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f40022a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"logback-8" #60 daemon prio=5 os_prio=0 cpu=0,38ms elapsed=84,71s tid=0x00007f29516ca800 nid=0x33789 waiting on condition  [0x00007f28c69fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@11.0.18/Native Method)
	- parking to wait for  <0x00000007f40022a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.18/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.18/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:1177)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.18/ScheduledThreadPoolExecutor.java:899)
	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.18/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.18/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.18/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(java.base@11.0.18/Thread.java:829)


"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=0,70ms elapsed=117,36s tid=0x00007f29b9237800 nid=0x3372e waiting on condition  [0x00007f29813f6000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ref.Reference.waitForReferencePendingList(java.base@11.0.18/Native Method)
	at java.lang.ref.Reference.processPendingReferences(java.base@11.0.18/Reference.java:241)
	at java.lang.ref.Reference$ReferenceHandler.run(java.base@11.0.18/Reference.java:213)


"Finalizer" #3 daemon prio=8 os_prio=0 cpu=0,60ms elapsed=117,36s tid=0x00007f29b923c000 nid=0x3372f in Object.wait()  [0x00007f29812f6000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.18/Native Method)
	- waiting on <0x00000007f400b408> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@11.0.18/ReferenceQueue.java:155)
	- waiting to re-lock in wait() <0x00000007f400b408> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove(java.base@11.0.18/ReferenceQueue.java:176)
	at java.lang.ref.Finalizer$FinalizerThread.run(java.base@11.0.18/Finalizer.java:170)


"Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0,41ms elapsed=117,35s tid=0x00007f29b9251800 nid=0x33730 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE


"Service Thread" #5 daemon prio=9 os_prio=0 cpu=0,10ms elapsed=117,35s tid=0x00007f29b9253800 nid=0x33731 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE


"C2 CompilerThread0" #6 daemon prio=9 os_prio=0 cpu=2036,35ms elapsed=117,35s tid=0x00007f29b9255800 nid=0x33732 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task


"C1 CompilerThread0" #14 daemon prio=9 os_prio=0 cpu=2985,92ms elapsed=117,35s tid=0x00007f29b9257800 nid=0x33733 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task


"Sweeper thread" #18 daemon prio=9 os_prio=0 cpu=45,36ms elapsed=117,35s tid=0x00007f29b925a000 nid=0x33734 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE


"Attach Listener" #31 daemon prio=9 os_prio=0 cpu=4,42ms elapsed=116,37s tid=0x00007f2968001000 nid=0x33748 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE


"VM Thread" os_prio=0 cpu=15,97ms elapsed=117,37s tid=0x00007f29b922f800 nid=0x3372d runnable  


"GC Thread#0" os_prio=0 cpu=11,02ms elapsed=117,41s tid=0x00007f29b8045800 nid=0x33727 runnable  


"GC Thread#1" os_prio=0 cpu=8,66ms elapsed=115,55s tid=0x00007f296c031000 nid=0x33749 runnable  


"GC Thread#2" os_prio=0 cpu=14,93ms elapsed=115,55s tid=0x00007f296c032800 nid=0x3374a runnable  


"GC Thread#3" os_prio=0 cpu=9,34ms elapsed=115,55s tid=0x00007f296c034000 nid=0x3374b runnable  


"GC Thread#4" os_prio=0 cpu=8,27ms elapsed=115,55s tid=0x00007f296c036000 nid=0x3374c runnable  


"GC Thread#5" os_prio=0 cpu=8,30ms elapsed=115,55s tid=0x00007f296c037800 nid=0x3374d runnable  


"GC Thread#6" os_prio=0 cpu=16,33ms elapsed=115,55s tid=0x00007f296c039000 nid=0x3374e runnable  


"GC Thread#7" os_prio=0 cpu=8,52ms elapsed=115,55s tid=0x00007f296c03a800 nid=0x3374f runnable  


"GC Thread#8" os_prio=0 cpu=8,53ms elapsed=115,55s tid=0x00007f296c03c000 nid=0x33750 runnable  


"GC Thread#9" os_prio=0 cpu=12,48ms elapsed=115,55s tid=0x00007f296c03d800 nid=0x33751 runnable  


"GC Thread#10" os_prio=0 cpu=8,30ms elapsed=115,55s tid=0x00007f296c03f000 nid=0x33752 runnable  


"GC Thread#11" os_prio=0 cpu=8,27ms elapsed=115,55s tid=0x00007f296c040800 nid=0x33753 runnable  


"GC Thread#12" os_prio=0 cpu=8,15ms elapsed=115,55s tid=0x00007f296c042000 nid=0x33754 runnable  


"G1 Main Marker" os_prio=0 cpu=1,72ms elapsed=117,41s tid=0x00007f29b80a4000 nid=0x33728 runnable  


"G1 Conc#0" os_prio=0 cpu=12,98ms elapsed=117,41s tid=0x00007f29b80a6000 nid=0x33729 runnable  


"G1 Conc#1" os_prio=0 cpu=11,83ms elapsed=115,53s tid=0x00007f2984001000 nid=0x33755 runnable  


"G1 Conc#2" os_prio=0 cpu=12,51ms elapsed=115,53s tid=0x00007f2984002800 nid=0x33756 runnable  


"G1 Refine#0" os_prio=0 cpu=0,70ms elapsed=117,38s tid=0x00007f29b915e000 nid=0x3372a runnable  


"G1 Young RemSet Sampling" os_prio=0 cpu=17,14ms elapsed=117,38s tid=0x00007f29b9160000 nid=0x3372b runnable  


"StrDedup" os_prio=0 cpu=0,51ms elapsed=117,38s tid=0x00007f29b9168000 nid=0x3372c runnable  


"VM Periodic Task Thread" os_prio=0 cpu=65,46ms elapsed=117,13s tid=0x00007f29b9c04800 nid=0x3373b waiting on condition  
JNI global refs: 24, weak refs: 0

@fanf
Copy link
Author

fanf commented Apr 14, 2023

Stringer things: the above example with Console.printLine ALWAYS blocks, even with foreach. If I switch back to ZIO.attempt(println(...)), it works as before (blocks on foreachPar with input >1, works with foreach and input of size 1)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment