Jetty multiple acceptors blocked waiting to lock and object locked by other acceptor
<p>I'm facing an issue where the application hangs and possibly is due to blocked Jetty threads, but i'm not 100% sure in how to interpret the thread dump, if is normal what is observed and how does the jetty acceptors lifecycle is.</p>

<p>This is a particular part of the thread dump i'm observing:</p>

<pre><code>"qtp1291915867-82 Acceptor0 SelectChannelConnector@0.0.0.0:3581 STARTING" prio=5 tid=0x52 state=BLOCKED
    at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:281)
      - waiting to lock java.lang.Object@71918ba2 [owned by "qtp1291915867-84 Acceptor2 SelectChannelConnector@0.0.0.0:3581 STARTING" tid=0x54]
    at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:97)
    at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:833)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:598)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:533)
    at java.lang.Thread.run(Thread.java:804)

"qtp1291915867-83 Acceptor1 SelectChannelConnector@0.0.0.0:3581 STARTING" prio=5 tid=0x53 state=BLOCKED
    at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:281)
      - waiting to lock java.lang.Object@71918ba2 [owned by "qtp1291915867-84 Acceptor2 SelectChannelConnector@0.0.0.0:3581 STARTING" tid=0x54]
    at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:97)
    at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:833)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:598)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:533)
    at java.lang.Thread.run(Thread.java:804)

"qtp1291915867-84 Acceptor2 SelectChannelConnector@0.0.0.0:3581 STARTING" prio=5 tid=0x54 state=RUNNABLE (running in native)
    at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
    at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:305)
      - locked java.lang.Object@71918ba2
    at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:97)
    at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:833)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:598)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:533)
    at java.lang.Thread.run(Thread.java:804)

"qtp1291915867-85 Acceptor3 SelectChannelConnector@0.0.0.0:3581 STARTING" prio=5 tid=0x55 state=BLOCKED
    at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:281)
      - waiting to lock java.lang.Object@71918ba2 [owned by "qtp1291915867-84 Acceptor2 SelectChannelConnector@0.0.0.0:3581 STARTING" tid=0x54]
    at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:97)
    at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:833)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:598)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:533)
    at java.lang.Thread.run(Thread.java:804)
</code></pre>

<p>AFAIK, the amount of acceptors should be between 1 and # of CPU's, so i'm assuming this is running in a 4 Core CPU, the question is, from jetty perspective, is it normal that other acceptors be blocked waiting to other acceptors resource? if so, what this resource could be?</p>

<p>I'm not very fond on Jetty so i don't really know if this is normal and they are just waiting for they turn to attend incoming connections, if so, then the issue is in different place but i would like to discard the jetty part.</p>

<p>Btw, the jetty version is 7.5.4</p>

<p>Any ideas?</p>

<p>Thanks</p>