Wednesday, March 21, 2012

Threading stories: about robust thread pools

Another blog of my threading series. This time it's about thread pools, robust thread pool settings in particular. In Java thread pools are implemented by the ThreadPoolExecutor class introduced in Java 5. The Javadoc of that class is very well organized. So I spare me the effort to give a general introduction here. Basically, what ThreadPoolExecutor does is, it creates and manages threads that process Runnable tasks that were submitted to a work queue by an arbitrary client. It's a mechanism to perform work asynchronously, which is an important capability in times of multi-core machines and cloud computing.


Thread pool types

Some of the parameters shown in the construtor above are very sensible ones in terms of resource consumption and resulting system stability. Based on the different parameter settings of the constructor it's possible to distinguish some fundamental categories of thread pools. Here are some default thread pool settings offered by the Executors class.

In the "cached thread pool" the number of threads is unbounded. This is caused by a maximumPoolSize of Integer.MAX_VALUE in conjunction with SynchronousQueue. If you submit tasks in a burst to that thread pool, it will likely create a thread for each single task. In this scenario created threads terminate when they were idle for 60 seconds. The second example shows a "fixed thread pool", where maximumPoolSize is set to a specific fixed value. The pools thread count will never exceed this value. If tasks arive in a burst and all threads are busy then they will queue up in the work queue (here a LinkedBlockingQueue). Threads in this fixed thread pool never die. The drawback of unbounded pools is obvious: both settings can cause JVM memory trouble (you get OutOfMemoryErrors - if you're lucky).

Let's look at some bounded thread pool settings:


Thread pool boundaries

As shown above there are two basic approaches to define thread pools: bounded and unbounded thread pools. Unbounded thread pools, like the default ones of Executors class work fine, as long you don't submit too many tasks in a burst. If that happens unbounded thread pools can harm your systems stability. Either too many threads are created by a cached thread pool, or too many tasks are queued in a fixed thread pool. The letter is more difficult to achieve, but still possible. For production use it may be better to set the boundaries to some meaningfull values like in the last two thread pool settings. Because it can be tricky to define those "meaningfull boundaries" I have developed a little program that does that work for me.




Target queue memory usage (bytes): 100000
Formula: 100000 / 40
* Recommended queue capacity (bytes): 2500
Number of CPU: 2
Target utilization: 1.0
Elapsed time (nanos): 3000000000
Compute time (nanos): 906250000
Wait time (nanos): 2093750000
* Optimal thread count: 6.0

The 'recommended queue capacity' and the 'optimal thread count' are the important values. An ideal setting for my AsynchronousTask would be as follows:


OK, that's all in terms of robust thread pools for now. I hope you enjoyed it a little. And don't blame me if the formula isn't 100% accurate in terms of maximum pool size. As I said, it's not an exact science, it's about getting an idea of the ideal pool size.


  1. Good one. Surprisingly Thread pool is out there from long time but many programmer is still fell short of it. they not even use it so frequently,hope this article will help you to understand more.Another useful thread related feature is JDK 7 fork join framework which can help you to take advantage of multi-core processors

  2. Cool! BTW, Little's Law is also related. So is the M/M/1 queuing model.

  3. I didn't quite understand the fixed testtime of 3000 and its relation to wait time.

  4. Testtime - CPU Time during Testtime = Wait time

    1. Hi, Can you kindly tell me how to set this EPSYLON value? why when I run my own task, it always throws out this "Test not accurate" exception, due to this value is not apropriate? And is there any relationships between the task execution time and the EPSYLON value?

    2. just a so simple task,
      public class Task implements Runnable {

      public Task() {}

      public void run() {
      String str = "";
      try {

      if (Thread.interrupted()) {

      str = str + 't';


      } catch (InterruptedException e) {

      return ;


    3. Try make the sleep 1 millisecond ...
      You need to have the average duration of your task below the EPSYLON -> average task time < EPSYLON. The purpose is to ensure that a minimum amount of executions is ensured. The test also fails if the system is under very high load (the the test is inaccurate).

  5. Niklas, I did get, what is "warm-up" and why task should be started three times in a row?

  6. Hi, I just want to make sure that I don't measure JIT compilation CPU usage. Therefore I warm up the same task before I enter the test interval. Cheers, Niklas

    ! But yeah, thanx for spending time to discuss this topic
    here on your website.
    My web site:

  8. Hi, I want to use Thread Pool in this below program. How can I use?
    import java.util.*;

    public class MaxDemo2
    public static void main(String[] args)
    MaxSquare ms = new MaxSquare(4, 5);
    System.out.println("total squares found : " + MaxSquare.cnts);
    }catch(Exception e){}


    class MaxSquare implements Runnable {

    Thread t[];
    static int cnts;

    class Square {
    int size;
    int r1;
    int r2;
    int c1;
    int c2;

    public Square(int x1, int x2, int y1, int y2) {
    r1 = x1; r2 = x2;
    c1 = y1; c2 = y2;
    size = (r2-r1+1)*(c2-c1+1);
    cnts = 0;

    protected int[][] a;
    int rows;
    int cols;
    int i,j;
    int S[][];
    public MaxSquare(int r, int c) {
    rows = r;
    cols = c;
    a = new int[r][c];

    }catch(Exception e){}

    protected void initMatrix() {

    a[0][0] = 1;
    a[0][1] = 1;
    a[0][2] = 0;
    a[0][3] = 0;
    a[0][4] = 1;
    a[1][0] = 1;
    a[1][1] = 1;
    a[1][2] = 1;
    a[1][3] = 1;
    a[1][4] = 0;
    a[2][0] = 1;
    a[2][1] = 1;
    a[2][2] = 0;
    a[2][3] = 1;
    a[2][4] = 1;
    a[3][0] = 1;
    a[3][1] = 1;
    a[3][2] = 0;
    a[3][3] = 0;
    a[3][4] = 1;


    public void dumpMatrix() {

    protected void findSquares()
    t = new Thread[rows];

    S = new int[rows][cols];
    S[i][j] = a[i][j];

    t[i] = new Thread(this,"demo");


    for(i = 1; i < rows; i++)

    }catch(Exception e){}


    public void run()

    for(j = 1; j < cols; j++)
    if(a[i][j] == 1)

    S[i][j] = Math.min(Math.min(S[i][j-1], S[i-1][j]), S[i-1][j-1]) + 1;

    }catch(Exception e){}