JCP - Applying thread pool

# sizing thread pool

Thread pool size should rarely be hard-coded, instead it should be provided by configuration mechanism or computed dynamically.

If size 'too big', threads compete for scare CPU and memory resources, resulting in possible resources exhaustion.

If size 'too small', throughput suffers.

The optimal pool size is:

size = N(cpu) * U(cpu) * (1 + wait time / compute time)

U(cpu): target CPU utilization[0,1]

int N_CPUS= Runtime.getRuntime().availableProcessors();

# ThreadPoolExecutor

ThreadPoolExecutor provides the base implementation for executors returned by below methods:

Executors.newSingleThreadExecutor();
Executors.newFixedThreadPool(3);
Executors.newCachedThreadPool();

we can instantiate ThreadPoolExecutor through its constructor:

ThreadPoolExecutor pool = new ThreadPoolExecutor(
                                /*corePoolSize*/ 	5 ,  
                                /*maximumPoolSize*/ 	10,  
                                /*keepAliveTime*/ 	60L,  
                                /*unit*/ 		TimeUnit.SECONDS,  
                                /*workQueue*/ 		new ArrayBlockingQueue<Runnable>(20),  
				/*threadFactory*/	new ThreadFactory() {	 		                                                                    @Override		 						                                    public Thread newThread(Runnable r) { 							                return new Thread(r);		 					                            } 
                                                          }, 
                                /*handler*/		new ThreadPoolExecutor.CallerRunsPolicy());

- corePoolSize: is the target size, we will maintain this size even there are no tasks to execute.

- maximumPoolSize: is the upper bound on how many threads can be active at once.

- keepAliveTime: a thread has been idle for longer than this time becomes a candidate for reaping.

- workQueue: BlockingQueue to hold tasks awaiting execution.

# managing queued tasks

If the arrival rate for new requests exceeds the rate at which they can be handled, requests will queue up.

# saturation polies

when workqueue is full and there is still new requests arrive, saturation policies come into play:

1.  AbortPolicy: default one; throws 'RejectedExecutionException'

2. DiscardPolicy: silently discards newly submitted task.

3. DiscardOldestPolicy: discard next task.

4. CallerRunsPolicy: push back tasks to the caller(main thread, TCP layer, client ..)