这里粘出的源码基于JDK1.8
ThreadPoolExecutor 线程池类。是Executor框架中最核心的类。
主要包括:
线程池状态介绍以及状态流转
线程池的创建
线程池构造器的相关参数以及作用
Exectors 提供的线程池介绍以及存在的缺陷
线程池的任务提交流程以及源码分析
线程池的关闭
线程池状态
private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));
private static final int COUNT_BITS = Integer.SIZE - 3;
private static final int CAPACITY = (1 << COUNT_BITS) - 1;
// runState is stored in the high-order bits
private static final int RUNNING = -1 << COUNT_BITS;
private static final int SHUTDOWN = 0 << COUNT_BITS;
private static final int STOP = 1 << COUNT_BITS;
private static final int TIDYING = 2 << COUNT_BITS;
private static final int TERMINATED = 3 << COUNT_BITS;
// Packing and unpacking ctl
private static int runStateOf(int c) { return c & ~CAPACITY; }
private static int workerCountOf(int c) { return c & CAPACITY; }
private static int ctlOf(int rs, int wc) { return rs | wc; }
线程状态控制变量 - ctl
ctl 定义为一个AtomicInteger。
其高3三位,用于记录线程池当前的状态,低29位,表示线程池中任务的数量。
线程池状态 - runState。
RUNNING - 运行中。Accept new tasks and process queued tasks.
SHUTDOWN - 关闭。Don't accept new tasks, but process queued tasks.
STOP - 停止。Don't accept new tasks, don't process queued tasks and interrupt in-progress tasks.
TIDYING - 整理。All tasks have terminated, workerCountis zero, the thread transitioning to state TIDYING will return the terminated() hook method. 如果用户想在线程池变为TIDYING时进行相应的处理,则可以重载 terminated方法。
TERMINATED - 终止。terminated() has completed。默认是空的,由用户重载来自定义操作。
线程池状态的转换:
1)RUNNING -> SHUTDOWN
调用 shutdown() 方法,该方法有时候隐式的在finalize() 方法中调用。
2)RUNNING or SHUTDOWN -> STOP
调用 shutdownNow() 方法
3)SHUTDOWN -> TIDYING
当等待队列和线程池都为空时。(When both queue and pool are empty)
4)STOP -> TIDYING
当线程池为空时。(When pool is empty)
5)TIDYING -> TERMINATED
当 terminated() 钩子函数执行完成的时候。
当调用 awaitTermination() 时,线程会等待state达到 TERMINATED 状态时,会返回。
(Threads wating in awaitTermination() will return when the state reaches TERMINATED)
状态转换如下:
创建线程池
public ThreadPoolExecutor(int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue,
ThreadFactory threadFactory,
RejectedExecutionHandler handler) {
if (corePoolSize < 0 ||
maximumPoolSize <= 0 ||
maximumPoolSize < corePoolSize ||
keepAliveTime < 0)
throw new IllegalArgumentException();
if (workQueue == null || threadFactory == null || handler == null)
throw new NullPointerException();
this.acc = System.getSecurityManager() == null ?
null :
AccessController.getContext();
this.corePoolSize = corePoolSize;
this.maximumPoolSize = maximumPoolSize;
this.workQueue = workQueue;
this.keepAliveTime = unit.toNanos(keepAliveTime);
this.threadFactory = threadFactory;
this.handler = handler;
}
一共有七个参数:
corePoolSize:核心线程数
maxmunPoolSize:最大线程数
keepAliveTime:空闲线程的保持活跃时间
unit:keepAliveTime的单位
workQueue:保存待执行任务的阻塞队列
threadFactory:创建线程的工厂
hander:线程池的拒绝策略(源码注释:the handler to use when execution is blocked because the thread bounds and queue capacities are reached)
这里特别说一下拒绝策略,和 线程工厂。很多人会忽略。
拒绝策略: RejectedExecutionHandler
是一个接口,主要作用是,当线程池拒绝任务之后,执行该接口定义的方法来执行拒绝以后的行为。
可以自定义行为,当然,Java也提供了几种策略供选择:
1)new ThreadPoolExecutor.CallerRunsPolicy()
重试执行当前任务,但会交由调用者线程来执行该任务。
2)new ThreadPoolExecutor.AbortPolicy()
抛出RejectedExecutionException
异常
3)new ThreadPoolExecutor.DiscardPolicy()
直接抛弃任务,不做任何响应动作
4)new ThreadPoolExecutor.DiscardOldestPolicy()
抛弃线程池中最后要执行的那个任务,并执行新传入的任务。
线程工厂:ThreadFactory
用于设置创建线程的工厂。
默认使用Executors.defaultThreadFactory(),返回的是 default的线程工厂对象。当然也可以由我们自己实现 ThreadFactory接口,来实现自定义的线程工厂。
源码如下:
public static ThreadFactory defaultThreadFactory() {
return new DefaultThreadFactory();
}
/**
* The default thread factory
*/
static class DefaultThreadFactory implements ThreadFactory {
private static final AtomicInteger poolNumber = new AtomicInteger(1);
private final ThreadGroup group;
private final AtomicInteger threadNumber = new AtomicInteger(1);
private final String namePrefix;
DefaultThreadFactory() {
SecurityManager s = System.getSecurityManager();
group = (s != null) ? s.getThreadGroup() :
Thread.currentThread().getThreadGroup();
namePrefix = "pool-" +
poolNumber.getAndIncrement() +
"-thread-";
}
public Thread newThread(Runnable r) {
Thread t = new Thread(group, r,
namePrefix + threadNumber.getAndIncrement(),
0);
if (t.isDaemon())
t.setDaemon(false);
if (t.getPriority() != Thread.NORM_PRIORITY)
t.setPriority(Thread.NORM_PRIORITY);
return t;
}
}
线程池
Executor框架为我们已经提供好了一部分线程池,同时线程池还都提供了一些重载的方法,支持自定义线程工厂。
newFixedThreadPool
newCachedThreadPool
newSingleThreadPool
它们都可以通过工具类Executors类来创建。
在 java.util.cuncurrent.Executors
下,Java提供了四种基本的线程池(还有其他的暂时不看了):
1)newFixedThreadPool:
固定线程数量的线程池(核心线程数=最大线程数),提交任务到线程最大数量后,就把任务放到无界队列中(LinkedBlockingQueue)。
缺点:工作队列大小无限制,但是可能造成工作队列提交任务过多,资源耗尽。另外,工作队列过长的话,也会导致提交的任务过长时间的等待。
源码如下:
public static ExecutorService newFixedThreadPool(int nThreads) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue());
}
// 自定义ThreadFactory
public static ExecutorService newFixedThreadPool(int nThreads, ThreadFactory threadFactory) {
return new ThreadPoolExecutor(nThreads, nThreads,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue(),
threadFactory);
}
2)newCachedThreadPool:
可缓存线程池(核心线程数=0,最大线程数=Integer.MAX_VALUE)。可以回收空闲线程(默认设定60秒过期时间),当有任务提交时,启动新线程执行。工作队列使用同步队列(SynchronousQueue),意味着只要有任务提交就会被交给线程池的worker来处理。
缺点:最大线程数无限制,当提交任务过多时,提交的速度远大于线程池消费的速度,可能造成创建大量线程,导致资源耗尽。
所以使用该线程池时,一定要控制并发的任务数,否则创建大量的线程可能导致严重的性能问题
源码如下:
public static ExecutorService newCachedThreadPool() {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue());
}
// 自定义ThreadFactory
public static ExecutorService newCachedThreadPool(ThreadFactory threadFactory) {
return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
60L, TimeUnit.SECONDS,
new SynchronousQueue(),
threadFactory);
}
3)newSingleThreadExecutor:
单线程化的线程池,只会用唯一的工作线程来执行任务,保证所有被提交的任务按照FIFO的顺序执行。
工作队列是一个阻塞队列(LinkedBlockingQueue)
缺点:依然是阻塞队列没有限制大小,如果任务提交过多,可能导致队列中任务阻塞过多导致资源耗尽。
源码如下:
public static ExecutorService newSingleThreadExecutor() {
return new FinalizableDelegatedExecutorService
(new ThreadPoolExecutor(1, 1,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue()));
}
// 自定义ThreadFactory
public static ExecutorService newSingleThreadExecutor(ThreadFactory threadFactory) {
return new FinalizableDelegatedExecutorService
(new ThreadPoolExecutor(1, 1,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue(),
threadFactory));
}
4)newScheduledThreadPool:
大小无限制的线程池,支持定时和周期性的执行线程。由ScheduledThreadPoolExecutor创建(其又继承自ThreadPoolExecutor)。
源码如下:
public static ScheduledExecutorService newScheduledThreadPool(int corePoolSize) {
return new ScheduledThreadPoolExecutor(corePoolSize);
}
// 自定义ThreadFactory
public static ScheduledExecutorService newScheduledThreadPool(
int corePoolSize, ThreadFactory threadFactory) {
return new ScheduledThreadPoolExecutor(corePoolSize, threadFactory);
}
// 单线程化的ScheduledThreadPool
public static ScheduledExecutorService newSingleThreadScheduledExecutor() {
return new DelegatedScheduledExecutorService
(new ScheduledThreadPoolExecutor(1));
}
// 单线程化的ScheduledThreadPool 自定义ThreadFactory
public static ScheduledExecutorService newSingleThreadScheduledExecutor(ThreadFactory threadFactory) {
return new DelegatedScheduledExecutorService
(new ScheduledThreadPoolExecutor(1, threadFactory));
}
5)newWorkStealingPool:
返回拥有多个任务队列的线程池,结合JDK1.7中引入的 Fork/Join 框架来理解。(TODO)
源码如下:
public static ExecutorService newWorkStealingPool(int parallelism) {
return new ForkJoinPool
(parallelism,
ForkJoinPool.defaultForkJoinWorkerThreadFactory,
null, true);
}
public static ExecutorService newWorkStealingPool() {
return new ForkJoinPool
(Runtime.getRuntime().availableProcessors(),
ForkJoinPool.defaultForkJoinWorkerThreadFactory,
null, true);
}
任务提交
线程池根据业务不同,提供了两种方式来提交任务:
Executor.execute() 。返回void
ExecutorService.submit()。返回该任务的Future
具体流程参考自源码:
java.util.concurrent.ThreadPoolExecutor.execute(Runnable command)
// Executor类
public interface Executor {
void execute(Runnable command);
}
// 该方法是 ThreadPoolExecutor中实现的 execute
public void execute(Runnable command) {
if (command == null)
throw new NullPointerException();
/*
* Proceed in 3 steps:
*
* 1. If fewer than corePoolSize threads are running, try to
* start a new thread with the given command as its first
* task. The call to addWorker atomically checks runState and
* workerCount, and so prevents false alarms that would add
* threads when it shouldn't, by returning false.
*
* 2. If a task can be successfully queued, then we still need
* to double-check whether we should have added a thread
* (because existing ones died since last checking) or that
* the pool shut down since entry into this method. So we
* recheck state and if necessary roll back the enqueuing if
* stopped, or start a new thread if there are none.
*
* 3. If we cannot queue task, then we try to add a new
* thread. If it fails, we know we are shut down or saturated
* and so reject the task.
*/
int c = ctl.get();
if (workerCountOf(c) < corePoolSize) {
if (addWorker(command, true))
return;
c = ctl.get();
}
if (isRunning(c) && workQueue.offer(command)) {
int recheck = ctl.get();
if (! isRunning(recheck) && remove(command))
reject(command);
else if (workerCountOf(recheck) == 0)
addWorker(null, false);
}
else if (!addWorker(command, false))
reject(command);
}
其实从源码中就可以看出来
主要分为三个步骤:
1)如果线程池中的线程数小于核心线程数,则会调用addWorker创建新线程执行任务,该任务被创建的新线程立即执行,成功的话返回true。否则执行步骤2
2)如果线程池状态为RUNNING,则尝试加入阻塞队列,如果加入队列成功,则尝试进行Double check。如果加入失败,则执行步骤3
3)如果线程池不是RUNNING状态,或者加如阻塞队列失败,则尝试创建新线程来执行任务,直到活跃线程数达到maxPoolSize,如果失败,则调用reject()方法执行相应的拒绝策略。
线程池任务提交主要流程如图:
addWorker
addWorker(Runnable firstTask, boolean core) 方法用于创建线程和执行任务
源码就不放了,具体可以看JDK中的源码。
主要分为三个步骤:
1)判断当前线程池是否可以添加任务,如果可以则往下走,否则return false.
2)通过内嵌for(;;)循环,CAS方式对 workerCount + 1
3)获取主锁。①如果线程处于RUNNING状态,或者处于SHUTDOWN状态且firstTask==null。则将任务添加到wokers Queue中。然后释放主锁,②启动线程,return true。如果中途失败导致 workerStated=false,则调用addWorkerFailed()方法处理。
// 部分源码
// ①
if (rs < SHUTDOWN || (rs == SHUTDOWN && firstTask == null)) {
if (t.isAlive()) // precheck that t is startable
throw new IllegalThreadStateException();
workers.add(w);
int s = workers.size();
if (s > largestPoolSize)
largestPoolSize = s;
workerAdded = true;
}
// ②
if (workerAdded) {
t.start();
workerStarted = true;
}
Worker
在新建线程任务时,是将Runnable包装成一个Worker对象,Worker为ThreadPoolExecutor的内部类。
源码如下:
private final class Worker extends AbstractQueuedSynchronizer
implements Runnable
{
/**
* This class will never be serialized, but we provide a
* serialVersionUID to suppress a javac warning.
*/
private static final long serialVersionUID = 6138294804551838833L;
/** Thread this worker is running in. Null if factory fails. */
final Thread thread;
/** Initial task to run. Possibly null. */
Runnable firstTask;
/** Per-thread task counter */
volatile long completedTasks;
/**
* Creates with given first task and thread from ThreadFactory.
* @param firstTask the first task (null if none)
*/
Worker(Runnable firstTask) {
setState(-1); // inhibit interrupts until runWorker
this.firstTask = firstTask;
this.thread = getThreadFactory().newThread(this);
}
/** Delegates main run loop to outer runWorker */
public void run() {
runWorker(this);
}
// Lock methods
//
// The value 0 represents the unlocked state.
// The value 1 represents the locked state.
protected boolean isHeldExclusively() {
return getState() != 0;
}
protected boolean tryAcquire(int unused) {
if (compareAndSetState(0, 1)) {
setExclusiveOwnerThread(Thread.currentThread());
return true;
}
return false;
}
protected boolean tryRelease(int unused) {
setExclusiveOwnerThread(null);
setState(0);
return true;
}
public void lock() { acquire(1); }
public boolean tryLock() { return tryAcquire(1); }
public void unlock() { release(1); }
public boolean isLocked() { return isHeldExclusively(); }
void interruptIfStarted() {
Thread t;
if (getState() >= 0 && (t = thread) != null && !t.isInterrupted()) {
try {
t.interrupt();
} catch (SecurityException ignore) {
}
}
}
}
从源码可以看到,Worker内部类,继承了AQS,并且实现了Runnable类。
所以Worker不仅可以执行任务,而且可以利用AQS框架进行获取和释放锁的操作。这里主要是方便线程的中断处理。
有两个地方需要注意:
1)构造函数。
主要做了三件事。1、设置同步状态为-1(注释可以看出到,0表示unlocked,1表示locked)。2、给firstTask参数赋值。3、使用线程工厂创建当前Worker对象(this)的线程实例。
2)run()方法。
调用了ThreadPoolExecutor内部的一个final方法,runWorker(Worker w);
源码如下:
final void runWorker(Worker w) {
Thread wt = Thread.currentThread();
Runnable task = w.firstTask;
w.firstTask = null;
w.unlock(); // allow interrupts
boolean completedAbruptly = true;
try {
while (task != null || (task = getTask()) != null) {
w.lock();
// If pool is stopping, ensure thread is interrupted;
// if not, ensure thread is not interrupted. This
// requires a recheck in second case to deal with
// shutdownNow race while clearing interrupt
if ((runStateAtLeast(ctl.get(), STOP) ||
(Thread.interrupted() &&
runStateAtLeast(ctl.get(), STOP))) &&
!wt.isInterrupted())
wt.interrupt();
try {
beforeExecute(wt, task);
Throwable thrown = null;
try {
task.run();
} catch (RuntimeException x) {
thrown = x; throw x;
} catch (Error x) {
thrown = x; throw x;
} catch (Throwable x) {
thrown = x; throw new Error(x);
} finally {
afterExecute(task, thrown);
}
} finally {
task = null;
w.completedTasks++;
w.unlock();
}
}
completedAbruptly = false;
} finally {
processWorkerExit(w, completedAbruptly);
}
}
主要步骤:
1)从woker获取需要执行的Runnable task任务。(因为在worker初始化的时候可能已经放进去了一个task)
然后调用w.unlock() 将锁的状态置为0。(因为初始化woker的时候,状态为-1,置为0是为了允许中断,因为interruptWokers()方法只有在state>=0的时候才会执行)
2)判断当前task,或者从getTask() 方法获取任务。
如果没有获取到任务则直接走到最后。
如果获取到任务了,则会对woker加锁 w.lock()。然后调用task.run()执行,执行完成之后,释放woker的锁。w.unlock()
3)在任务执行前后,调用 beforeExecute(wt, task) 和 afterExecute(wt, task)。这两个方法在 ThreadPoolExecutor中是空实现。protected修饰,可以由用户自定义实现。
4)可以看到 获取task那段代码是一个 while循环。如果一直能获取到task,则一直循环获取task执行任务。如果阻塞队列为空,则根据是否超时来判断是否需要阻塞(这段逻辑在getTask()方法中)
5)如果task==null 或者抛出异常导致 worker线程终止,则调用 processWorkerExit()方法处理woker退出流程。
线程终止
线程池ThreadPoolExecutor 提供了两个方法来关闭线程池:
1)shutdown():按任务提交的顺序,发起一个有序的关闭动作。并且不会接受新的任务。
2)shutdownNow():尝试停止所有的活动执行任务,暂停等待队列中任务的处理,并且返回等待执行的任务列表。
源码如下:
shutdown:
/**
* Initiates an orderly shutdown in which previously submitted
* tasks are executed, but no new tasks will be accepted.
* Invocation has no additional effect if already shut down.
*
* This method does not wait for previously submitted tasks to
* complete execution. Use {@link #awaitTermination awaitTermination}
* to do that.
*
* @throws SecurityException {@inheritDoc}
*/
public void shutdown() {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
checkShutdownAccess();
advanceRunState(SHUTDOWN); // 推进线程状态为SHUTDOWN
interruptIdleWorkers(); // 中断空闲的线程
onShutdown(); // hook for ScheduledThreadPoolExecutor (子类实现的关闭钩子)
} finally {
mainLock.unlock();
}
tryTerminate();
}
shutdownNow:
/**
* Attempts to stop all actively executing tasks, halts the
* processing of waiting tasks, and returns a list of the tasks
* that were awaiting execution. These tasks are drained (removed)
* from the task queue upon return from this method.
*
* This method does not wait for actively executing tasks to
* terminate. Use {@link #awaitTermination awaitTermination} to
* do that.
*
*
There are no guarantees beyond best-effort attempts to stop
* processing actively executing tasks. This implementation
* cancels tasks via {@link Thread#interrupt}, so any task that
* fails to respond to interrupts may never terminate.
*
* @throws SecurityException {@inheritDoc}
*/
public List shutdownNow() {
List tasks;
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
checkShutdownAccess();
advanceRunState(STOP); // 推进线程状态为STOP
interruptWorkers(); // 中断所有的工作线程
tasks = drainQueue(); // 返回等待队列中未执行的任务
} finally {
mainLock.unlock();
}
tryTerminate();
return tasks;
}
(如果有什么错误或者建议,欢迎留言指出)
(本文内容是对各个知识点的转载整理,用于个人技术沉淀,以及大家学习交流用)
参考资料:
【死磕Java并发】—–J.U.C之线程池:ThreadPoolExecutor
源码:ThreadPoolExecutor(JDK1.8)