JAVA-异步编程-线程和线程池实现异步编程,线程池源码导读
1.文章目录
- 如何使用线程，线程池实现异步编程，以及其各自的优缺点；
- 线程池的原理，源码导读；
2.使用线程实现异步编程
任务类
public class Task {
// taskA
public static void doSomethingA() {
try {
// 模拟耗时
Thread.sleep(200);
System.out.println("taskA done");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
// taskA
public static void doSomethingB() {
try {
// 模拟耗时
Thread.sleep(200);
System.out.println("taskB done");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}同步编程：
测试:耗时约4s左右
public class SyncExample {
public static void main(String[] argv) throws InterruptedException {
long start = System.currentTimeMillis();
Task.doSomethingA();
Task.doSomethingB();
System.out.println(System.currentTimeMillis() - start);
}
}
显式使用线程：
package AsynchronousProgramming;
import java.util.concurrent.CountDownLatch;
/**
* @Author: SoftWareKang
* @Name:JAVALEARN
* @Date: 2020/5/30 15:16
*/
public class SyncExample {
private static CountDownLatch countDownLatch = new CountDownLatch(2);
public static void main(String[] argv) throws InterruptedException {
long start = System.currentTimeMillis();
// do A
new Thread(() -> {
try {
Task.doSomethingA();
countDownLatch.countDown();
} catch (Exception e) {
e.printStackTrace();
}
}
).start();
// do B
new Thread(() -> {
try {
Task.doSomethingB();
countDownLatch.countDown();
} catch (Exception e) {
e.printStackTrace();
}
}
).start();
countDownLatch.await();
System.out.println(System.currentTimeMillis() - start);
}
}
测试耗时：约2.5s左右
如上代码，我们使用lamba创建Java.lang.Runnable接口的实现类；上述代码，耗时2.5左右，可以看出异步编程可以大大缩减任务，当任务数多的时候效果更明显；
JAVA中Deamon与No Deamon的区别，默认情况下我们创建的线程是No Deamon的，线程的类型与JVM退出条件有关，在JAVA中当JVM进程中不存在No Deamon线程就会退出。我们可以显式的setDaemon(true)方法设置线程为Deamon线程；
显示用线程编程的缺点：
- 每次执行异步任务，会直接创建一个Thread来执行异步任务,生产环境是不可以的，因为线程创建，销毁是有开销的，随意的滥用会消耗完系统线程，从而出错；推荐使用线程池来执行异步任务，线程池也有效的限制线程数量；
- 上述的线程执行异步任务没有返回值，如果需要我们可以用JDK的Future;
3.线程池实现异步编程
public class ThreadPoolTest {
// 定义线程池
private final static int AVALIABLE_PROCESSORS = Runtime.getRuntime().availableProcessors();
private final static ThreadPoolExecutor POOL_EXECUTOR = new ThreadPoolExecutor(
AVALIABLE_PROCESSORS, AVALIABLE_PROCESSORS * 2, 1, TimeUnit.MINUTES,
new LinkedBlockingQueue<>(5), new ThreadPoolExecutor.CallerRunsPolicy()
);
public static void main(String[] argv) throws InterruptedException {
long start = System.currentTimeMillis();
POOL_EXECUTOR.execute(() -> {
Task.doSomethingA();
});
// Task.doSomethingB();
POOL_EXECUTOR.execute(() -> {
Task.doSomethingB();
});
System.out.println(System.currentTimeMillis() - start);
Thread.currentThread().join();
}
}- 上述代码创建了线程池：核心线程数为CPU核数，最大线程数为2*CPU核数；线程池阻塞队列大小为5；拒绝策略为CallerRunsPolicy，当线程池任务饱和，不会丢弃新任务，而是使用调用线程执行；
- 上述代码，我们减轻了main线程的负担，把任务交给线程池处理，我们再去完成其他任务；
测试结果：我们发现JVM任务执行完，没有退出，因为线程池的线程不是Deamon线程，JVM检测到存在NO Deamon线程所以不退出；
因此我们可以调用POOL_EXECUTOR.shutdown(); //POOL_EXECUTOR.shutdownNow();关闭线程池；
- 以上的没有实现异步返回的任务，我们测试下；
4.线程池实现异步返回任务编程
创建任务：返回“taskC"
// TaskC
public static String doSomethingC() {
try {
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(" taskC done");
return "taskC";
}测试：
public class AsncThreadPoolTest {
// 定义线程池
private final static int AVALIABLE_PROCESSORS = Runtime.getRuntime().availableProcessors();
private final static ThreadPoolExecutor POOL_EXECUTOR = new ThreadPoolExecutor(
AVALIABLE_PROCESSORS, AVALIABLE_PROCESSORS * 2, 1, TimeUnit.MINUTES,
new LinkedBlockingQueue<>(),new ThreadPoolExecutor.CallerRunsPolicy()
);
public static void main(String[] argv) throws ExecutionException, InterruptedException {
Future future = POOL_EXECUTOR.submit(() -> Task.doSomethingC());
// 阻塞，等待异步任务结果
System.out.println(future.get());
POOL_EXECUTOR.shutdown();
}
}
- 后续我们针对Future原理，源码角度来看如果实现；
5.线程池源码，原理解析
基本属性
// 线程池状态(高3位) 线程个数（低29)
private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));
// 线程掩码位
private static final int COUNT_BITS = Integer.SIZE - 3;
// 容量
private static final int CAPACITY = (1 << COUNT_BITS) - 1;
// runState is stored in the high-order bits
// 线程池状态
// 111
private static final int RUNNING = -1 << COUNT_BITS;
// 000
private static final int SHUTDOWN = 0 << COUNT_BITS;
// 001
private static final int STOP = 1 << COUNT_BITS;
// 010
private static final int TIDYING = 2 << COUNT_BITS;
// 011
private static final int TERMINATED = 3 << COUNT_BITS;
// 获取运行状态
private static int runStateOf(int c) { return c & ~CAPACITY; }
//线程个数
private static int workerCountOf(int c) { return c & CAPACITY; }
// 计算ctl新值
private static int ctlOf(int rs, int wc) { return rs | wc; }
// 任务阻塞队列
private final BlockingQueue workQueue;
// lock
private final ReentrantLock mainLock = new ReentrantLock();
// woker集合
private final HashSet workers = new HashSet();
// codition条件集合
private final Condition termination = mainLock.newCondition();
// 最大线程数
private int largestPoolSize;
// 任务完成个数
private long completedTaskCount;
// 线程工厂
private volatile ThreadFactory threadFactory;
// 拒绝策略
private volatile RejectedExecutionHandler handler;
// 线程空闲时间
private volatile long keepAliveTime;
// 是否允许timeount
private volatile boolean allowCoreThreadTimeOut;
// 核心线程数
private volatile int corePoolSize;
//
private volatile int maximumPoolSize;
/**
* The default rejected execution handler
*/
private static final RejectedExecutionHandler defaultHandler =
new AbortPolicy(); - corePoolSize:线程池核心线程个数.
- workQueue:用于保存等待执行任务的阻塞队列；
- maximunPoolSize:线程池最大线程数量;
- threadFactory:线程工厂类；
- defaultHandler:饱和策略，当任务队列满后，线程个数达到了maximunPoolSize就执行饱和策略,默认AbortPolicy(抛出异常），Caller Runs Policy（使用调用者所在的线程来运行任务),DiscardOldestPolicy(丢弃一个任务，执行当前的任务），DiscardPolicy(直接丢弃);
初始化方法：
// 使用默认threadFactory,拒绝策略
public ThreadPoolExecutor(int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue) {
this(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue,
Executors.defaultThreadFactory(), defaultHandler);
}
// 自定义线程工厂使用
public ThreadPoolExecutor(int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue,
ThreadFactory threadFactory) {
this(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue,
threadFactory, defaultHandler);
}
// 自定义线程工厂，拒绝策略
public ThreadPoolExecutor(int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue,
RejectedExecutionHandler handler) {
this(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue,
Executors.defaultThreadFactory(), handler);
}
// 核心方法
public ThreadPoolExecutor(int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue,
ThreadFactory threadFactory,
RejectedExecutionHandler handler) {
// 参数校验赋值
if (corePoolSize < 0 ||
maximumPoolSize <= 0 ||
maximumPoolSize < corePoolSize ||
keepAliveTime < 0)
throw new IllegalArgumentException();
if (workQueue == null || threadFactory == null || handler == null)
throw new NullPointerException();
this.acc = System.getSecurityManager() == null ?
null :
AccessController.getContext();
this.corePoolSize = corePoolSize;
this.maximumPoolSize = maximumPoolSize;
this.workQueue = workQueue;
this.keepAliveTime = unit.toNanos(keepAliveTime);
this.threadFactory = threadFactory;
this.handler = handler;
} 默认线程工厂:defaultThreadFactory
static class DefaultThreadFactory implements ThreadFactory {
// 原子类
private static final AtomicInteger poolNumber = new AtomicInteger(1);
// 线程组
private final ThreadGroup group;
// 技术
private final AtomicInteger threadNumber = new AtomicInteger(1);
// threadname前缀
private final String namePrefix;
DefaultThreadFactory() {
SecurityManager s = System.getSecurityManager();
group = (s != null) ? s.getThreadGroup() :
Thread.currentThread().getThreadGroup();
namePrefix = "pool-" +
poolNumber.getAndIncrement() +
"-thread-";
}
public Thread newThread(Runnable r) {
// 创建一个线程
Thread t = new Thread(group, r,
namePrefix + threadNumber.getAndIncrement(),
0);
// 如果线程位Daemon，置为非Daemon
if (t.isDaemon())
t.setDaemon(false);
// 重置线程优先级为正常5
if (t.getPriority() != Thread.NORM_PRIORITY)
t.setPriority(Thread.NORM_PRIORITY);
return t;
}
}execut:线程池执行任务的方法：
public void execute(Runnable command) {
if (command == null)
throw new NullPointerException();
// 获取ctl(运行状态&线程数)
int c = ctl.get();
// 如果线程数小于核心线程数，直接创建一个线程
if (workerCountOf(c) < corePoolSize) {
// true表示，当前创建线程数量应<=核心线程数
if (addWorker(command, true))
return;
// 如果没有添加成功，获取新的ctl
c = ctl.get();
}
// 如果线程池处于Running状态，添加任务到阻塞队列
if (isRunning(c) && workQueue.offer(command)) {
// 二次校验
int recheck = ctl.get();
// 如果不是running则从队列删除任务，并执行拒绝策略
if (! isRunning(recheck) && remove(command))
reject(command);
// 如果线程数为0
else if (workerCountOf(recheck) == 0)
// 添加一个线程
addWorker(null, false);
}
// 如果线程数达到了核心线程数，且添加阻塞队列失败，则创建一个线程
else if (!addWorker(command, false))
// 失败则，拒绝
reject(command);
}addWorker（Runnable, bool)方法：创建一个线程，运行runnable任务，bool决定最大线程为核心线程数，还是最大线程数来执行拒绝策略
private boolean addWorker(Runnable firstTask, boolean core) {
retry:
for (;;) {
// 获取ctl
int c = ctl.get();
// 获取运行状态
int rs = runStateOf(c);
// 检测运行状态以及队列状态
if (rs >= SHUTDOWN &&
! (rs == SHUTDOWN &&
firstTask == null &&
! workQueue.isEmpty()))
return false;
for (;;) {
// 获取线程数
int wc = workerCountOf(c);
// 如果线程数大于容量/（核心线程数/最大线程数)core决定
if (wc >= CAPACITY ||
wc >= (core ? corePoolSize : maximumPoolSize))
return false;
// CAS算法，增加线程数，成功跳出循环
if (compareAndIncrementWorkerCount(c))
break retry;
c = ctl.get(); // Re-read ctl
if (runStateOf(c) != rs)
continue retry;
// else CAS failed due to workerCount change; retry inner loop
}
}
// 任务start,worker添加成功标志
boolean workerStarted = false;
boolean workerAdded = false;
Worker w = null;
try {
// worker包装firsrTask
w = new Worker(firstTask);
final Thread t = w.thread;
if (t != null) {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
// Recheck while holding lock.
// Back out on ThreadFactory failure or if
// shut down before lock acquired.
int rs = runStateOf(ctl.get());
// 校验线程池状态
if (rs < SHUTDOWN ||
(rs == SHUTDOWN && firstTask == null)) {
if (t.isAlive()) // precheck that t is startable
throw new IllegalThreadStateException();
workers.add(w);
int s = workers.size();
if (s > largestPoolSize)
largestPoolSize = s;
workerAdded = true;
}
} finally {
mainLock.unlock();
}
// 如果添加成功，运行
if (workerAdded) {
t.start();
workerStarted = true;
}
}
} finally {
if (! workerStarted)
addWorkerFailed(w);
}
return workerStarted;
}几种拒绝策略：
// 使用调用者的线程运行
public static class CallerRunsPolicy implements RejectedExecutionHandler {
/**
* Creates a {@code CallerRunsPolicy}.
*/
public CallerRunsPolicy() { }
/**
* Executes task r in the caller's thread, unless the executor
* has been shut down, in which case the task is discarded.
*
* @param r the runnable task requested to be executed
* @param e the executor attempting to execute this task
*/
public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
if (!e.isShutdown()) {
r.run();
}
}
}
// 抛出异常
public static class AbortPolicy implements RejectedExecutionHandler {
/**
* Creates an {@code AbortPolicy}.
*/
public AbortPolicy() { }
/**
* Always throws RejectedExecutionException.
*
* @param r the runnable task requested to be executed
* @param e the executor attempting to execute this task
* @throws RejectedExecutionException always
*/
public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
throw new RejectedExecutionException("Task " + r.toString() +
" rejected from " +
e.toString());
}
}
// 默认不管
public static class DiscardPolicy implements RejectedExecutionHandler {
/**
* Creates a {@code DiscardPolicy}.
*/
public DiscardPolicy() { }
/**
* Does nothing, which has the effect of discarding task r.
*
* @param r the runnable task requested to be executed
* @param e the executor attempting to execute this task
*/
public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
}
}
// 抛出队列中的一个，然后运行他
public static class DiscardOldestPolicy implements RejectedExecutionHandler {
/**
* Creates a {@code DiscardOldestPolicy} for the given executor.
*/
public DiscardOldestPolicy() { }
/**
* Obtains and ignores the next task that the executor
* would otherwise execute, if one is immediately available,
* and then retries execution of task r, unless the executor
* is shut down, in which case task r is instead discarded.
*
* @param r the runnable task requested to be executed
* @param e the executor attempting to execute this task
*/
public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
if (!e.isShutdown()) {
e.getQueue().poll();
e.execute(r);
}
}
}前置，后置方法;子类可以进行扩展
protected void beforeExecute(Thread t, Runnable r) { }
protected void afterExecute(Runnable r, Throwable t) { }线程池提供的三种异步Future方式：都是包装为RunnableFuture执行
public Future submit(Runnable task) {
if (task == null) throw new NullPointerException();
RunnableFuture ftask = newTaskFor(task, null);
execute(ftask);
return ftask;
}
/**
* @throws RejectedExecutionException {@inheritDoc}
* @throws NullPointerException {@inheritDoc}
*/
public Future submit(Runnable task, T result) {
if (task == null) throw new NullPointerException();
RunnableFuture ftask = newTaskFor(task, result);
execute(ftask);
return ftask;
}
/**
* @throws RejectedExecutionException {@inheritDoc}
* @throws NullPointerException {@inheritDoc}
*/
public Future submit(Callable task) {
if (task == null) throw new NullPointerException();
RunnableFuture ftask = newTaskFor(task);
execute(ftask);
return ftask;
}
Worker:我们线程池调度这东西，看下他源码；
// run方法
public void run() {
runWorker(this);
}
// 核心方法
final void runWorker(Worker w) {
Thread wt = Thread.currentThread();
Runnable task = w.firstTask;
w.firstTask = null;
w.unlock(); // allow interrupts
boolean completedAbruptly = true;
try {
//如果task为null 任务队列获取的也是null
while (task != null || (task = getTask()) != null) {
w.lock();
// shutDownNow可以打断当前线程，可以自己测试下
if ((runStateAtLeast(ctl.get(), STOP) ||
(Thread.interrupted() &&
runStateAtLeast(ctl.get(), STOP))) &&
!wt.isInterrupted())
wt.interrupt();
try {
// 执行前
beforeExecute(wt, task);
Throwable thrown = null;
try {
// 核心，也是我们写的task
task.run();
} catch (RuntimeException x) {
thrown = x; throw x;
} catch (Error x) {
thrown = x; throw x;
} catch (Throwable x) {
thrown = x; throw new Error(x);
} finally {
// 执行后处理
afterExecute(task, thrown);
}
} finally {
task = null;
w.completedTasks++;
w.unlock();
}
}
completedAbruptly = false;
} finally {
// 执行清理
processWorkerExit(w, completedAbruptly);
}
}- 可以看出来，我们的worker就是一个thread,不断的执行任务；
任务运行完：执行清理
private void processWorkerExit(Worker w, boolean completedAbruptly) {
if (completedAbruptly) // If abrupt, then workerCount wasn't adjusted
decrementWorkerCount();
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
// 完成任务数
completedTaskCount += w.completedTasks;
// woker集合删除此woker
workers.remove(w);
} finally {
mainLock.unlock();
}
// 尝试设置线程池状态为terminated
tryTerminate();
int c = ctl.get();
if (runStateLessThan(c, STOP)) {
if (!completedAbruptly) {
int min = allowCoreThreadTimeOut ? 0 : corePoolSize;
if (min == 0 && ! workQueue.isEmpty())
min = 1;
if (workerCountOf(c) >= min)
return; // replacement not needed
}
addWorker(null, false);
}
}关闭线程池：
ShutDown:关闭线程池，线程池不会接受新任务，工作队列的任务执行完；
public void shutdown() {
final ReentrantLock mainLock = this.mainLock;
// 保证线程同步
mainLock.lock();
try {
checkShutdownAccess();
// 设置线程状态
advanceRunState(SHUTDOWN);
// 设置中断标志
interruptIdleWorkers();
onShutdown(); // hook for ScheduledThreadPoolExecutor
} finally {
mainLock.unlock();
}
tryTerminate();
} private void advanceRunState(int targetState) {
for (;;) {
int c = ctl.get();
if (runStateAtLeast(c, targetState) ||
ctl.compareAndSet(c, ctlOf(targetState, workerCountOf(c))))
break;
}
} private void interruptIdleWorkers(boolean onlyOne) {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
for (Worker w : workers) {
Thread t = w.thread;
// 如果线程没有被中断，并且没有运行
if (!t.isInterrupted() && w.tryLock()) {
try {
// 中断
t.interrupt();
} catch (SecurityException ignore) {
} finally {
w.unlock();
}
}
if (onlyOne)
break;
}
} finally {
mainLock.unlock();
}
}
shutDownNow:中断所有线程，包括正在运行的
public List shutdownNow() {
List tasks;
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
checkShutdownAccess();
advanceRunState(STOP);
//打断所有线程
interruptWorkers();
tasks = drainQueue();
} finally {
mainLock.unlock();
}
tryTerminate();
return tasks;
} private void interruptWorkers() {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
// 全部打断
for (Worker w : workers)
w.interruptIfStarted();
} finally {
mainLock.unlock();
}
}6.总结
- 本文描述了异步执行的好处，以及其具体实现；
- 显式线程的优缺点，线程池优点以及其实现原理，重要源码导读；
- 线程池线程增加策略：如果当前线程数<=核心线程数，新建一个线程；然后给阻塞队列添加，队列满了，当线程数<=最大线程数则新建线程；
- 上述的异步并不是比较好的实现，Future必须调用Get阻塞当前线程才可以拿到任务返回值，后续文章，会对Future源码，以及JDK新增的CompletableAbleFuture实现异步编程；