java forkjoin MySQL_Java并发编程(五) ForkJoinPool的使用
一.前言
之前在整理线程使用的时候,无意间看到了ForkJoinPool,在JDK1.7时(新)加入的,就学习了如何使用;
二. ForkJoinPool 使用
2.1 ForkJoinPool的使用姿势
ForkJoinPool采用工作窃取算法,将一个大任务根据阈值分割成很多个子任务,最后根据场景是否要合并子任务运算结果;
根据是否需要合并子任务运算结果,任务需要继承抽象类RecursiveAction,RecursiveTask,后者为需要合并子任务结果,泛型为结果类型;
我们只需要实现抽象方法 protected Void compute(),定义子任务拆分规则和任务算法就可以了;
我们查看类继承图可知其属于Future的一个实现,Future的使用在之前有过介绍;
有2种使用方案:
fork : 递归划分子任务,无需合并子任务结果;
fork & join : 递归划分子任务,最后合并子任务计算结果;
2.2 Only Fork
不需要合并子任务运算结果的场景;
下面的模拟场景是将集合sender中的元素发送到receiver中;
package com.river.thread;
import lombok.AllArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.util.StopWatch;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
@Slf4j
public class ForkJoinPoolTest {
private final static List sender = new ArrayList(21000000);
private final static List receiver = new ArrayList<>(21000000);
private final static List receiver2 = new ArrayList<>(21000000);
private final static AtomicInteger i = new AtomicInteger(0);
static {
log.info("prepare data");
while (i.get() < 21000000) {
sender.add(i.get());
i.incrementAndGet();
}
log.info("prepare over");
}
public static void main(String[] args) throws InterruptedException, ExecutionException {
StopWatch stopWatch = new StopWatch();
stopWatch.start();
SendTask sendTask = new SendTask(0, 210000, sender);
log.info("Task Start !");
ForkJoinPool forkJoinPool = ForkJoinPool.commonPool();
forkJoinPool.submit(sendTask);
forkJoinPool.awaitTermination(10, TimeUnit.SECONDS);
forkJoinPool.shutdown();
stopWatch.stop();
log.info("sender.size -> {}", sender.size());
log.info("receiver.size -> {}", receiver.size());
log.info("TotalTimeMillis1 -> "+stopWatch.getTotalTimeMillis());
}
@Slf4j
@AllArgsConstructor
public static class SendTask extends RecursiveTask {
//定义递归子任务的阈值
private final static int preSize = 100;
private int start;
private int end;
private List tempList;
private final static AtomicInteger taskId = new AtomicInteger(0);
/**
* The main computation performed by this task.
*/
@Override
protected Void compute() {
if (end - start < preSize) {
//log.info("add start {} to end {}", start, end);
for (int i = start; i < end; i++) {
add(this.tempList.get(i));
}
} else {
int middle = (start + end) / 2;
RecursiveTask sendTaskLeft = new SendTask(start, middle, this.tempList);
RecursiveTask sendTaskRight = new SendTask(middle, end, this.tempList);
SendTask.invokeAll(sendTaskLeft, sendTaskRight);
}
return null;
}
//防止并发,list.add()方法并发插入,因为在第一次没有在add方法做同步限制,导致并发,找个好久问题,如果发短信什么的业务操作不需要做同步处理
public void add(int i){
synchronized (SendTask.class) {
receiver.add(i);
}
}
}
}
我们查看日志:
2018-09-15 16:49:41.383 myAppName [main] INFO com.river.thread.ForkJoinPoolTest - prepare data
2018-09-15 16:49:46.986 myAppName [main] INFO com.river.thread.ForkJoinPoolTest - prepare over
2018-09-15 16:49:46.987 myAppName [main] INFO com.river.thread.ForkJoinPoolTest - Task Start !
2018-09-15 16:49:47.106 myAppName [main] INFO com.river.thread.ForkJoinPoolTest - sender.size -> 21000000
2018-09-15 16:49:47.108 myAppName [main] INFO com.river.thread.ForkJoinPoolTest - receiver.size -> 2100000
2018-09-15 16:49:47.108 myAppName [main] INFO com.river.thread.ForkJoinPoolTest - TotalTimeMillis1 -> 119
我们在查看线程情况,有如下4条线程在执行任务,并且我的电脑就是4核的;
ForkJoinPool.commonPool-worker-3
ForkJoinPool.commonPool-worker-2
ForkJoinPool.commonPool-worker-1
main
在这可以看到,与我们之前使用多线程不同的是,这里将main也作为执行任务线程之一,
2.3 Fork & Join
package com.river.thread;
import lombok.AllArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.util.StopWatch;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
@Slf4j
public class ForkJoinPoolTest {
private final static List sender = new ArrayList(21000000);
private final static List receiver = new ArrayList<>(21000000);
private final static List receiver2 = new ArrayList<>(21000000);
private final static AtomicInteger i = new AtomicInteger(0);
static {
log.info("prepare data");
while (i.get() < 21000000) {
sender.add(i.get());
i.incrementAndGet();
}
log.info("prepare over");
}
public static void main(String[] args) throws InterruptedException, ExecutionException {
int count = 5000000;
SumTask sumTask = new SumTask(0, count);
ForkJoinPool forkJoinPool = ForkJoinPool.commonPool();
ForkJoinTask submit = forkJoinPool.submit(sumTask);
System.out.println(submit.get());
int s = 0;
for (int i = 0;i<=count;i++) {
s +=i;
}
System.out.println(s);
}
@Slf4j
@AllArgsConstructor
public static class SumTask extends RecursiveTask{
private final static int threshold = 5000;
private int start;
private int end;
/**
* The main computation performed by this task.
*
* @return the result of the computation
*/
@Override
protected Integer compute() {
int sum = 0;
if (end - start < threshold){
for (int i = start; i< end; i++){
sum +=i;
}
}else {
int middle = (start + end) / 2;
SumTask sumTask = new SumTask(start, middle);
SumTask sumTask1 = new SumTask(middle, end);
SumTask.invokeAll(sumTask, sumTask1);
sum = sumTask.join() + sumTask1.join();
}
return sum;
}
}
}
三.相关使用
在JDK8中lamdba有个stream操作parallelStream,底层也是使用ForkJoinPool实现的;
我们可以通过Executors.newWorkStealingPool(int parallelism)快速创建ForkJoinPool线程池,无参默认使用CPU数量的线程数执行任务;