JAVA FORK-JOIN的使用例子
JAVA FORK-JOIN的使用例子
java的fork-join简介
先贴官方原文介绍：
The fork/join framework is an implementation of the ExecutorService interface that helps you take advantage of multiple processors. It is designed for work that can be broken into smaller pieces recursively. The goal is to use all the available processing power to enhance the performance of your application.
fork/join 框架是ExecutorService接口的一个实现，它可以帮你利用处理器的优势。它设计的原理是把可以分解的大的任务分解成一个个小任务。它的目标是利用所有可用的处理能力去增强你的应用的性能。
As with any ExecutorService implementation, the fork/join framework distributes tasks to worker threads in a thread pool. The fork/join framework is distinct because it uses a work-stealing algorithm. Worker threads that run out of things to do can steal tasks from other threads that are still busy.
就像ExecutorService的其他实现，fork/join框架分派任务给worker线程，让他们在线程池跑。fork/join框架有点特别，因为它使用了work-stealing算法。Worker线程跑完任务后，可以从其他还在忙着的线程去窃取任务。
The center of the fork/join framework is the ForkJoinPool class, an extension of the AbstractExecutorService class. ForkJoinPool implements the core work-stealing algorithm and can execute ForkJoinTask processes.
fork/join框架的核心是ForkJoinPool类，一个AbstractExecutorService类的扩展。ForkJoinPool实现了核心work-stealing算法，可以执行ForkJoinTask里编写的任务。
英语不太好，请见谅。。根据我的理解，fork/join可以让你把大的任务分解成一个个小任务，然后利用多线程能力，并发地跑这些任务，这样处理起来，就快很多了。如果每个小任务都有返回值，还可以把这些返回值汇总起来进行处理。
有的ForkJoinTask子类还可以递归分解任务：如果大任务分解出来的小任务还不够小，就还可以继续分，一直递归。
举个例子，学校需要收校服的费用，如果只派一个老师去一个个同学那儿收，那需要很长时间才能收完。如果这位老师这样做：他安排了一些负责人，有年级层级的负责人，年纪层级下有班级负责人。班级负责人就负责该班的学费收集，收完就提交给年级负责人，年级负责人收集完该年级的费用后，统一交给这位老师。这样任务经过分解后，可以同时收集不同群体的费用，这样的效率必定是高很多的。
但需要注意的是，fork/join只适用于可以分解任务的情况。所以不是所有的大型操作都可以使用fork/join。
代码演示
如果仅仅看上面的解释，估计很多人都还是不明白具体是要怎么操作。究竟如何定义大任务？如何分解任务？每个小任务完成后，如何把所有结果汇总？
这里直接贴出代码演示下，不过多解释，看了估计就懂了。（下面代码的场景是模拟一大堆数据需要插入到数据库，我把它不断分解，直到每个任务最多只需插入两条数据）：
任务类，该类记载了具体要做什么任务，如何分解成小任务，如何汇总。
package top.usjava.learn.javaarchitecturelearn.design;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.RecursiveTask;
/**
* 批量插入数据任务类
* 利用fork-join框架对数据插入任务进行分组，变成分组插入
*
* @author Owen
* create time:2018/9/23 11:49
*/
public class BatchInsertTask extends RecursiveTask<Integer> {
//要插入的数据
List<Integer> records;
public BatchInsertTask(List<Integer> records) {
this.records = records;
}
@Override
protected Integer compute() {
//当要插入的数据少于3，则直接插入
if (records.size() < 3) {
return computeDirectly();
} else {
//如果要插入的数据大于等于3，则进行分组插入
int size = records.size();
//第一个分组
BatchInsertTask aTask = new BatchInsertTask(records.subList(0, size / 2));
//第二个分组
BatchInsertTask bTask = new BatchInsertTask(records.subList(size / 2, records.size()));
//两个任务并发执行起来
invokeAll(aTask, bTask);
//两个分组的插入的行数加起来
return aTask.join() + bTask.join();
}
}
/**
* 真正插入数据的逻辑
*/
private int computeDirectly() {
try {
Thread.sleep((long) (records.size() * 1000));
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("插入了：" + Arrays.toString(records.toArray()));
return records.size();
}
}
Main方法，先初始化具有12个整数的list，赋值给BatchInsertTask这个大任务，然后通过ForkJoinPool 执行BatchInsertTask 任务。
package top.usjava.learn.javaarchitecturelearn.unitlearn;
import top.usjava.learn.javaarchitecturelearn.design.BatchInsertTask;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.ForkJoinTask;
/**
* 练习使用fork-join
*
* @author Owen
* create time:2018/9/23 11:49
*/
public class TestForkJoin {
public static void main(String[] args) throws ExecutionException, InterruptedException {
ForkJoinPool forkJoinPool = new ForkJoinPool(8);
List<Integer> list = new ArrayList<>();
list.add(1);
list.add(1);
list.add(1);
list.add(1);
list.add(1);
list.add(1);
list.add(1);
list.add(1);
list.add(1);
list.add(1);
list.add(1);
list.add(1);
BatchInsertTask batchInsertTask = new BatchInsertTask(list);
long t1 = System.currentTimeMillis();
ForkJoinTask<Integer> reslut = forkJoinPool.submit(batchInsertTask);
System.out.println(reslut.get());
long t2 = System.currentTimeMillis();
System.out.println(t2-t1);
}
}
运行结果：
从上面代码看出，核心的分解逻辑、结果汇总逻辑都在compute()方法里，上面代码达到的效果是：最底层执行数据插入操作，然后把插入的行数返回给上一级，上一级把下面的结果收集完后，再返回给上上级，这样递归执行。
具体invokeAll()、join()方法是啥意思，建议去看官方文档。
顺便贴出没有返回值汇总的用法（官方例子）
注意：没有返回值的是继承RecursiveAction类的。
/*
* Copyright (c) 2010, 2013, Oracle and/or its affiliates. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* - Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
*
* - Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* - Neither the name of Oracle or the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
* IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
* THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
import java.awt.image.BufferedImage;
import java.io.File;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.RecursiveAction;
import javax.imageio.ImageIO;
/**
* ForkBlur implements a simple horizontal image blur. It averages pixels in the
* source array and writes them to a destination array. The sThreshold value
* determines whether the blurring will be performed directly or split into two
* tasks.
*
* This is not the recommended way to blur images; it is only intended to
* illustrate the use of the Fork/Join framework.
*/
public class ForkBlur extends RecursiveAction {
private int[] mSource;
private int mStart;
private int mLength;
private int[] mDestination;
private int mBlurWidth = 15; // Processing window size, should be odd.
public ForkBlur(int[] src, int start, int length, int[] dst) {
mSource = src;
mStart = start;
mLength = length;
mDestination = dst;
}
// Average pixels from source, write results into destination.
protected void computeDirectly() {
int sidePixels = (mBlurWidth - 1) / 2;
for (int index = mStart; index < mStart + mLength; index++) {
// Calculate average.
float rt = 0, gt = 0, bt = 0;
for (int mi = -sidePixels; mi <= sidePixels; mi++) {
int mindex = Math.min(Math.max(mi + index, 0), mSource.length - 1);
int pixel = mSource[mindex];
rt += (float) ((pixel & 0x00ff0000) >> 16) / mBlurWidth;
gt += (float) ((pixel & 0x0000ff00) >> 8) / mBlurWidth;
bt += (float) ((pixel & 0x000000ff) >> 0) / mBlurWidth;
}
// Re-assemble destination pixel.
int dpixel = (0xff000000)
| (((int) rt) << 16)
| (((int) gt) << 8)
| (((int) bt) << 0);
mDestination[index] = dpixel;
}
}
protected static int sThreshold = 10000;
@Override
protected void compute() {
if (mLength < sThreshold) {
computeDirectly();
return;
}
int split = mLength / 2;
invokeAll(new ForkBlur(mSource, mStart, split, mDestination),
new ForkBlur(mSource, mStart + split, mLength - split,
mDestination));
}
// Plumbing follows.
public static void main(String[] args) throws Exception {
String srcName = "red-tulips.jpg";
File srcFile = new File(srcName);
BufferedImage image = ImageIO.read(srcFile);
System.out.println("Source image: " + srcName);
BufferedImage blurredImage = blur(image);
String dstName = "blurred-tulips.jpg";
File dstFile = new File(dstName);
ImageIO.write(blurredImage, "jpg", dstFile);
System.out.println("Output image: " + dstName);
}
public static BufferedImage blur(BufferedImage srcImage) {
int w = srcImage.getWidth();
int h = srcImage.getHeight();
int[] src = srcImage.getRGB(0, 0, w, h, null, 0, w);
int[] dst = new int[src.length];
System.out.println("Array size is " + src.length);
System.out.println("Threshold is " + sThreshold);
int processors = Runtime.getRuntime().availableProcessors();
System.out.println(Integer.toString(processors) + " processor"
+ (processors != 1 ? "s are " : " is ")
+ "available");
ForkBlur fb = new ForkBlur(src, 0, src.length, dst);
ForkJoinPool pool = new ForkJoinPool();
long startTime = System.currentTimeMillis();
pool.invoke(fb);
long endTime = System.currentTimeMillis();
System.out.println("Image blur took " + (endTime - startTime) +
" milliseconds.");
BufferedImage dstImage =
new BufferedImage(w, h, BufferedImage.TYPE_INT_ARGB);
dstImage.setRGB(0, 0, w, h, dst, 0, w);
return dstImage;
}
}