课程笔记：Parallel Programming in Java（第一周）
Parallel Programming in Java（第一周）
Parallel Programming in Java 是 Coursera 的上的一门课程，一共有四周课程内容，讲述Java中的并行程序设计。这里是第一周课程的内容笔记。主要内容为 Task Parallel，即 任务并行设计
Task Parallel
Task Parallel 包括 task 创建和 task 终止两部分主要操作。
- 使用 async 指明那些 task 是可以被并行执行的
- 使用 finish block 等待任务同步结束，回到顺序执行的部分
使用 PCDP 包可以很好地在代码中支持 async 和 finish block 标注
Fork-Join Framework
Java 标准中需要使用 Fork-Join Framework 。将可以并行的部分使用 fork 创建一个新的进程执行，在需要同步的地方使用 join 等待该进行的结束。Java 中的封装机制使得 fork 和 join 函数对应一个类实例执行，即：
instance.fork();
instance.function();
instance.join();
完成一个并行 task 的部署和同步。
计算图的使用
进行程序设计时，可以使用计算图——有向无环图——来表示 task 级并行的执行顺序与相互关系。
计算图的组成：
- 节点：表示并行模型中的 task
- 有向边：task 之间的相互转移关系
- Continue edge：表示需要顺序执行的 task
- Fork edge：表示 fork 操作对应的 child task
- Join edge：表示 join 操作，将不同的 child task 同步
计算图的性质：
-
如何判断两个任务之间能否并行？
图中两个节点之间如果有直接路径则表示有相互关系，不可并行；如果两个节点之间没有路径表示可以并行
-
数据竞争（data race）
要防止相互并行的节点之间有读写访问的竞争性
-
衡量并行模型的表现
- WORK：表示模型中 task 串行时的表现，即图中所有节点表示的 task 的执行时间和
- SPAN：图中最长的路径长度，表示整个并行模型的执行时间
- ideal parallelism = W O R K ( G ) S P A N ( G ) \text{ideal parallelism}=\frac{WORK(G)}{SPAN(G)} ideal parallelism=SPAN(G)WORK(G)
多处理器规划与并行加速
T p T_{p} Tp 表示有 p p p 个处理器的执行时间。多处理器的执行时间不仅和任务本身的执行时间、计算图有关，而且和任务规划中的执行方式有关。满足：
T 1 = W O R K T ∞ = S P A N T ∞ ≤ T p ≤ T 1 \begin{aligned} T_{1}&=WORK \\ T_{\infin}&=SPAN \\ T_{\infin}&\le{T_{p}}\le{T_{1}} \\ \end{aligned} T1T∞T∞=WORK=SPAN≤Tp≤T1
由此得到规划方式的加速比计算：
S P E E D U P = T 1 T p S P E E D U P ≤ p S P E E D U P ≤ ideal parallelism SPEEDUP=\frac{T_{1}}{T_{p}} \\ SPEEDUP\le{p} \\ SPEEDUP\le{\text{ideal parallelism}} SPEEDUP=TpT1SPEEDUP≤pSPEEDUP≤ideal parallelism
阿姆达尔定律（Amdahl’s law）
给定一组 task，需要多少处理器能够得到最佳的加速比？
-
已知计算图的情况下，可以通过计算 W O R K S P A N \frac{WORK}{SPAN} SPANWORK 得到 S P E E D U P SPEEDUP SPEEDUP 的上界
-
未知计算图的情况下，根据 Amdahl’s law，考虑 task 中顺序执行的部分所占的比例 q q q， S P E E D U P ≤ 1 q SPEEDUP\le{\frac{1}{q}} SPEEDUP≤q1
∵ S P A N ≥ q × W O R K ∴ S P E E D U P ≤ W O R K S P A N = W O R K q × W O R K = 1 q \begin{aligned} &\because{SPAN\ge{q\times{WORK}}} \\ &\therefore{SPEEDUP\le{\frac{WORK}{SPAN}}=\frac{WORK}{q\times{WORK}}=\frac{1}{q}} \end{aligned} ∵SPAN≥q×WORK∴SPEEDUP≤SPANWORK=q×WORKWORK=q1
程序实例
使用 async-finish API
package edu.coursera.parallel;
import static edu.rice.pcdp.PCDP.async;
import static edu.rice.pcdp.PCDP.finish;
public final class Compare {
private static double sum1;
private static double sum2;
Compare() {
}
public static void main(String[] args) {
double[] array = new double[20000000];
for (int i = 0; i < 20000000; i++) {
array[i] = i + 1;
}
for (int i = 0; i < 10; i++) {
System.out.println(String.format("This is run for %d/5", i + 1));
parallel(array);
sequential(array);
}
}
public static void sequential(final double... array) {
sum1 = 0.;
sum2 = 0.;
long start = System.nanoTime();
for (int i = 0; i < array.length / 2; i++) {
sum1 += 1 / array[i];
}
for (int j = array.length / 2; j < array.length; j++) {
sum2 += 1 / array[j];
}
double seqResult = sum1 + sum2;
long seqTime = System.nanoTime() - start;
System.out.println(String
.format("sequential version time: %#.2f, result is %#.4f",
seqTime / 1e6, seqResult));
}
public static void parallel(final double... array) {
sum1 = 0.;
sum2 = 0.;
long start = System.nanoTime();
finish(() -> {
async(() -> {
for (int i = 0; i < array.length / 2; i++) {
sum1 += 1 / array[i];
}
});
for (int j = array.length / 2; j < array.length; j++) {
sum2 += 1 / array[j];
}
});
double parResult = sum1 + sum2;
long parTime = System.nanoTime() - start;
System.out.println(String
.format("parallel version time: %#.2f, result is %#.4f",
parTime / 1e6, parResult));
}
}
Result:
This is run for 9/5
parallel version time: 11.39, result is 17.3885
sequential version time: 22.93, result is 17.3885
This is run for 10/5
parallel version time: 11.87, result is 17.3885
sequential version time: 21.44, result is 17.3885
使用 fork-join Framework
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.RecursiveAction;
/**
* Class wrapping methods for implementing reciprocal array sum in parallel.
*/
public final class ReciprocalArraySum {
/**
* Default constructor.
*/
private ReciprocalArraySum() {
}
/**
* Sequentially compute the sum of the reciprocal values for a given array.
*
* @param input Input array
* @return The sum of the reciprocals of the array input
*/
protected static double seqArraySum(final double[] input) {
double sum = 0;
// Compute sum of reciprocals of array elements
for (int i = 0; i < input.length; i++) {
sum += 1 / input[i];
}
return sum;
}
/**
* This class stub can be filled in to implement the body of each task
* created to perform reciprocal array sum in parallel.
*/
private static class ReciprocalArraySumTask extends RecursiveAction {
static int SEQUENTIAL_THRESHOLD = 5;
private static int chunkSize;
/**
* Starting index for traversal done by this task.
*/
private final int startIndexInclusive;
/**
* Ending index for traversal done by this task.
*/
private final int endIndexExclusive;
/**
* Input array to reciprocal sum.
*/
private final double[] input;
/**
* Intermediate value produced by this task.
*/
private double value;
/**
* Constructor.
* @param setStartIndexInclusive Set the starting index to begin
* parallel traversal at.
* @param setEndIndexExclusive Set ending index for parallel traversal.
* @param setInput Input values
*/
ReciprocalArraySumTask(final int setStartIndexInclusive,
final int setEndIndexExclusive, final double[] setInput) {
this.startIndexInclusive = setStartIndexInclusive;
this.endIndexExclusive = setEndIndexExclusive;
this.input = setInput;
SEQUENTIAL_THRESHOLD = setInput.length + chunkSize - 1) / chunkSize;
}
public static void setChunkSize(int chunk) {
chunkSize = chunk;
}
/**
* Getter for the value produced by this task.
* @return Value produced by this task
*/
public double getValue() {
return value;
}
@Override
protected void compute() {
if (endIndexExclusive - startIndexInclusive <= SEQUENTIAL_THRESHOLD) {
for (int i = startIndexInclusive; i < endIndexExclusive; i++) {
value += 1 / input[i];
}
} else {
ReciprocalArraySumTask left =
new ReciprocalArraySumTask(startIndexInclusive,
(startIndexInclusive + endIndexExclusive)/2, input);
ReciprocalArraySumTask right =
new ReciprocalArraySumTask(
(startIndexInclusive + endIndexExclusive)/2,
endIndexExclusive, input);
left.fork();
right.compute();
left.join();
value = left.getValue() + right.getValue();
}
}
}
/**
* TODO: Extend the work you did to implement parArraySum to use a set
* number of tasks to compute the reciprocal array sum. You may find the
* above utilities getChunkStartInclusive and getChunkEndExclusive helpful
* in computing the range of element indices that belong to each chunk.
*
* @param input Input array
* @param numTasks The number of tasks to create
* @return The sum of the reciprocals of the array input
*/
protected static double parManyTaskArraySum(final double[] input,
final int numTasks) {
System.setProperty("java.util.concurrent.ForkJoinPool.common"
+ ".parallelism", String.format("%d", numTasks+2));
ReciprocalArraySumTask.setChunkSize(numTasks);
ReciprocalArraySumTask t = new ReciprocalArraySumTask(0, input.length
, input);
ForkJoinPool.commonPool().invoke(t);
double sum = t.getValue();
return sum;
}
}