二十一、Flink源码阅读--ExecutionGraph生成过程

在上一篇服务端处理jobGraph提到，jobGraph是转换为executionGraph，最后被执行调度的，那么本篇我们看下这一过程的源码

源码分析

从JobMaster的启动开始讲起
JobMaster.start—>JobMaster.startJobExecution—>resetAndScheduleExecutionGraph方法将JobGraph转换为 ExecutionGraph

private void resetAndScheduleExecutionGraph() throws Exception {
	validateRunsInMainThread();

	final CompletableFuture executionGraphAssignedFuture;

	if (executionGraph.getState() == JobStatus.CREATED) {
		executionGraphAssignedFuture = CompletableFuture.completedFuture(null);
	} else {
		suspendAndClearExecutionGraphFields(new FlinkException("ExecutionGraph is being reset in order to be rescheduled."));
		final JobManagerJobMetricGroup newJobManagerJobMetricGroup = jobMetricGroupFactory.create(jobGraph);
		final ExecutionGraph newExecutionGraph = createAndRestoreExecutionGraph(newJobManagerJobMetricGroup);//生成executionGraph

		executionGraphAssignedFuture = executionGraph.getTerminationFuture().handleAsync(
			(JobStatus ignored, Throwable throwable) -> {
				assignExecutionGraph(newExecutionGraph, newJobManagerJobMetricGroup);
				return null;
			},
			getMainThreadExecutor());
	}

	executionGraphAssignedFuture.thenRun(this::scheduleExecutionGraph);//执行executionGraph
}

详细步骤进入createAndRestoreExecutionGraph方法

private ExecutionGraph createAndRestoreExecutionGraph(JobManagerJobMetricGroup currentJobManagerJobMetricGroup) throws Exception {
	ExecutionGraph newExecutionGraph = createExecutionGraph(currentJobManagerJobMetricGroup);
	...
	return newExecutionGraph;
}
-----> 
private ExecutionGraph createExecutionGraph(JobManagerJobMetricGroup currentJobManagerJobMetricGroup) throws JobExecutionException, JobException {
	return ExecutionGraphBuilder.buildGraph(
		null,
		jobGraph,
		jobMasterConfiguration.getConfiguration(),
		scheduledExecutorService,
		scheduledExecutorService,
		slotPool.getSlotProvider(),
		userCodeLoader,
		highAvailabilityServices.getCheckpointRecoveryFactory(),
		rpcTimeout,
		restartStrategy,
		currentJobManagerJobMetricGroup,
		blobServer,
		jobMasterConfiguration.getSlotRequestTimeout(),
		log);
}
-----> 
public static ExecutionGraph buildGraph(
		@Nullable ExecutionGraph prior,
		JobGraph jobGraph,
		Configuration jobManagerConfig,
		ScheduledExecutorService futureExecutor,
		Executor ioExecutor,
		SlotProvider slotProvider,
		ClassLoader classLoader,
		CheckpointRecoveryFactory recoveryFactory,
		Time rpcTimeout,
		RestartStrategy restartStrategy,
		MetricGroup metrics,
		int parallelismForAutoMax,
		BlobWriter blobWriter,
		Time allocationTimeout,
		Logger log)
	throws JobExecutionException, JobException {

	...

	for (JobVertex vertex : jobGraph.getVertices()) {
		String executableClass = vertex.getInvokableClassName();
		if (executableClass == null || executableClass.isEmpty()) {//检查operator的class
			throw new JobSubmissionException(jobId,
					"The vertex " + vertex.getID() + " (" + vertex.getName() + ") has no invokable class.");
		}

		if (vertex.getParallelism() == ExecutionConfig.PARALLELISM_AUTO_MAX) {
			if (parallelismForAutoMax < 0) {
				throw new JobSubmissionException(
					jobId,
					PARALLELISM_AUTO_MAX_ERROR_MESSAGE);
			}
			else {
				vertex.setParallelism(parallelismForAutoMax);//设置最大并发
			}
		}

		try {
			vertex.initializeOnMaster(classLoader);
		}
		catch (Throwable t) {
				throw new JobExecutionException(jobId,
						"Cannot initialize task '" + vertex.getName() + "': " + t.getMessage(), t);
		}
	}

...

	executionGraph.attachJobGraph(sortedTopology);//核心代码，根据JobVertex列表生成executionGraph

...

	//设置checkpoint相关

    //设置metric相关

	return executionGraph;
}

跟进到attachJobGraph方法，遍历JobVertex列表,每个jobVertex生成对应的ExecutionJobVertex

ExecutionJobVertex ejv = new ExecutionJobVertex(//jobVertex生成ExecutionJobVertex
	this,
	jobVertex,
	1,
	rpcTimeout,
	globalModVersion,
	createTimestamp);

ejv.connectToPredecessors(this.intermediateResults);//将ExecutionJobVertex和IntermediateResult连接起来

核心方法就是ExecutionJobVertex的构造函数

// create the intermediate results
this.producedDataSets = new IntermediateResult[jobVertex.getNumberOfProducedIntermediateDataSets()];//根据
IntermediateDataSet个数创建IntermediateDataReult数组，数组大小一致

for (int i = 0; i < jobVertex.getProducedDataSets().size(); i++) {//将IntermediateDataSet 转为 IntermediateDataResult
	final IntermediateDataSet result = jobVertex.getProducedDataSets().get(i);

	this.producedDataSets[i] = new IntermediateResult(
			result.getId(),
			this,
			numTaskVertices,
			result.getResultType());
}

// create all task vertices，将每个ExecutionJobVertex 转为 ExecutionVertex
for (int i = 0; i < numTaskVertices; i++) {
	ExecutionVertex vertex = new ExecutionVertex(
			this,
			i,
			producedDataSets,
			timeout,
			initialGlobalModVersion,
			createTimestamp,
			maxPriorAttemptsHistoryLength);

	this.taskVertices[i] = vertex;
}

我们在看一看ExecutionVertex的构造函数

for (IntermediateResult result : producedDataSets) {//将IntermediateResult转为IntermediateResultPartition中，放到map集合
	IntermediateResultPartition irp = new IntermediateResultPartition(result, this, subTaskIndex);
	result.setPartition(subTaskIndex, irp);

	resultPartitions.put(irp.getPartitionId(), irp);
}

this.inputEdges = new ExecutionEdge[jobVertex.getJobVertex().getInputs().size()][];//二维数组

this.priorExecutions = new EvictingBoundedList<>(maxPriorExecutionHistoryLength);

this.currentExecution = new Execution(//创建物理Execution，即具体的物理执行
	getExecutionGraph().getFutureExecutor(),
	this,
	0,
	initialGlobalModVersion,
	createTimestamp,
	timeout);

具体的将ExecutionJobVertex和IntermediateResult连接起来的方法connectToPredecessors

public void connectToPredecessors(Map intermediateDataSets) throws JobException {

	List inputs = jobVertex.getInputs();

	if (LOG.isDebugEnabled()) {
		LOG.debug(String.format("Connecting ExecutionJobVertex %s (%s) to %d predecessors.", jobVertex.getID(), jobVertex.getName(), inputs.size()));
	}

	for (int num = 0; num < inputs.size(); num++) {
		JobEdge edge = inputs.get(num);

		if (LOG.isDebugEnabled()) {
			if (edge.getSource() == null) {
				LOG.debug(String.format("Connecting input %d of vertex %s (%s) to intermediate result referenced via ID %s.",
						num, jobVertex.getID(), jobVertex.getName(), edge.getSourceId()));
			} else {
				LOG.debug(String.format("Connecting input %d of vertex %s (%s) to intermediate result referenced via predecessor %s (%s).",
						num, jobVertex.getID(), jobVertex.getName(), edge.getSource().getProducer().getID(), edge.getSource().getProducer().getName()));
			}
		}

		// fetch the intermediate result via ID. if it does not exist, then it either has not been created, or the order
		// in which this method is called for the job vertices is not a topological order
		IntermediateResult ires = intermediateDataSets.get(edge.getSourceId());
		if (ires == null) {
			throw new JobException("Cannot connect this job graph to the previous graph. No previous intermediate result found for ID "
					+ edge.getSourceId());
		}

		this.inputs.add(ires);

		int consumerIndex = ires.registerConsumer();

		for (int i = 0; i < parallelism; i++) {
			ExecutionVertex ev = taskVertices[i];
			ev.connectSource(num, ires, edge, consumerIndex);// 将ExecutionVertex与IntermediateResult关联起来
		}
	}
}

在进入 connectAllToAll 方法 ，其实就是ExecutionEdge，IntermediateResultPartition关联。

总结：

• 在 ExecutionGraph 中，节点对应的类是 ExecutionJobVertex，对应的就是 JobGraph 中的 JobVertex。每一个 ExexutionJobVertex 都是由一个 JobVertex 生成的
• 一个ExecutionJobVertex在底层等于并行度个ExecutionVertex
• 一个IntermediateDataset相当于一个IntermediateResult
• 一个ExecutionVertex相当于最后的一个Execution，Execution 是对 ExecutionVertex 的一次执行，通过 ExecutionAttemptId 来唯一标识
• 一个IntermediateResult分为多个IntermediateResultPartition

JobGraph : JobVertex + IntermediateDateSet
IntermediateDataSet 的 producer是JobVertex，consumers是List
JobEdge的target 是JobVertex , source是IntermediateDataSet

ExecutionGraph: ExecutionJobVertex + IntermediateDateResult
IntermediateResult 的 producer 是ExecutionJobVertex

物理执行图：ExecutionVertex + IntermediatePartition
IntermediatePartition的producer是ExecutionVertex，consumers是List
ExecutionEdge的的source是IntermediateResultPartition， target 是ExecutionVertex