[flink-009]jar包在flink的执行过程

1.执行java包的命令是 ./bin/flink -c MainClassName x.jar

2. bin/flink脚本,启动 类 org.apache.flink.client.cli.CliFrontend 执行jar包。该类位于flink/flink-clients/src/main/java/org/apache/flink/client/cli/CliFrontend.java

3.本地StandAlone集群的任务提交和执行流程

CliFrontend.main()-->

CliFrontend.parseParameters(args)-->

CliFrontend.run(args)-->

CliFrontend.buildProgram(runOptions)-->

CliFrontend.runProgram(customCommandLine, commandLine, runOptions, program)-->

ClusterDescriptor.deploySessionCluster(clusterSpecification)-->

ClusterDescriptor.executeProgram(program, client, userParallelism)-->

PackagedProgram.invokeInteractiveModeForExecution()-->

PackagedProgram.callMainMethod(Class entryClass, String[] args)-->

mainMethod.invoke(null, (Object) args)

任务提交后,不需要发送到JobManger,就在当前进程内执行。

4.分布式集群提交执行

CliFrontend.main()-->

CliFrontend.parseParameters(args)-->

CliFrontend.run(args)-->

CliFrontend.buildProgram(runOptions)-->

CliFrontend.runProgram(customCommandLine, commandLine, runOptions, program)-->

final JobGraph jobGraph = PackagedProgramUtils.createJobGraph(program, configuration, parallelism)-->

clusterDescriptor.deployJobCluster(clusterSpecification,jobGraph,runOptions.getDetachedMode())-->

如果是提交到分布式集群,JobGraph jobGraph = PackagedProgramUtils.createJobGraph(program, configuration, parallelism)这句比较重要,jobGraph是提交给JobManger的,它是JobManger接受的任务对象。

在deployJobCluter函数调用时,Yarn集群的client,会根据jobGraph里的信息,把任务的所有jar包上传到Yarn集群的文件存储home,然后启动任务。

 

你可能感兴趣的:(flink,flink)