Flink on yarn遇到的问题
Flink版本：flink-1.9.1
下载地址：https://flink.apache.org/downloads.html
我是分开部署的，部署了hadoop 3.1.2
所以下载 两个文件 Apache Flink 1.9.1 for Scala 2.12 和Pre-bundled Hadoop 2.8.3 （一个jar包）
官方文档：https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/deployment/yarn_setup.html
Flink on yarn 部署：
1. 直接解压 flink-1.9.1-bin-scala_2.12.tgz
然后将上面的jar包（flink-shaded-hadoop-2-uber-2.8.3-7.0.jar）放在解压后的flink-1.9.1/lib/ 下
2.hadoop环境配置即hadoop的路径要配置进 /etc/profile
- Test if
YARN_CONF_DIR,HADOOP_CONF_DIRorHADOOP_CONF_PATHare set (in that order). If one of these variables is set, it is used to read the configuration. - If the above strategy fails (this should not be the case in a correct YARN setup), the client is using the
HADOOP_HOMEenvironment variable. If it is set, the client tries to access$HADOOP_HOME/etc/hadoop(Hadoop 2) and$HADOOP_HOME/conf(Hadoop 1).
export HADOOP_HOME=/home/hadoop-3.1.23.提交任务
flink-1.9.1/bin/flink run -m yarn-cluster -yjm 2048 -ytm 2048 /home/wordCount.jar去hadoop集群上查看任务
http://127.0.0.1:8088/
4.查看运行日志 在hadoop的logs中，taskmanager.log
常见错误
The program didn't contain a Flink job. Perhaps you forgot to call execute() on the execution environment
明明运行了execute()，这个时候要查日志找到具体原因
日志在各node节点的/hadoop-3.1.2/logs/userlogs/ 下面的jobmanager.log和taskmanager.log中
问题
1.org.apache.kafka.common.errors.TimeoutException: Failed to get offsets by times in 60000ms
flink+kafka 遇到的问题，防火墙读关了，本地正常，服务器上就跑不了，google也查不到解决方式。
原因是kafka的问题
解决：
修改kafka配置 kafka_2.12-2.3.0/config/server.properties
#listeners=PLAINTEXT://host_name:9092
这个注释解开，配上yarn集群所在机器能访问的 ip或者hostname
2.其他很多错误，反正大部分是yarn集群的问题导致的，所以要确认yarn集群正确，并不是跑起来就可以
比如：
Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
The number of requested virtual cores for application master 1 exceeds the maximum number of virtual cores 0 available in the Yarn Cluster.
其他等等，解决这些问题要重新配置好yarn集群，参考我的另一篇博客
https://blog.csdn.net/cgb278653219/article/details/103699317
3.jar包不正确导致的无法运行
打包方式网上搜一下