flink遇到的问题

报错Could not find Flink job

Job has been submitted with JobID 52e00d8f89544a7df6de38b45e33686e
------------------------------------------------------------
 The program finished with the following exception:
org.apache.flink.client.program.ProgramInvocationException: The main method caused an error:
 org.apache.flink.runtime.rest.util.RestClientException: [org.apache.flink.runtime.rest.handler.RestHandlerException: 
 org.apache.flink.runtime.messages.FlinkJobNotFoundException: Could not find Flink job 
 (52e00d8f89544a7df6de38b45e33686e)
        at org.apache.flink.runtime.rest.handler.job.JobExecutionResultHandler.propagateException
        (JobExecutionResultHandler.java:94)

在运行最简单的样例的时候bin/flink run -m yarn-cluster examples/batch/WordCount.jar，出现了这种报错。当时的情况是第一次成功了，后来执行的就一直报这种错误，就算重启集群也不行。如果不kill掉的话，继续开启一个新的样例时，日志会打印Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster这种日志，最后才发现是因为配置文件里面配置了cluster.id。在使用cluster模式时，如果使用per job模式，不能手动设置cluster.id，每次提交任务时，会自动生成。检查配置参数，不要配置cluster.id就会解决。当时我看官网里有这些配置就配上去了，结果导致了这种错误，搞了好多天才搞定。

Flink CDC 运行jar包读取mysql报错

	Caused by: org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
	at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42)
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleEvent(MySqlStreamingChangeEventSource.java:369)
	at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1118)
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:966)
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:606)
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:850)
	at java.lang.Thread.run(Thread.java:748)
Caused by: io.debezium.DebeziumException: Error processing binlog event
	... 6 more
Caused by: io.debezium.DebeziumException: Encountered change event for table test.test whose schema isn't known to this connector
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.informAboutUnknownTableIfRequired(MySqlStreamingChangeEventSource.java:647)
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleUpdateTableMetadata(MySqlStreamingChangeEventSource.java:627)
	at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleEvent(MySqlStreamingChangeEventSource.java:352)
	... 5 more

这个是因为在本机运行成功了，要到集群上跑就需要改ip，改了ip后就建了一个mysql账号，结果导致报了这个错误。开始还以为是mysql binlog出问题了，在网上搜索说是数据库、表名、字段名大小写的问题，结果并不是，搞了几天也没搞定。最后使用新建的账号查询数据发现没有权限，这才知道了原因，给予mysql账号相应的权限即可。

recipient is unreachable

org.apache.flink.runtime.rpc.exceptions.RecipientUnreachableException: Could not send message 
[RemoteRpcInvocation(TaskExecutorGateway.submitTask(TaskDeploymentDescriptor, JobMasterId, Time))] from sender
 [Actor[akka://flink/temp/taskmanager_0$nrzc]] to recipient [Actor[akka.tcp://flink@xxxxxxxxx:41076/user/rpc/taskmanager_0#-1541313153]], 
 because the recipient is unreachable. This can either mean that the recipient has been terminated or that the remote RpcService is currently not reachable.

在flink的webUI上的Exceptions里看的报错，这儿属于是表象，要查其他日志找到具体原因，可以通过查TaskManagers的Stdout查看日志找到原因，有可能是数据倾斜或者主键重复（未使用upsert）等原因。