记录:Dinky 连接Flink推送任务local class incompatible: stream classdesc serialVersionUID = -61472550461454874

 

 Dinky推送成功,Flink 日志异常

2022-09-23 14:37:39
org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot instantiate user function.
	at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:340)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.createOperator(OperatorChain.java:713)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.createOperatorChain(OperatorChain.java:686)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.createOutputCollector(OperatorChain.java:626)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:187)
	at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.(RegularOperatorChain.java:63)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:666)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:654)
	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958)
	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:927)
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.InvalidClassException: org.apache.flink.streaming.connectors.elasticsearch.table.KeyExtractor; local class incompatible: stream classdesc serialVersionUID = -6147255046145487447, local class serialVersionUID = 3716815239130141371
	at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:699)
	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1885)
	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1751)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2042)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
	at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:617)
	at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:602)
	at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:589)
	at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:543)
	at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:324)
	... 12 more

 

场景:

我的Dinky版本0.6.7,对应的Flink版本是1.14部署到linux上,

记录:Dinky 连接Flink推送任务local class incompatible: stream classdesc serialVersionUID = -61472550461454874_第1张图片

 

linux上部署的Flink版本是flink-1.14.5,

引用的JAR

记录:Dinky 连接Flink推送任务local class incompatible: stream classdesc serialVersionUID = -61472550461454874_第2张图片

 

ES7.6

根据一下连接进行Demo练习,在flink SQL中执行这个demo是没问题的

基于 Flink CDC 构建 MySQL 和 Postgres 的 Streaming ETL

,随后我想在dlink中连接flink执行,就一直报错,

上述异常,解决方案,最后把Flink的lib下的ES Jar包,版本降低到1.14.0

记录:Dinky 连接Flink推送任务local class incompatible: stream classdesc serialVersionUID = -61472550461454874_第3张图片

 异常2:java.lang.ClassNotFoundException: org.apache.flink.formats.json.JsonRowDataSerializationSchema

 这个原因是因为Flink没有引入flink.json的Jar包

记录:Dinky 连接Flink推送任务local class incompatible: stream classdesc serialVersionUID = -61472550461454874_第4张图片

 

 重启flink

记录:Dinky 连接Flink推送任务local class incompatible: stream classdesc serialVersionUID = -61472550461454874_第5张图片

 排查原因

Dinky成功推送的,Flink运行失败,原因还在于Flink,需要从Flink这边排查

在此记录一下问题

你可能感兴趣的:(Dinky,Flink,java,spring,junit)