Spark kryo Jar包冲突问题排查

错误日志

Exception in thread "adaptive-query-stage-0" java.lang.NoSuchMethodError: com.esotericsoftware.kryo.Kryo.setInstantiatorStrategy(Lorg/objenesis/strategy/InstantiatorStrategy;)V
	at com.twitter.chill.KryoBase.setInstantiatorStrategy(KryoBase.scala:96)
	at com.twitter.chill.EmptyScalaKryoInstantiator.newKryo(ScalaKryoInstantiator.scala:59)
	at org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:97)
	at org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:324)
	at org.apache.spark.serializer.KryoSerializerInstance.(KryoSerializer.scala:309)

问题分析

• KryoBase的setInstantiatorStrategy 方法参数类型为 org.objenesis.strategy.InstantiatorStrategy，super调用Kryo.setInstantiatorStrategy() 方法，但是该方法的参数类型为com.esotericsoftware.shaded.org.objenesis.strategy.InstantiatorStrategy。
• 通过反编译 com.esotericsoftware.kryo.Kryo发现源码中确实是在引用com.esotericsoftware.shaded.org.objenesis.strategy.InstantiatorStrategy 类，太坑了.
• 和官方给的源码都不一致。在官方发布的包中出现引用shaded的类，导致chill_2.11-0.8.4.jar 和Kryo中的方法冲突。看来官方还是推荐使用kryo-shaded包，但是kryo-shaded 中jar的代码被打包时搞乱了，别人也没法用～～～
• 通过对比发现新的spark 使用了kryo-shaded-3.0.3.jar ,所以删除 kryo-2.22.jar 包，问题解决。

$ sm -d com.esotericsoftware.kryo.Kryo setInstantiatorStrategy
 declaring-class  com.twitter.chill.KryoBase                                                                                                                  
 method-name      setInstantiatorStrategy                                                                                                                     
 modifier         public                                                                                                                                      
 annotation                                                                                                                                                   
 parameters       org.objenesis.strategy.InstantiatorStrategy                                                                                                 
 return           void                                                                                                                                        
 exceptions                                                                                                                                                   

 declaring-class  com.esotericsoftware.kryo.Kryo                                                                                                              
 method-name      setInstantiatorStrategy                                                                                                                     
 modifier         public                                                                                                                                      
 annotation                                                                                                                                                   
 parameters       com.esotericsoftware.shaded.org.objenesis.strategy.InstantiatorStrategy                                                                     
 return           void                                                                                                                                        
 exceptions 

    org.objenesis.strategy.InstantiatorStrategy

$ sc -d *InstantiatorStrategy
 class-info        com.esotericsoftware.shaded.org.objenesis.strategy.InstantiatorStrategy                                                                    
 code-source       /usr/lib/spark-2.3.0-bin-ae/jars/kryo-2.22.jar                                                                                             
 name              com.esotericsoftware.shaded.org.objenesis.strategy.InstantiatorStrategy                                                                    
 isInterface       true                                                                                                                                       
 isAnnotation      false                                                                                                                                      
 isEnum            false                                                                                                                                      
 isAnonymousClass  false                                                                                                                                      
 isArray           false                                                                                                                                      
 isLocalClass      false                                                                                                                                      
 isMemberClass     false                                                                                                                                      
 isPrimitive       false                                                                                                                                      
 isSynthetic       false                                                                                                                                      
 simple-name       InstantiatorStrategy                                                                                                                       
 modifier          abstract,interface,public                                                                                                                  
 annotation                                                                                                                                                   
 interfaces                                                                                                                                                   
 super-class                                                                                                                                                  
 class-loader      +-sun.misc.Launcher$AppClassLoader@3cd1a2f1                                                                                                
                     +-sun.misc.Launcher$ExtClassLoader@d6da883                                                                                               
 classLoaderHash   3cd1a2f1                                                                                                                                   

 class-info        org.objenesis.strategy.InstantiatorStrategy                                                                                                
 code-source       /usr/lib/spark-2.3.0-bin-ae/jars/objenesis-2.1.jar                                                                                         
 name              org.objenesis.strategy.InstantiatorStrategy                                                                                                
 isInterface       true                                                                                                                                       
 isAnnotation      false                                                                                                                                      
 isEnum            false                                                                                                                                      
 isAnonymousClass  false                                                                                                                                      
 isArray           false                                                                                                                                      
 isLocalClass      false                                                                                                                                      
 isMemberClass     false                                                                                                                                      
 isPrimitive       false                                                                                                                                      
 isSynthetic       false                                                                                                                                      
 simple-name       InstantiatorStrategy                                                                                                                       
 modifier          abstract,interface,public                                                                                                                  
 annotation                                                                                                                                                   
 interfaces                                                                                                                                                   
 super-class                                                                                                                                                  
 class-loader      +-sun.misc.Launcher$AppClassLoader@3cd1a2f1                                                                                                
                     +-sun.misc.Launcher$ExtClassLoader@d6da883                                                                                               
 classLoaderHash   3cd1a2f1 


## JAD 反编译的源码
jad com.esotericsoftware.kryo.Kryo

import com.esotericsoftware.shaded.org.objenesis.instantiator.ObjectInstantiator;
import com.esotericsoftware.shaded.org.objenesis.strategy.InstantiatorStrategy;

[hadoop@hostname jars]$ ll chill*
-rw-r----- 1 hadoop hadoop 223573 7月   5 2018 chill_2.11-0.8.0.jar
-rw-r----- 1 hadoop hadoop  50619 7月   5 2018 chill-java-0.8.0.jar
[hadoop@hostname jars]$ ll kryo-shaded-3.0.3.jar 
-rw-r----- 1 hadoop hadoop 358390 7月   5 2018 kryo-shaded-3.0.3.jar

[hadoop@hostname jars]$ ll chill*
-rw-r--r-- 1 hadoop hadoop 224167 1月  16 10:57 chill_2.11-0.8.4.jar
-rw-r--r-- 1 hadoop hadoop  60483 1月  16 10:57 chill-java-0.8.4.jar
[hadoop@hostname jars]$ ll kryo-*
-rw-r--r-- 1 hadoop hadoop 442805 1月  16 10:57 kryo-2.22.jar
-rw-r--r-- 1 hadoop hadoop 358390 1月  16 10:57 kryo-shaded-3.0.3.jar