java.lang.SecurityException: Invalid signature file digest for Manifest main attributes

hadoop@master:/usr/local/hadoop/spark/spark-data$ spark-submit --master spark:192.168.71.128:7077 --class PageRank project.jar
Exception in thread "main" java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
at sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVerifier.java:284)
at sun.security.util.SignatureFileVerifier.process(SignatureFileVerifier.java:238)
at java.util.jar.JarVerifier.processEntry(JarVerifier.java:316)
at java.util.jar.JarVerifier.update(JarVerifier.java:228)
at java.util.jar.JarFile.initializeVerifier(JarFile.java:383)
at java.util.jar.JarFile.getInputStream(JarFile.java:450)
at sun.misc.JarIndex.getJarIndex(JarIndex.java:137)
at sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:675)
at sun.misc.URLClassPath$JarLoader$1.run(URLClassPath.java:667)
at java.security.AccessController.doPrivileged(Native Method)
at sun.misc.URLClassPath$JarLoader.ensureOpen(URLClassPath.java:666)
at sun.misc.URLClassPath$JarLoader.(URLClassPath.java:639)
at sun.misc.URLClassPath$3.run(URLClassPath.java:366)
at sun.misc.URLClassPath$3.run(URLClassPath.java:356)
at java.security.AccessController.doPrivileged(Native Method)
at sun.misc.URLClassPath.getLoader(URLClassPath.java:355)
at sun.misc.URLClassPath.getLoader(URLClassPath.java:332)
at sun.misc.URLClassPath.getResource(URLClassPath.java:198)
at java.net.URLClassLoader$1.run(URLClassLoader.java:364)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)

在写mapreduce程序时，打包到集群上面运行时，会出现这样的错误。

打开META-INF目录，将*.SF,*.DSA,*.RSA文件删除，即可。应为有些包有签名，导致错误。

在Linux系统下直接运行：

 

hadoop@master:/usr/local/hadoop/spark/spark-data$ sudo zip -d project.jar META-INF/*.RSA META-INF/*.DSA META-INF/*.SF
[sudo] hadoop 的密码：
zip warning: name not matched: META-INF/*.RSA
deleting: META-INF/BCKEY.SF
deleting: META-INF/BCKEY.DSA
deleting: META-INF/DUMMY.SF
deleting: META-INF/DUMMY.DSA
deleting: META-INF/DEV.SF
deleting: META-INF/DEV.DSA
hadoop@master:/usr/local/hadoop/spark/spark-data$

 

这个问题是在maven打包之后由于重复引用某些依赖导致生成了一些.SF等文件，运行jar时会抛出。