Failing this attempt.Diagnostics: [2019-03-28 15:05:27.576]File does not exist:

在搬砖过程,出现了一个很奇怪的现象,程序虽然报错了,但是有正常的数据输出。

Application application_1547711386718_69741 failed 2 times due to AM Container for appattempt_1547711386718_69741_000002 exited with exitCode: -1000
Failing this attempt.Diagnostics: [2019-03-28 15:05:27.576]File does not exist: hdfs://nameservice1/user/dc/.sparkStaging/application_1547711386718_69741/__spark_libs__7345683000560804009.zip
java.io.FileNotFoundException: File does not exist: hdfs://nameservice1/user/dc/.sparkStaging/application_1547711386718_69741/__spark_libs__7345683000560804009.zip
at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1498)
at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1491)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1506)
at org.apache.hadoop.yarn.util.FSDownload.verifyAndCopy(FSDownload.java:269)
at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:67)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:414)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:411)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1685)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:411)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.doDownloadCall(ContainerLocalizer.java:242)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:235)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer$FSDownloadWrapper.call(ContainerLocalizer.java:223)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
For more detailed output, check the application tracking page: http://hadoop-namenode-2:8088/cluster/app/application_1547711386718_69741 Then click on links to logs of each attempt.
. Failing the application.

最后发现是由于在本地调试的时候 `conf.setMaster(“local”)没有注释掉

参考:https://blog.csdn.net/adorechen/article/details/78746363

你可能感兴趣的:(日常BUG)