scala写MR作业时,出现容器启动失败container lunch failed

使用scala写MapReduce作业代码,在运行时出现错误:

14/03/21 17:49:43 INFO mapreduce.Job: Job job_1395395314485_0001 failed with state FAILED due to: Application application_1395395314485_0001 failed 2 times due to AM Container for appattempt_1395395314485_0001_000002 exited with  exitCode: 1 due to: Exception from container-launch: 
org.apache.hadoop.util.Shell$ExitCodeException: 
at org.apache.hadoop.util.Shell.runCommand(Shell.java:458)
at org.apache.hadoop.util.Shell.run(Shell.java:373)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:578)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:208)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:282)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:76)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

根据网友的经验,https://issues.apache.org/jira/browse/YARN-1473 ,认为是yarn-site.xml中的参数‘yarn.application.classpath’导致,默认值为:


	yarn.application.classpath
	$HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/share/hadoop/common/*,$HADOOP_COMMON_HOME/share/hadoop/common/lib/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/*,$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*,$YARN_HOME/share/hadoop/mapreduce/*,$YARN_HOME/share/hadoop/mapreduce/lib/*,
去掉该参数即可。

我新建了集群进行了测试,果真添上这个参数,程序便会出现上述错误。但问题在于,原来的集群也没有设置这个参数,该参数的默认值就是如上所示的,这个时候我们该怎么办?难道要覆写这个参数?



你可能感兴趣的:(工具)