我是hadoop-2.7.6双namenode集群
1.问题:sqoop上传数据到hdfs报错:Exception: Job Failed with status:3
2015-05-08 14:29:12 CST: RUNNING - 84.47 %
2015-05-08 14:29:22 CST: RUNNING - 84.47 %
2015-05-08 14:29:32 CST: RUNNING - 84.47 %
2015-05-08 14:29:42 CST: RUNNING - 84.47 %
2015-05-08 14:29:52 CST: RUNNING - 84.47 %
2015-05-08 14:30:02 CST: RUNNING - 84.47 %
2015-05-08 14:30:13 CST: RUNNING - 84.47 %
2015-05-08 14:30:23 CST: RUNNING - 84.47 %
2015-05-08 14:30:33 CST: RUNNING - 84.47 %
2015-05-08 14:30:43 CST: RUNNING - 84.47 %
2015-05-08 14:30:53 CST: RUNNING - 84.47 %
2015-05-08 14:31:04 CST: RUNNING - 84.47 %
2015-05-08 14:31:14 CST: RUNNING - 84.47 %
2015-05-08 14:31:24 CST: RUNNING - 84.47 %
2015-05-08 14:31:34 CST: RUNNING - 84.47 %
2015-05-08 14:31:44 CST: RUNNING - 84.47 %
2015-05-08 14:31:54 CST: RUNNING - 84.47 %
2015-05-08 14:32:04 CST: RUNNING - 84.47 %
2015-05-08 14:32:14 CST: RUNNING - 100.00 %
2015-05-08 14:32:47 CST: FAILED
Exception: Job Failed with status:3
原因:该问题是由于内存不足造成!!
解决方案:
限制资源
修改hadoop配置文件:etc/hadoop/yarn-site.xml
注意:以下的qy0102是我的主机名,记得改成自己的!!!
yarn.resourcemanager.ha.enabled
true
yarn.resourcemanager.cluster-id
yrc
yarn.resourcemanager.ha.rm-ids
rm1,rm2
yarn.resourcemanager.hostname.rm1
qy0102
yarn.resourcemanager.hostname.rm2
lxy0117
yarn.resourcemanager.zk-address
qy0102:2181,lxy0117:2181,zys0125:2181
yarn.nodemanager.aux-services
mapreduce_shuffle