HQL报错:Error: Java heap space Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

一 错误

Starting Job = job_1547323088343_0010, Tracking URL = http://hadoop01:8088/proxy/application_1547323088343_0010/
Kill Command = /home/hadoop/install/hadoop-2.5.0-cdh5.3.6//bin/hadoop job  -kill job_1547323088343_0010
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2019-01-15 16:46:21,433 Stage-1 map = 0%,  reduce = 0%
2019-01-15 16:46:44,308 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_1547323088343_0010 with errors
Error during job, obtaining debugging information...
Examining task ID: task_1547323088343_0010_m_000000 (and more) from job job_1547323088343_0010

Task with the most failures(4): 
-----
Task ID:
  task_1547323088343_0010_m_000000

URL:
  http://hadoop01:8088/taskdetails.jsp?jobid=job_1547323088343_0010&tipid=task_1547323088343_0010_m_000000
-----
Diagnostic Messages for this Task:
Error: Java heap space

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

二 解决方案

(一)

查阅了很多博客,发现是HADOOP_HEAPSIZE的设置的太小,可以将其进行修改

  1. 在hive-env.sh中进行更改
export HADOOP_HEAPSIZE=4096

注意:设置完成后,重新启动hive的两个服务,此时应该可以了!!!

(二)

如果此方案不好使,可以试试这个

  1. 可以在Hadoop的/etc/hadoop下的mapred-site.xml中添加如下配置:

   yarn.app.mapreduce.am.command-opts
   -Djava.net.preferIPv4Stack=true -Xmx100m
 
 
   mapreduce.map.java.opts
   -Djava.net.preferIPv4Stack=true -Xmx100m 
 
 
   mapreduce.reduce.java.opts
   -Djava.net.preferIPv4Stack=true -Xmx100m
 

注意:重新启动集群!!!

可以参考的链接:https://blog.csdn.net/u011563666/article/details/79033226
http://community.cloudera.com/t5/Batch-SQL-Apache-Hive/Hive-Java-heap-error-running-query-exit-code-143/td-p/8668

你可能感兴趣的:(常见错误总结)