Hive memory error:beyond the cluster container capability

阅读更多

问题：
java.lang.Exception: java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex's TaskResource is beyond the cluster container capability,Vertex=vertex_1517380657411_0232_1_00 [Map 1], Requested TaskResource=, Cluster MaxContainerCapability=


look at the values for following two variables mapreduce.map.memory.mb and mapreduce.reduce.memory.mb

and you could make the vaules as same as the container size.

there is nice HCC article explaining how this allocations work here

alternately if you don't have any more resources you may use the mr-engine instead of Tez ( though it is slow will complete the tasks with lesser concurrent memory utilization) // to change the engine run "hive.execution.engine=mr" for this query.