解决CDH-6.3.2 hive on spark 运行报错: Error while processing statement: FAILED: Execution Error, return co

在进行CDH-6.3.2 hive on spark 作业时,Hue一直报错下面内容

Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.spark.SparkTask.
 Job hasn't been submitted after 61s. Aborting it. Possible reasons include network issues, errors in remote driver or the cluster 
 has no available resources, etc. Please check YARN or Spark driver's logs for further information. The timeout is controlled by 
 hive.spark.job.monitor.timeout.

报错截图如下:
在这里插入图片描述
yarn上报错如下:

Uncaught exception: org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException: Invalid resource request! Cannot allocate containers as requested resource is greater than maximum allowed allocation. Requested resource type=[vcores], Requested resource=, maximum allowed allocation=, please note that maximum allowed allocation is calculated by scheduler based on maximum resource of registered NodeManagers, which might be less than configured maximum allocation=

费两个多小时,最终解决了
解决步骤如下:
1、去yarn All Applications查看作业情况,如下图:
解决CDH-6.3.2 hive on spark 运行报错: Error while processing statement: FAILED: Execution Error, return co_第1张图片
上图报错内容是我在执行作业时申请的cpu核数超过了yarn的最大核数4个,为此报错。

2、去hive的配置页面设置spark作业申请的核数,如下图
解决CDH-6.3.2 hive on spark 运行报错: Error while processing statement: FAILED: Execution Error, return co_第2张图片
3、再次在hue里面提交hive SQL执行就成功啦
解决CDH-6.3.2 hive on spark 运行报错: Error while processing statement: FAILED: Execution Error, return co_第3张图片
解决CDH-6.3.2 hive on spark 运行报错: Error while processing statement: FAILED: Execution Error, return co_第4张图片

你可能感兴趣的:(大数据开发平台搭建踩坑记录,spark,hive,大数据,Hue,CDH-6.3.2)