运行一个hive脚本报错:
[2013-02-25 14:54:00.372]Hadoop job information for Stage-1: number of mappers: 25; number of reducers: 50
[2013-02-25 14:54:00.377]
org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters - Counters=120 Limit=120
[2013-02-25 14:54:00.377] at org.apache.hadoop.mapred. Counters$Group. getCounterForName(Counters. java:312)
[2013-02-25 14:54:00.377] at org.apache.hadoop.mapred. Counters.findCounter(Counters. java:431)
[2013-02-25 14:54:00.377] at org.apache.hadoop.mapred. Counters.getCounter(Counters. java:495)
[2013-02-25 14:54:00.377] at org.apache.hadoop.hive.ql. exec.Operator.updateCounters( Operator.java:1150)
[2013-02-25 14:54:00.377] at org.apache.hadoop.hive.ql. exec.ExecDriver. updateCounters(ExecDriver. java:1000)
[2013-02-25 14:54:00.377] at org.apache.hadoop.hive.ql. exec.HadoopJobExecHelper. updateCounters( HadoopJobExecHelper.java:85)
[2013-02-25 14:54:00.377] at org.apache.hadoop.hive.ql. exec.HadoopJobExecHelper. progress(HadoopJobExecHelper. java:312)
[2013-02-25 14:54:00.378] at org.apache.hadoop.hive.ql. exec.HadoopJobExecHelper. progress(HadoopJobExecHelper. java:685)
[2013-02-25 14:54:00.378] at org.apache.hadoop.hive.ql. exec.ExecDriver.execute( ExecDriver.java:458)
[2013-02-25 14:54:00.378] at org.apache.hadoop.hive.ql. exec.MapRedTask.execute( MapRedTask.java:136)
[2013-02-25 14:54:00.378] at org.apache.hadoop.hive.ql. exec.Task.executeTask(Task. java:133)
[2013-02-25 14:54:00.378] at org.apache.hadoop.hive.ql. exec.TaskRunner.runSequential( TaskRunner.java:57)
[2013-02-25 14:54:00.378] at org.apache.hadoop.hive.ql. exec.TaskRunner.run( TaskRunner.java:47)
[2013-02-25 14:54:00.378]Ended Job = job_201301181421_100273 with exception 'org.apache.hadoop.mapred. Counters$ CountersExceededException( Error: Exceeded limits on number of counters - Counters=120 Limit=120)'
[2013-02-25 14:54:02.122]FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql. exec.MapRedTask
解决:
./conf/mapred-site.xml 追加:
<property>
<name>mapreduce.job.counters. limit</name>
<value>1400</value>
</property>