FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

• 问题描述
  现在要将以下字符串中的每个iid解析出来

[ nt6u5iystv32ne2 ; 1hnc62lteegcjhu ; 9124fswltwxsc48 ; 8tkjl4n622ft97f ; 1tuxop0iyhlmmz5 ; uiq0rxuywu508qr ; 0lyo1h8np3w3trk ; e7hi6lep1yc51ca ; 4pyadnvmlz56qhp ; m2lbjjlfidqc8iu ]

• 问题分析

  • 首先需要去除’[‘和’]’
  • 然后去除多余的空格
  • 最后按照’;’split一下就可以了

• 运行代码（假定记录在tag_model.temptemp_test_x表）

select split(regexp_replace(iids, '[\\[\\]\\s]', ''), '\073') from from tag_model.temptemp_test_x limit 1;

• 运行结果
  

  
  
  

• 注意问题
  如果在使用regexp_replace(str, reg, rep)函数的时候，若没有进行转义操作（即，在对应字符前加’\’字符），则会报错，详细如下代码：

hive> select split(regexp_replace(iids, '[]', ''), '\073') from tag_model.temptemp_test_x limit 1;
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
[GC 27106K->10124K(83008K), 0.0020190 secs]
[GC 27117K->10108K(83008K), 0.0026310 secs]
Starting Job = job_201612260917_53446, Tracking URL = http://hadoop01.chcloud.com:50030/jobdetails.jsp?jobid=job_201612260917_53446
Kill Command = /opt/hadoop_client/libexec/../bin/hadoop job  -kill job_201612260917_53446
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2017-01-16 16:09:45,315 Stage-1 map = 0%,  reduce = 0%
2017-01-16 16:10:40,651 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201612260917_53446 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://hadoop01.chcloud.com:50030/jobdetails.jsp?jobid=job_201612260917_53446
Examining task ID: task_201612260917_53446_m_000002 (and more) from job job_201612260917_53446

Task with the most failures(4): 
-----
Task ID:
  task_201612260917_53446_m_000000

URL:
  http://hadoop01.chcloud.com:50030/taskdetails.jsp?jobid=job_201612260917_53446&tipid=task_201612260917_53446_m_000000
-----
Diagnostic Messages for this Task:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"iids":"[ w91tvwknrmu56gb ; 2ajcesik107a4hz ; 7gj7hcs736st842 ; h4a3dqxk0jhmog5 ; bzfkv5se8qaqel2 ; 50bc0b7h43smw9w ; ewzd8p5he7q7iti ; au82hc6hpfpjr7i ; 9g2gnrb3p9i4kyn ; cctv5jia201609021200 ]"}
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:195)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
        at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"iids":"[ w91tvwknrmu56gb ; 2ajcesik107a4hz ; 7gj7hcs736st842 ; h4a3dqxk0jhmog5 ; bzfkv5se8qaqel2 ; 50bc0b7h43smw9w ; ewzd8p5he7q7iti ; au82hc6hpfpjr7i ; 9g2gnrb3p9i4kyn ; cctv5jia201609021200 ]"}
        at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:550)
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177)
        ... 8 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute method public org.apache.hadoop.io.Text org.apache.hadoop.hive.ql.udf.UDFRegExpReplace.evaluate(org.apache.hadoop.io.Text,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text)  on object org.apache.hadoop.hive.ql.udf.UDFRegExpReplace@68d36ff3 of class org.apache.hadoop.hive.ql.udf.UDFRegExpReplace with arguments {[ w91tvwknrmu56gb ; 2ajcesik107a4hz ; 7gj7hcs736st842 ; h4a3dqxk0jhmog5 ; bzfkv5se8qaqel2 ; 50bc0b7h43smw9w ; ewzd8p5he7q7iti ; au82hc6hpfpjr7i ; 9g2gnrb3p9i4kyn ; cctv5jia201609021200 ]:org.apache.hadoop.io.Text, []:org.apache.hadoop.io.Text, :org.apache.hadoop.io.Text} of size 3
        at org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:1243)
        at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.evaluate(GenericUDFBridge.java:182)
        at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
        at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
        at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator$DeferredExprObject.get(ExprNodeGenericFuncEvaluator.java:77)
        at org.apache.hadoop.hive.ql.udf.generic.GenericUDFSplit.evaluate(GenericUDFSplit.java:66)
        at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator._evaluate(ExprNodeGenericFuncEvaluator.java:166)
        at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
        at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
        at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:79)
        at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:793)
        at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:92)
        at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:793)
        at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:540)
        ... 9 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:1219)
        ... 22 more
Caused by: java.util.regex.PatternSyntaxException: Unclosed character class near index 1
[]
 ^
        at java.util.regex.Pattern.error(Pattern.java:1713)
        at java.util.regex.Pattern.clazz(Pattern.java:2254)
        at java.util.regex.Pattern.sequence(Pattern.java:1818)
        at java.util.regex.Pattern.expr(Pattern.java:1752)
        at java.util.regex.Pattern.compile(Pattern.java:1460)
        at java.util.regex.Pattern.(Pattern.java:1133)
        at java.util.regex.Pattern.compile(Pattern.java:823)
        at org.apache.hadoop.hive.ql.udf.UDFRegExpReplace.evaluate(UDFRegExpReplace.java:57)
        ... 27 more


FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched: 
Job 0: Map: 1   HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

• 参考资料
  http://blog.csdn.net/unknownli/article/details/8520616