sqoop导出hive数据到mysql错误ava.util.NoSuchElementException解决办法

sqoop错误日志:

14/05/08 12:25:40 INFO mapred.JobClient: Task Id : attempt_201405071437_0013_m_000000_0, Status : FAILED
java.io.IOException: Can't export data, please check task tracker logs
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(ArrayList.java:794)
at wordcount.__loadFromFields(wordcount.java:198)
at wordcount.parse(wordcount.java:147)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83)
... 10 more


14/05/08 12:25:40 INFO mapred.JobClient: Task Id : attempt_201405071437_0013_m_000001_0, Status : FAILED
java.io.IOException: Can't export data, please check task tracker logs
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(ArrayList.java:794)
at wordcount.__loadFromFields(wordcount.java:198)
at wordcount.parse(wordcount.java:147)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83)
... 10 more


sqoop导出语句:

sqoop export --connect jdbc:mysql://icity0:3306/sqoop --username root --password root --table wordcount --input-fields-terminated-by '\001' --export-dir '/user/hive/warehouse/wordcount'


这个错误的原因是指定Hive中表字段之间使用的分隔符错误,供Sqoop读取解析不正确,由于此处hive中的数据是由hdfs中导入的,其分隔符是'\t',故修改从下列的导出语句即可:

sqoop export --connect jdbc:mysql://icity0:3306/sqoop --username root --password root --table wordcount --input-fields-terminated-by '\t' --export-dir '/user/hive/warehouse/wordcount'


注:如果是由hive执行mapreduce操作汇总的结果,默认的分隔符是 '\001'.

你可能感兴趣的:(sqoop)