SQOOP运行缺少java-json jar报错
SQOOP运行缺少java-json jar报错
运行SQOOP遇到此类错误
[hadoop@xkhadoop shell]$ sh -x ods_init.sh ruozedata_supplier
+ '[' 1 '!=' 1 ']'
++ date '-d-1 day' +%Y-%m-%d
+ YESTERDAY=2020-01-04
++ date +%Y-%m-%d
+ TODAY=2020-01-05
+ YESTERDAYTIME='2019-12-22 00:00:00'
+ TODAYTIME='2019-12-23 00:00:00'
+ MySQL_DATABASENAME=erp
+ MySQL_TABLENAME=ruozedata_supplier
+ MySQL_URL=jdbc:mysql://xkhadoop:3306
+ MySQL_USER=xk
+ MySQL_PASSWORD=123456
+ HDFS_PATH=hdfs://xkhadoop:9000
+ HIVE_DATABASENAME=xk_erp
+ HIVE_ODS_TABLENAME=ods_ruozedata_supplier
+ HIVE_PATH=hdfs://xkhadoop:9000/user/hive/warehouse/xk_erp.db/ods_ruozedata_supplier
+ sqoop import --connect jdbc:mysql://xkhadoop:3306/erp --username xk --password 123456 --query 'select * from ruozedata_supplier where updatetime<'\''2019-12-23 00:00:00'\'' and $CONDITIONS' -m 1 --hive-import --delete-target-dir --target-dir hdfs://xkhadoop:9000/user/hive/warehouse/xk_erp.db/ods_ruozedata_supplier --hive-database xk_erp --hive-table ods_ruozedata_supplier --null-string '\\N' --null-non-string '\\N' --fields-terminated-by '\t' --lines-terminated-by '\n'
Warning: /home/hadoop/app/sqoop/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /home/hadoop/app/sqoop/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/hadoop/app/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /home/hadoop/app/sqoop/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
20/01/05 16:55:37 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.16.2
20/01/05 16:55:37 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
20/01/05 16:55:37 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
20/01/05 16:55:37 INFO tool.CodeGenTool: Beginning code generation
20/01/05 16:55:38 INFO manager.SqlManager: Executing SQL statement: select * from ruozedata_supplier where updatetime<'2019-12-23 00:00:00' and (1 = 0)
20/01/05 16:55:38 INFO manager.SqlManager: Executing SQL statement: select * from ruozedata_supplier where updatetime<'2019-12-23 00:00:00' and (1 = 0)
20/01/05 16:55:38 INFO manager.SqlManager: Executing SQL statement: select * from ruozedata_supplier where updatetime<'2019-12-23 00:00:00' and (1 = 0)
20/01/05 16:55:38 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop/app/hadoop
注: /tmp/sqoop-hadoop/compile/58eabab32721dde0e180deb03e6f0430/QueryResult.java使用或覆盖了已过时的 API。
注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
20/01/05 16:55:40 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/58eabab32721dde0e180deb03e6f0430/QueryResult.jar
20/01/05 16:55:40 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
20/01/05 16:55:42 INFO tool.ImportTool: Destination directory hdfs://xkhadoop:9000/user/hive/warehouse/xk_erp.db/ods_ruozedata_supplier is not present, hence not deleting.
20/01/05 16:55:42 INFO mapreduce.ImportJobBase: Beginning query import.
Exception in thread "main" java.lang.NoClassDefFoundError: org/json/JSONObject
at org.apache.sqoop.util.SqoopJsonUtil.getJsonStringforMap(SqoopJsonUtil.java:43)
at org.apache.sqoop.SqoopOptions.writeProperties(SqoopOptions.java:784)
at org.apache.sqoop.mapreduce.JobBase.putSqoopOptionsToConfiguration(JobBase.java:392)
at org.apache.sqoop.mapreduce.JobBase.createJob(JobBase.java:378)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:256)
at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:747)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:515)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: java.lang.ClassNotFoundException: org.json.JSONObject
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 14 more
+ exit根据错误定位导缺少java-json jar包
然后去下载这个jar
http://www.java2s.com/Code/Jar/j/Downloadjavajsonjar.htm
把jar copy导SQOOP_HOME/lib下面，然后重新运行SQOOP
[hadoop@xkhadoop lib]$ ll
总用量 31428
-rw-rw-r--. 1 hadoop hadoop 224277 6月 3 2019 ant-contrib-1.0b3.jar
-rw-rw-r--. 1 hadoop hadoop 36455 6月 3 2019 ant-eclipse-1.0-jvm1.2.jar
-rw-rw-r--. 1 hadoop hadoop 477209 6月 3 2019 avro-1.7.6-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop 181328 6月 3 2019 avro-mapred-1.7.6-cdh5.16.2-hadoop2.jar
-rw-rw-r--. 1 hadoop hadoop 58160 6月 3 2019 commons-codec-1.4.jar
-rw-rw-r--. 1 hadoop hadoop 425643 6月 3 2019 commons-compress-1.11.jar
-rw-rw-r--. 1 hadoop hadoop 109043 6月 3 2019 commons-io-1.4.jar
-rw-rw-r--. 1 hadoop hadoop 267634 6月 3 2019 commons-jexl-2.1.1.jar
-rw-rw-r--. 1 hadoop hadoop 434678 6月 3 2019 commons-lang3-3.4.jar
-rw-rw-r--. 1 hadoop hadoop 62050 6月 3 2019 commons-logging-1.1.3.jar
-rw-rw-r--. 1 hadoop hadoop 14582557 6月 3 2019 fastutil-6.3.jar
-rw-rw-r--. 1 hadoop hadoop 706710 6月 3 2019 hsqldb-1.8.0.10.jar
-rw-rw-r--. 1 hadoop hadoop 36519 6月 3 2019 jackson-annotations-2.3.1.jar
-rw-rw-r--. 1 hadoop hadoop 197986 6月 3 2019 jackson-core-2.3.1.jar
-rw-rw-r--. 1 hadoop hadoop 227535 6月 3 2019 jackson-core-asl-1.8.10.jar
-rw-rw-r--. 1 hadoop hadoop 917322 6月 3 2019 jackson-databind-2.3.1-cloudera.2.jar
-rw-rw-r--. 1 hadoop hadoop 672097 6月 3 2019 jackson-mapper-asl-1.8.10-cloudera.2.jar
-rw-r--r--. 1 hadoop hadoop 84697 10月 16 2013 java-json.jar
-rw-rw-r--. 1 hadoop hadoop 2201079 6月 3 2019 kite-data-core-1.0.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop 1808186 6月 3 2019 kite-data-hive-1.0.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop 1773706 6月 3 2019 kite-data-mapreduce-1.0.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop 1772115 6月 3 2019 kite-hadoop-compatibility-1.0.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop 872300 1月 5 16:37 mysql-connector-java-5.1.27.jar
-rw-rw-r--. 1 hadoop hadoop 19827 6月 3 2019 opencsv-2.3.jar
-rw-rw-r--. 1 hadoop hadoop 29555 6月 3 2019 paranamer-2.3.jar
-rw-rw-r--. 1 hadoop hadoop 106454 6月 3 2019 parquet-avro-1.5.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop 956036 6月 3 2019 parquet-column-1.5.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop 41085 6月 3 2019 parquet-common-1.5.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop 278926 6月 3 2019 parquet-encoding-1.5.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop 384616 6月 3 2019 parquet-format-2.1.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop 212649 6月 3 2019 parquet-hadoop-1.5.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop 931451 6月 3 2019 parquet-jackson-1.5.0-cdh5.16.2.jar
-rw-rw-r--. 1 hadoop hadoop 26084 6月 3 2019 slf4j-api-1.7.5.jar
-rw-rw-r--. 1 hadoop hadoop 995968 6月 3 2019 snappy-java-1.0.4.1.jar重新运行SQOOP job
[hadoop@xkhadoop lib]$ cd ../
[hadoop@xkhadoop sqoop]$ cd ~
[hadoop@xkhadoop ~]$ cd shell/
[hadoop@xkhadoop shell]$ sh -x ods_init.sh ruozedata_supplier发现后面运行成功无此类报错了