[sqoop1 error] java.lang.ClassNotFoundException org.apache.hadoop.hive.conf.Hiv

阅读更多

 

 

问题：

[zkkafka@yanfabu2-37 ~]$ sqoop import \
> --connect jdbc:mysql://10.156.50.36:3306/mqh \
> --username root \
> --password root \
> --table device \
> --fields-terminated-by '\t' \
> --delete-target-dir \
> --num-mappers 1 \
> --hive-import \
> --hive-database test \
> --hive-table hive_bbs_product_snappy;
Warning: /home/zkkafka/sqoop/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/zkkafka/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
19/05/29 12:00:27 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
19/05/29 12:00:27 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/05/29 12:00:27 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
19/05/29 12:00:27 INFO tool.CodeGenTool: Beginning code generation
Wed May 29 12:00:27 CST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
19/05/29 12:00:27 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `device` AS t LIMIT 1
19/05/29 12:00:27 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `device` AS t LIMIT 1
19/05/29 12:00:27 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/zkkafka/hadoop
注: /tmp/sqoop-zkkafka/compile/0edae8bed32dad40d1fe8ca424e30387/device.java使用或覆盖了已过时的 API。
注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
19/05/29 12:00:31 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-zkkafka/compile/0edae8bed32dad40d1fe8ca424e30387/device.jar
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/zkkafka/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/zkkafka/hbase/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
19/05/29 12:00:32 INFO tool.ImportTool: Destination directory device deleted.
19/05/29 12:00:32 WARN manager.MySQLManager: It looks like you are importing from mysql.
19/05/29 12:00:32 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
19/05/29 12:00:32 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
19/05/29 12:00:32 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
19/05/29 12:00:32 INFO mapreduce.ImportJobBase: Beginning import of device
19/05/29 12:00:32 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
19/05/29 12:00:32 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
Wed May 29 12:00:39 CST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
19/05/29 12:00:39 INFO db.DBInputFormat: Using read commited transaction isolation
19/05/29 12:00:40 INFO mapreduce.JobSubmitter: number of splits:1
19/05/29 12:00:40 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1558676658010_0008
19/05/29 12:00:41 INFO impl.YarnClientImpl: Submitted application application_1558676658010_0008
19/05/29 12:00:41 INFO mapreduce.Job: The url to track the job: http://master1:8088/proxy/application_1558676658010_0008/
19/05/29 12:00:41 INFO mapreduce.Job: Running job: job_1558676658010_0008
19/05/29 12:00:49 INFO mapreduce.Job: Job job_1558676658010_0008 running in uber mode : false
19/05/29 12:00:49 INFO mapreduce.Job:  map 0% reduce 0%
19/05/29 12:00:57 INFO mapreduce.Job:  map 100% reduce 0%
19/05/29 12:00:59 INFO mapreduce.Job: Job job_1558676658010_0008 completed successfully
19/05/29 12:01:00 INFO mapreduce.Job: Counters: 30
	File System Counters
		FILE: Number of bytes read=0
		FILE: Number of bytes written=128603
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=87
		HDFS: Number of bytes written=156
		HDFS: Number of read operations=4
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=2
	Job Counters 
		Launched map tasks=1
		Other local map tasks=1
		Total time spent by all maps in occupied slots (ms)=5465
		Total time spent by all reduces in occupied slots (ms)=0
		Total time spent by all map tasks (ms)=5465
		Total vcore-milliseconds taken by all map tasks=5465
		Total megabyte-milliseconds taken by all map tasks=5596160
	Map-Reduce Framework
		Map input records=6
		Map output records=6
		Input split bytes=87
		Spilled Records=0
		Failed Shuffles=0
		Merged Map outputs=0
		GC time elapsed (ms)=103
		CPU time spent (ms)=1380
		Physical memory (bytes) snapshot=168935424
		Virtual memory (bytes) snapshot=2104311808
		Total committed heap usage (bytes)=93323264
	File Input Format Counters 
		Bytes Read=0
	File Output Format Counters 
		Bytes Written=156
19/05/29 12:01:00 INFO mapreduce.ImportJobBase: Transferred 156 bytes in 27.7324 seconds (5.6252 bytes/sec)
19/05/29 12:01:00 INFO mapreduce.ImportJobBase: Retrieved 6 records.
19/05/29 12:01:00 INFO mapreduce.ImportJobBase: Publishing Hive/Hcat import job data to Listeners for table device
Wed May 29 12:01:00 CST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
19/05/29 12:01:00 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `device` AS t LIMIT 1
19/05/29 12:01:00 WARN hive.TableDefWriter: Column register_time had to be cast to a less precise type in Hive
19/05/29 12:01:00 INFO hive.HiveImport: Loading uploaded data into Hive
19/05/29 12:01:00 ERROR hive.HiveConfig: Could not load org.apache.hadoop.hive.conf.HiveConf. Make sure HIVE_CONF_DIR is set correctly.
19/05/29 12:01:00 ERROR tool.ImportTool: Import failed: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf
	at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:50)
	at org.apache.sqoop.hive.HiveImport.getHiveArgs(HiveImport.java:392)
	at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:379)
	at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:337)
	at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:241)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:537)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:44)
	... 12 more

 

解决方法：

cp /home/zkkafka/hive/lib/hive-exec-**.jar  /home/zkkafka/sqoop/lib/

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

捐助开发者 

在兴趣的驱动下,写一个免费的东西，有欣喜，也还有汗水，希望你喜欢我的作品，同时也能支持一下。 当然，有钱捧个钱场（支持支付宝和微信 以及扣扣群），没钱捧个人场，谢谢各位。

 

个人主页：http://knight-black-bob.iteye.com/

 
 
 谢谢您的赞助，我会做的更好！