hive数据库初始化问题
版本信息
hadoop 3.2.2
hbase 1.7.1
mysql 5.7.35
hive 2.3.9
报错信息
- java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument
[root@centos76-1 conf]# schematool -dbType mysql -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
at org.apache.hadoop.mapred.JobConf.setJar(JobConf.java:536)
at org.apache.hadoop.mapred.JobConf.setJarByClass(JobConf.java:554)
at org.apache.hadoop.mapred.JobConf.(JobConf.java:448)
at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:4051)
at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:4014)
at org.apache.hive.beeline.HiveSchemaTool.(HiveSchemaTool.java:82)
at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1117)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
错误原因：
系统找不到这个类所在的jar包或者jar包的版本不一样系统不知道使用哪个。hive启动报错的原因是后者。
解决方法：
com.google.common.base.Preconditions.checkArgument这个类所在的jar包为：guava.jar，在hadoop和hive中版本不一致导致。
删除hive中低版本jar包，将hadoop中高版本的复制到hive的lib中。
[root@centos76-1 lib]# ls $HADOOP_HOME/share/hadoop/common/lib/guava*
/usr/local/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar
[root@centos76-1 lib]# ls $HIVE_HOME/lib/guava*
/usr/local/hive/lib/guava-14.0.1.jar
[root@centos76-1 lib]# rm -f $HIVE_HOME/lib/guava*
[root@centos76-1 lib]# cp /usr/local/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar $HIVE_HOME/lib/
- org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
Underlying cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException : Communications link failure
[root@centos76-1 lib]# schematool -dbType mysql -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:mysql://centos76-1:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver : com.mysql.jdbc.Driver
Metastore connection User: root
Fri Dec 24 12:36:45 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+,
and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not
L the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and
truststore for server certificate verification.
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
Underlying cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException : Communications link failure
The last packet successfully received from the server was 1,050 milliseconds ago. The last packet sent successfully to the server was 1,021
onds ago.
SQL Error code: 0
Use --verbose for detailed stacktrace.
*** schemaTool failed ***
详细查看报错信息发现有SSL报错:javax.net.ssl.SSLHandshakeException: No appropriate protocol
[root@centos76-1 conf]# schematool -dbType mysql -initSchema --verbose
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:mysql://192.168.220.140:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver : com.mysql.jdbc.Driver
Metastore connection User: hive
Sat Dec 25 05:31:18 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
Underlying cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException : Communications link failure
The last packet successfully received from the server was 1,313 milliseconds ago. The last packet sent successfully to the server was 1,251 milliseconds ago.
SQL Error code: 0
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
at org.apache.hive.beeline.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:77)
at org.apache.hive.beeline.HiveSchemaTool.getConnectionToMetastore(HiveSchemaTool.java:144)
at org.apache.hive.beeline.HiveSchemaTool.testConnectionToMetastore(HiveSchemaTool.java:473)
at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:577)
at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:563)
at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 1,313 milliseconds ago. The last packet sent successfully to the server was 1,251 milliseconds ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:990)
at com.mysql.jdbc.ExportControlled.transformSocketToSSLSocket(ExportControlled.java:203)
at com.mysql.jdbc.MysqlIO.negotiateSSLConnection(MysqlIO.java:4901)
at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1659)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1226)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2188)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2219)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2014)
at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:776)
at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:386)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.hive.beeline.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:73)
... 11 more
Caused by: javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
at sun.security.ssl.HandshakeContext.(HandshakeContext.java:171)
at sun.security.ssl.ClientHandshakeContext.(ClientHandshakeContext.java:106)
at sun.security.ssl.TransportContext.kickstart(TransportContext.java:238)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:394)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:373)
at com.mysql.jdbc.ExportControlled.transformSocketToSSLSocket(ExportControlled.java:188)
... 29 more
*** schemaTool failed ***
解决方法：
在hive-site.xml中把ConnectionURL修改为如下所示
javax.jdo.option.ConnectionURL
jdbc:mysql://192.168.220.140:3306/hive?createDatabaseIfNotExist=true&useSSL=false
- com.ctc.wstx.exc.WstxParsingException: String ‘–’ not allowed in comment (missing ‘>’?)
[root@centos76-1 lib]# schematool -dbType mysql -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Exception in thread "main" java.lang.RuntimeException: com.ctc.wstx.exc.WstxParsingException: String '--' not allowed in comment (missing '>'?)
at [row,col,system-id]: [18,7,"file:/usr/local/hive/conf/hive-site.xml"]
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3040)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2989)
at org.apache.hadoop.conf.Configuration.loadProps(Configuration.java:2864)
at org.apache.hadoop.conf.Configuration.addResourceObject(Configuration.java:1012)
at org.apache.hadoop.conf.Configuration.addResource(Configuration.java:917)
at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:4061)
at org.apache.hadoop.hive.conf.HiveConf.(HiveConf.java:4014)
at org.apache.hive.beeline.HiveSchemaTool.(HiveSchemaTool.java:82)
at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1117)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
Caused by: com.ctc.wstx.exc.WstxParsingException: String '--' not allowed in comment (missing '>'?)
at [row,col,system-id]: [18,7,"file:/usr/local/hive/conf/hive-site.xml"]
at com.ctc.wstx.sr.StreamScanner.constructWfcException(StreamScanner.java:621)
at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:491)
at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:475)
at com.ctc.wstx.sr.BasicStreamReader.skipCommentOrCData(BasicStreamReader.java:3592)
at com.ctc.wstx.sr.BasicStreamReader.skipToken(BasicStreamReader.java:3443)
at com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2074)
at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1181)
at org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3336)
at org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3130)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3023)
... 14 more
错误原因：
在hive-default.xml.template模版里前面有一段注释的结尾–>，使用它作hive-site.xml时误删了–>
解决方法：
补回来
[root@centos76-1 conf]# head -17 hive-default.xml.template
- org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver
[root@centos76-1 conf]# schematool -dbType mysql -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:mysql://centos76-1:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver : com.mysql.jdbc.Driver
Metastore connection User: root
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver
Underlying cause: java.lang.ClassNotFoundException : com.mysql.jdbc.Driver
Use --verbose for detailed stacktrace.
*** schemaTool failed ***
错误原因：jdbc driver下错了。
解决方法：
应该下载下面图里的zip包，从里面解压出.jar文件上传到 $HIVE_HOME/lib/ 下