centos6.5 hive配置

因为项目需要，首先先安装mysql和hive连接。

hive安装教程链接：

https://www.cnblogs.com/dxxblog/p/8193967.html

https://blog.csdn.net/qq_36508766/article/details/81318996

一. hive数据库初始化报错

报错：Error: Syntax error: Encountered "" at line 1, column 64

metastore connection URL 以及metastore connection driver 都是默认的derby不是mysql，所以是跟mysql的连接出现问题。

[sunsi@master bin]$ ./schematool -dbType mysql -initSchema -verbose
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hadoop/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:	 jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :	 org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User:	 APP
Starting metastore schema initialization to 2.3.0
Initialization script hive-schema-2.3.0.mysql.sql
Connecting to jdbc:derby:;databaseName=metastore_db;create=true
Connected to: Apache Derby (version 10.10.2.0 - (1582446))
Driver: Apache Derby Embedded JDBC Driver (version 10.10.2.0 - (1582446))
Transaction isolation: TRANSACTION_READ_COMMITTED
0: jdbc:derby:> !autocommit on
Autocommit status: true
0: jdbc:derby:> /*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */
Error: Syntax error: Encountered "" at line 1, column 64. (state=42X01,code=30000)
Closing: 0: jdbc:derby:;databaseName=metastore_db;create=true
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
Underlying cause: java.io.IOException : Schema script failed, errorcode 2
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
	at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:590)
	at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:563)
	at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1145)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:226)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:141)
Caused by: java.io.IOException: Schema script failed, errorcode 2
	at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:980)
	at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:959)
	at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:586)
	... 8 more
*** schemaTool failed ***

解决方法：在hive安装目录/conf中的hive-site.xml配置文件中直接复制粘贴以下代码段，该文件中这四项都有默认值，插入的代码段没有生效被默认值覆盖，一个个修改即可。

    
        javax.jdo.option.ConnectionUserName用户名（这4项是新添加的，记住删除配置文件原有的哦！）
        root
    
    
        javax.jdo.option.ConnectionPassword密码
        123456
    
   
        javax.jdo.option.ConnectionURLmysql数据库hive如果不存在就创建一个
        jdbc:mysql://192.168.139.95:3306/hive?createDatabaseIfNotExist=true
    
    
        javax.jdo.option.ConnectionDriverNamemysql驱动程序
        com.mysql.jdbc.Driver
    

报错：org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver

[sunsi@master bin]$ ./schematool -dbType mysql -initSchema -verbose
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hadoop/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:	 jdbc:mysql://192.168.139.95:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver :	 com.mysql.jdbc.Driver
Metastore connection User:	 root
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver
Underlying cause: java.lang.ClassNotFoundException : com.mysql.jdbc.Driver
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver
	at org.apache.hive.beeline.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:79)
	at org.apache.hive.beeline.HiveSchemaTool.getConnectionToMetastore(HiveSchemaTool.java:144)
	at org.apache.hive.beeline.HiveSchemaTool.testConnectionToMetastore(HiveSchemaTool.java:473)
	at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:577)
	at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:563)
	at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1145)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:226)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:141)
Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at org.apache.hive.beeline.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:70)
	... 11 more
*** schemaTool failed ***

解决方案：将MySQL的驱动添加到hive安装目录的lib下  mysql-connector-java-5.1.47-bin.jar

下载地址：https://dev.mysql.com/downloads/connector/j/5.1.html

报错：org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.Underlying cause: java.sql.SQLException : Access denied for user 'root'@'master' (using password: YES)

解决方法：参考https://blog.csdn.net/peterchan88/article/details/78341852

用root登录mysql，mysql -u root -p

mysql> select user,host from mysql.user where user='root';
+------+-----------+
| user | host      |
+------+-----------+
| root | 127.0.0.1 |
| root | localhost |
| root | master    |
+------+-----------+
3 rows in set (0.02 sec)

'root'@'master'用户权限不足，利用root用户给该用户分配权限 

mysql> grant all on *.* to 'root'@'master' identified by '密码';
Query OK, 0 rows affected (0.00 sec)

测试'root'@'master.hadoop'是否能登录

[sunsi@master lib]$ mysql -u root -h 'master' -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 15
Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

二. 启动hive时报错

报错：Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

[sunsi@master ~]$ hive
which: no hbase in (/usr/lib/jdk1.8.0_201/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/hadoop/hadoop-2.7.7/bin:/usr/hadoop/hadoop-2.7.7/sbin:/usr/lib/apache-ant-1.9.14/bin:/usr/hive/bin:/home/sunsi/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hadoop/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in jar:file:/usr/hive/lib/hive-common-2.3.5.jar!/hive-log4j2.properties Async: true
Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
	at org.apache.hadoop.fs.Path.initialize(Path.java:205)
	at org.apache.hadoop.fs.Path.(Path.java:171)
	at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:659)
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:582)
	at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:549)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:750)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:226)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:141)
Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
	at java.net.URI.checkPath(URI.java:1823)
	at java.net.URI.(URI.java:745)
	at org.apache.hadoop.fs.Path.initialize(Path.java:202)
	... 12 more

解决方法：在HIVE_HOME目录下创建文件夹iotmp，修改hive-site.xml文件

  hive.exec.local.scratchdir
    $HIVE_HOME/iotmp
    Local scratch space for Hive jobs


   hive.querylog.location
    $HIVE_HOME/iotmp
    Location of Hive run time structured log file
 
 
   hive.downloaded.resources.dir
    $HIVE_HOME/iotmp
    Temporary local directory for added resources in the remote file system.
 

三.使用hive时报错

在hive中创建数据库create database hive_1 报错：Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStore

解决方法：重新开机后mysql没有开启服务，使用sudo service mysqld start开启服务

hive> show databases;
FAILED: IllegalArgumentException java.net.URISyntaxException: Relative path in absolute URI: file:./$HIVE_HOME/iotmp/813c5d3d-6b47-4834-9b88-1f0db45a945e/hive_2019-07-03_14-52-57_732_5214921909510155854-1

解决方法：修改hive-site.xml，把其中的$HIVE_HOME全部改成/usr/hive，重新进入hive，可能会报权限错误，sudo chmod -R 777 /usr/hive