apache-hive-3.1.2安装和部署

 

hive的安装：

1、下载apache-hive-3.1.2-bin.tar.gz

                     hive安装包路径

2、解压并配置环境变量

   a、解压包并重命名为hive-3.1.2

     [root@hadoop-01 local]# tar -zxvf apache-hive-3.1.2-bin.tar.gz 
     [root@hadoop-01 local]# mv apache-hive-3.1.2-bin hive-3.1.2

b、配置环境变量vim /etc/profile 

   添加HIVE_HOME目录

      export HIVE_HOME=/usr/local/hive-3.1.2

     export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin

[root@hadoop-01 local]# vim /etc/profile
[root@hadoop-01 local]# source /etc/profile

3、修改hive配置

复制hive-3.1.2/conf目录下hive-default.xml.template 为hive-site.xml并修改

 a、修改数据库配置，这里使用的是mysql

 添加驱动包到hive-3.1.2的lib目录下；我使用的是mysql-connector-java-5.1.30.jar

  
    javax.jdo.option.ConnectionPassword
    123456
    数据库密码
  
  
    javax.jdo.option.ConnectionURL
    jdbc:mysql://192.168.15.50:3306/hive_metadata?createDatabaseIfNotExist=true
    
      JDBC connect string for a JDBC metastore.
      To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
      For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
    
  
  
    javax.jdo.option.ConnectionDriverName
    com.mysql.jdbc.Driver
    Driver class name for a JDBC metastore
  

    javax.jdo.option.ConnectionUserName
    root
    Username to use against metastore database
  

 注意： 创建数据源时要求字符集使用latin1；否则报异常“MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes”  创建hive的mysql数据库，如：“create database hive_metadata character set latin1;”

b、配置hive自动创建表，改模式可两种模式

第一种：直接修改datanucleus.schema.autoCreateAll 为true；其默认值为false

 
    datanucleus.schema.autoCreateAll
    true
    Auto creates necessary schema on a startup if one doesn't exist. Set this to false, after creating it once.To enable auto create also set hive.metastore.schema.verification=false. Auto creation is not recommended for production use cases, run schematool command instead.
  

 第二种：添加表的自动创建属性

  
         datanucleus.schema.autoCreateSchema
         true
    
    
         datanucleus.schema.autoCreateSchema
         true
    

    
        datanucleus.schema.autoCreateTables
        true
    

    
        datanucleus.schema.autoCreateColumns
        true
    
   
   datanucleus.fixedDatastore
   false

3、修改${system:java.io.tmpdir}的路径

hive.exec.local.scratchdir 修改成/tmp/hive;hive.downloaded.resources.修改成/tmp/hive/resources;hive.querylog.location修改成./tmp/hive/query/log

hive.server2.logging.operation.log.location修改成/tmp/hive/operation_logs

  
    hive.exec.local.scratchdir
    ${system:java.io.tmpdir}/${system:user.name}
    Local scratch space for Hive jobs
  
  
    hive.downloaded.resources.dir
    ${system:java.io.tmpdir}/${hive.session.id}_resources
    Temporary local directory for added resources in the remote file system.
  
  
    hive.querylog.location
    ${system:java.io.tmpdir}/${system:user.name}
    Location of Hive run time structured log file
  
 
    hive.server2.logging.operation.log.location
    ${system:java.io.tmpdir}/${system:user.name}/operation_logs
    Top level directory where operation logs are stored if logging functionality is enabled
  

经过修改后可得到

  
    hive.exec.local.scratchdir
    /tmp/hive
    Local scratch space for Hive jobs
  
  
    hive.downloaded.resources.dir
    /tmp/hive/resources
    Temporary local directory for added resources in the remote file system.
  
  
    hive.querylog.location
    /tmp/hive/query/log
    Location of Hive run time structured log file
  
 
    hive.server2.logging.operation.log.location
    /tmp/hive/operation_logs
    Top level directory where operation logs are stored if logging functionality is enabled
  

4、最后启动bin目录下hive

注意：如果show tables；异常“FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient”时，执行schematool -dbType mysql -initSchema；如

[root@hadoop02 bin]# ./hive
which: no hbase in (/usr/local/java/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/local/zookeeper-3.4.14/bin:/usr/local/hadoop/bin:/usr/local/hadoop/sbin:/usr/local/flume/bin:/usr/local/kafka/bin:/root/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 6e0e8815-5256-4995-8af3-a7a5818f43f8

Logging initialized using configuration in jar:file:/usr/local/hive-3.1.2/lib/hive-common-3.1.2.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive> show tables;
FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
hive> show databases;
FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
hive> exit
    > ;
[root@hadoop02 bin]# ls
beeline  ext  hive  hive-config.sh  hiveserver2  hplsql  init-hive-dfs.sh  metatool  schematool
[root@hadoop02 bin]# ./schematool -dbType mysql -initSchema