Hive安装中元数据初始化遇到的坑

背景：

在已经部署了hadoop的linux中，安装hive，并元数据用MySQL数据库。

hive-default.xml.template  是坑中之最。

1.需要修改文件名。注意！这个文件名需要改。

cp hive-default.xml.template  hive-site.xml

cp beeline-log4j2.properties.template  beeline-log4j2.properties 
cp hive-env.sh.template  hive-env.sh
cp hive-exec-log4j2.properties.template hive-exec-log4j2.properties 
cp hive-log4j2.properties.template  hive-log4j2.properties
cp llap-cli-log4j2.properties.template llap-cli-log4j2.properties
cp llap-daemon-log4j2.properties.template llap-daemon-log4j2.properties

2.文件里面的配置参数。网上说的一些配置参数，在这个文件里都有默认的，而且文件里参数极多。我的方法是将hive-site.xml这个文件下载下来，然后通过关键字查找，来修改参数配置。再上传回去。注意！这个配置参数，最好是直接修改默认值。网上说，copy到文件上也可以。经过尝试，我将这段参数，粘贴到段首，会报错，初始化失败。可能是由于配置文件执行顺序是从上到下的，上面配置的mysql，到下面默认值会进行覆盖，又回到了默认的derby数据库。大家可以自行尝试放在文末，看是否会报错。我是直接把默认配置改了，没有出现问题。

  javax.jdo.option.ConnectionURL
  jdbc:mysql://Hadoop:3306/hivedb?createDatabaseIfNotExist=true
  JDBC connect string for a JDBC metastore


  javax.jdo.option.ConnectionDriverName
  com.mysql.jdbc.Driver
  Driver class name for a JDBC metastore


  javax.jdo.option.ConnectionUserName
  root
  username to use against metastore database


  javax.jdo.option.ConnectionPassword
  root
  password to use against metastore database


  hive.metastore.warehouse.dir
  /user/hive/warehouse
  hive default warehouse, if nessecory, change it

3.临时目录配置也坑

在进入hive是，报了如下的错误：

[hive@Hadoop ~]$ hive
which: no hbase in (/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hive/soft/hadoop-2.8.5/bin:/home/hive/soft/hadoop-2.8.5/sbin:/home/hive/soft/jdk1.8.0_91/bin:/home/hive/soft/apache-hive-3.1.1-bin/bin:/home/hive/bin:/home/hive/soft/hadoop-2.8.5/bin:/home/hive/soft/hadoop-2.8.5/sbin:/home/hive/soft/jdk1.8.0_91/bin:/home/hive/soft/apache-hive-2.3.5-bin/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hive/soft/apache-hive-2.3.5-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hive/soft/hadoop-2.8.5/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in file:/home/hive/soft/apache-hive-2.3.5-bin/conf/hive-log4j2.properties Async: true
Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
        at org.apache.hadoop.fs.Path.initialize(Path.java:254)
        at org.apache.hadoop.fs.Path.(Path.java:212)
        at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:659)
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:582)
        at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:549)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:750)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
        at java.net.URI.checkPath(URI.java:1823)
        at java.net.URI.(URI.java:745)
        at org.apache.hadoop.fs.Path.initialize(Path.java:251)
        ... 12 more

需要在hive-site.xml文件里，修改如下配置：

    Hive.exec.local.scratchdir
    /home/hive/soft/apache-hive-2.3.5-bin/tmp//${user.name}
    Local scratch space for Hive jobs
  

  
    hive.downloaded.resources.dir
    /home/hive/soft/apache-hive-2.3.5-bin/tmp/${hive.session.id}_resources
    Temporary local directory for added resources in the remote file system.
  


    hive.server2.logging.operation.log.location
    /home/hive/soft/apache-hive-2.3.5-bin/tmp/operation_logs
    Top level directory where operation logs are stored if logging functionality is enabled
  

 

mysql-connector-java-5.1.7-bin.jar驱动包也是个坑。

这驱动包，需要放在/hive/lib/目录下。如果没有这个jar包，hive会连接不到数据库。会报错。

[hive@Hadoop conf]$ schematool -dbType mysql -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hive/soft/apache-hive-2.3.5-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hive/soft/hadoop-2.8.5/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:        jdbc:mysql://xxx:3306/hivedb?createDatabaseIfNotExist=true
Metastore Connection Driver :    com.mysql.jdbc.Driver
Metastore connection User:       root
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver
Underlying cause: java.lang.ClassNotFoundException : com.mysql.jdbc.Driver
Use --verbose for detailed stacktrace.
*** schemaTool failed ***

初始化成功

[hive@Hadoop conf]$ schematool -dbType mysql -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hive/soft/apache-hive-2.3.5-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hive/soft/hadoop-2.8.5/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:        jdbc:mysql://xxx:3306/hivedb?createDatabaseIfNotExist=true
Metastore Connection Driver :    com.mysql.jdbc.Driver
Metastore connection User:       root
Starting metastore schema initialization to 2.3.0
Initialization script hive-schema-2.3.0.mysql.sql
Initialization script completed
schemaTool completed