mapreduce运行内存及JVM配置错误running beyond physical

---

title: Hive1.2.1安装笔记
date: 2016/8/22 0:47:36
tags: Hive
categories: 大数据

---

环境

ubuntu 16.04
4台机器的Hadoop2.7.2集群
Mysql安装在slave2中
hive安装在master上

下载Hive

$ wget http://mirrors.cnnic.cn/apache/hive/hive-1.2.1/apache-hive-1.2.1-bin.tar.gz
# 解压
$ tar -zxvf apache-hive-1.2.1-bin.tar.gz /home/ubuntu/cloud

配置Hive环境变量

$ sudo vim /etc/profile

#添加
export HIVE_HOME=/home/ubuntu/cloud/apache-hive-1.2.1-bin
export PATH=$PATH:$HIVE_HOME/bin

$source /etc/profile

在Mysql中创建Hive用户

mysql>CREATE USER 'hive' IDENTIFIED BY 'hive';
mysql>GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%' IDENTIFIED BY 'hive' WITH GRANT OPTION;
mysql>flush privileges;

创建Hive数据库

$ mysql -uhive -phive
mysql>create database hive;

配置Hive

进入Hive的conf目录,找到hive-default.xml.template，cp份为hive-site.xml

$ vim hive-site.xml
# 删除configuration标签里的所有内容 添加如下内容

    
        javax.jdo.option.ConnectionURL
        jdbc:mysql://slave2:3306/hive?createDatabaseIfNotExist=true
        JDBC connect string for a JDBC metastore    
       
     
        javax.jdo.option.ConnectionDriverName 
        com.mysql.jdbc.Driver 
        Driver class name for a JDBC metastore     
                   
 
     
        javax.jdo.option.ConnectionUserName
        hive
        username to use against metastore database
    
      
        javax.jdo.option.ConnectionPassword
        hive
        password to use against metastore database  
              

下载mysql-connector-java-5.1.32-bin.jar

这里用5.1.32版本测试不报错，5.1.38会报warn

#将连接jar包拷贝到Hive的lib目录
$ cp mysql-connector-java-5.1.32-bin.jar /home/ubuntu/cloud/apache-hive-1.2.1-bin/lib/

若要装hive客户端可在客户端节点设置

vim hive-site.xml

hive.metastore.uris
  thrift://master:9083
  Thrift uri for the remote metastore. Used by metastore client to connect to remote metastore.

 


  hive.metastore.warehouse.dir
  /user/hive/warehouse
  location of default database for the warehouse

Hive启动

要启动metastore服务

$ hive --service metastore &
$ jps
10288 RunJar  #多了一个进程
9365 NameNode
9670 SecondaryNameNode
11096 Jps
9944 NodeManager
9838 ResourceManager
9471 DataNode

启动hive命令行

ubuntu@master:~$ hive

Logging initialized using configuration in jar:file:/home/ubuntu/cloud/apache-hive-1.2.1-bin/lib/hive-common-1.2.1.jar!/hive-log4j.properties
hive> show tables;
OK
Time taken: 0.705 seconds

启动hiveserver2

hive --service hiveserver2 start &

问题解决

问题：创建表出先如下错误，删除表卡住

Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:For direct MetaStore DB connections, we don't support retries at the client level.

初始化
注意：初始化之前先删除hdfs上的metastore,否则会出错/user/hive/warehouse
在hive服务端输入以下命令

schematool -dbType mysql -initSchema