hive on tez 步骤

1.下载tez src

解压,修改pom.xml,将hadoop.version改为2.7.2,最好用非root用户编译

mvn clean package -DskipTests=true -Dmaven.javadoc.skip=true

2.编译成功后,在tez-dist/target目录下,能够发现如下文件

archive-tmp maven-archiver tez-0.8.4 tez-0.8.4-minimal tez-0.8.4-minimal.tar.gz tez-0.8.4.tar.gz tez-dist-0.8.4-tests.jar

3.将tez-0.8.4-minimal上传到hdfs上,本例中上传到/tez目录下

Hadoop fs –put tez-0.8.4-minimal.tar.gz /tez/

4, 将tez-0.8.4-minimal.tar.gz考到/usr/hive下,并解压到./tez下

taz –zxvf tez-0.8.4-minimal.tar.gz –C ./tez

拷贝tez的lib下或者主目录下jar包===》到hive主目录下lib中去

5.在客户端安装tez-0.8.4-minimal,并且在conf目录下建立tez-site.xml并正确配置

 
   tez.lib.uris
   ${fs.defaultFS}/user/tez/tez-0.8.5-minimal.tar.gz
 

  tez.use.cluster.hadoop-libs
  true

6.在hive的客户端配置环境变量 (可以配置到/etc/profile.d/Hadoop.sh

export TEZ_HOME=/usr/hive/tez

for jar in `ls ${TEZ_HOME}|grep jar`;do
  export HADOOP_CLASSPATH=${TEZ_HOME}/$jar:${HADOOP_CLASSPATH}
done
for jar in `ls ${TEZ_HOME}/lib/|grep jar`;do
 export HADOOP_CLASSPATH=${TEZ_HOME}/lib/$jar:${HADOOP_CLASSPATH}
done

7.在hive-site.xml中配置参数

 
   hive.user.install.directory
   /user/
   
     If hive (in tez mode only) cannot find a usable hive jar in "hive.jar.directory",
     it will upload the hive jar to "hive.user.install.directory/user.name"
     and use it to run queries.
   
 
 
   hive.execution.engine
   tez
   
     Expects one of [mr, tez, spark].
     Chooses execution engine. Options are: mr (Map reduce, default), tez,     spark.       While MR
   remains the default engine for historical reasons, it is itself a historical engine
   and is deprecated in Hive 2 line. It may be removed without further warning.
   
 

8.hive启动后执行

  set hive.execution.engine=tez;

 即可运行hive on tez任务。

补充:
 hive默认计算引擎为tez
 编辑/usr/hive/conf/hive-site.xml

你可能感兴趣的:(hive on tez 步骤)