重新编译并安装Tez on hive（Tez 0.8.5,Hive 2.3.3)

author by Fei Joe (Thupdi 技术创新中心)

一，

maven3.3.9

protoc的版本是2.5.0

二，安装nodejs

yum -y install gcc make gcc-c++ openssl

wget http://nodejs.org/dist/v0.12.2/node-v0.12.2.tar.gz 

解压后./configure 

如果报错 没有 python: ImportError: No module named bz2

解决方法：

yum install bzip2-devel

然后需要重新编译一下python，之后再import bz2就不会提示错误了。

1. cd Python-2.7.10

2. ./configure --prefix=/usr/local/python2

3. make && make install

4. 

make && make install    最后查看是否成功node -v

三，安装npm

curl -L https://www.npmjs.com/install.sh | sh

如果执行node -v和npm -v能看到版本就说明安装成功了

 

如果不成功 请看：： https://blog.csdn.net/mawenwu1983/article/details/78142253

四，安装bower

npm -g install bower

 

安装后执行下面bower install命令如果提示了如下命令说明bower安装成功。中这个错误提示的是root用户不能执行bower install 命令

如果需要安装 需要加 --allow-root命令

进入tez_ui 修改pom.xml

                node_modules/bower/bin/bower
                install
                --allow-root 
                --remove-unnecessary-resolutions=false
             

五，安装git

yum install git

 

 mvn clean package  -Dhadoop.version=2.7.3  -DskipTests=true -Dmaven.javadoc.skip=true

 

如果tez_ui2报错 去pom.xml 注释掉

如下：

編譯成功后：

Now, in the target directory of tez-dist you can see two tar balls are created:

tez-0.8.5.tar.gz

tez-0.8.5-minimal.tar.gz

 

Next, create a new folder with the name tez in your desired location. We are creating this folder in the home directory and then moving the two tar balls into the folder.

Step 17: Create the directory using the command mkdir tez

 

hadoop fs -mkdir /tez

Step 17: Copy the tez-0.8.5.tar.gz tar ball into the directory using the below command:

hadoop fs -put tez-0.8.5.tar.gz /tez/

You can check whether the file has copied to HDFS or not by using the below command:

hadoop fs -ls /tez/

Step 18: Now, within the tez folder, which is in your local file system, create two folders with name conf and tez using the below commands:

mkdir conf
mkdir tez

Step 19: Extract the contents of the file tez-0.8.1-alpha-minimal.tar.gz to the newly created tez folder.

Step 20: Now, create a file tez-site.xml in your newly created conf directory and add the below configurations in the tez-site.xml file:

tez.lib.uris
hdfs://localhost:8020/tez/tez-0.8.5.tar.gz

修改hive/conf/hive-site.xml 

Now close and save the file. Your Tez is ready! You can now integrate it with Hadoop by adding the following properties in your hadoop-env.sh file of Hadoop.

export TEZ_CONF_DIR=/mnt/moduled/conf/
export TEZ_JARS=/mnt/moduled/tez/tez/
export HADOOP_CLASSPATH=${TEZ_CONF_DIR}:${TEZ_JARS}/*:${TEZ_JARS}/lib/*:${HADOOP_CLASSPATH}:${MAPREDUCE_LIBS}