build from source code
1. download from http://tez.apache.org/install.html
if you want to get the lattest codes through this command
#git clone https://git-wip-us.apache.org/repos/asf/tez.git
#tar xvf apache-tez-0.5.1-src.tar.gz
#cd apache-tez-0.5.1-src
#mvn package -Dhadoop.version=2.3.0 -P\!hadoop24 -DskipTests
error:
DAGClientTimelineImpl.java:[38,33] package com.sun.jersey.api.client does not exist
solution: modify tez-api/pom.xml
<dependencies> <dependency> <groupId>com.sun.jersey</groupId> <artifactId>jersey-client</artifactId> <version>1.9</version> </dependency> ......
rerun
#mvn clean package -Dhadoop.version=2.3.0 -P\!hadoop24 -DskipTests
all build jars in tez-dist/target/ dir.
-------------------------------------------
Depoly tez to hadoop clusters
1. make TEZ_HOME
#mkdir -p /home/inok/tez
#cp tez-dist/target/tez-0.5.1-minimal.tar.gz /home/inok/tez/
#cp tez-dist/target/tez-0.5.1.tar.gz /home/inok/tez/
#cd /home/inok/tez
#mkdir conf
#mkdir jars
#cd conf
#vi tez-site.xml
<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>tez.lib.uris</name> <value>${fs.defaultFS}/apps/tez-0.5.1/tez-0.5.1.tar.gz</value> </property> <property> <name>tez.use.cluster.hadoop-libs</name> <value>false</value> </property> </configuration>
#cd ../
#tar xf tez-0.5.1-minimal.tar.gz -C jars/
distribute tez dir to all cluster nodes /home/inok/
2. upload tez-0.5.1.tar.gz to hdfs
#hdfs dfs -mkdir -p /apps/tez-0.5.1
#hdfs dfs -put tez-0.5.1.tar.gz /apps/tez-0.5.1/
3. configure hadoop
# modify hadoop-2.3.0/etc/hadoop/mapred-site.xml
<property>
<name>mapreduce.framework.name</name>
<value>yarn-tez</value>
</property>
# modify hadoop-2.3.0/etc/hadoop/hadoop-env.sh
export TEZ_CONF_DIR=/home/inok/tez/conf
export TEZ_JARS=/home/inok/tez/jars
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$TEZ_CONF_DIR:$TEZ_JARS/*:$TEZ_JARS/lib/*
update mapred-site.xml and hadoop-env.sh in all hadoop cluster nodes
4. restart hadoop clusters
#stop-all.sh
#start-all.sh
5. test tez
#hdfs dfs -mkdir test
# vi words
good morning good afternoon thanks my boy
#hdfs dfs -put words test/
#hadoop jar hadoop-2.3.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.3.0.jar wordcount test/words test/output
IF you see the following reds words, congratuations, tez works.
14/11/13 09:58:58 INFO client.RMProxy: Connecting to ResourceManager at /192.168.0.131:2015
14/11/13 09:58:58 INFO client.TezClient: Submitting DAG application with id: application_1415843888183_0002
14/11/13 09:58:58 INFO client.TezClient: Tez system stage directory hdfs://192.168.0.131:2014/mr-history/am/staging/inok/.staging/job_1415843888183_0002/.tez/application_1415843888183_0002 doesn't exist and is created
14/11/13 09:58:58 INFO client.TezClient: Submitting DAG to YARN, applicationId=application_1415843888183_0002, dagName=word count
14/11/13 09:58:58 INFO impl.YarnClientImpl: Submitted application application_1415843888183_0002
14/11/13 09:58:58 INFO client.TezClient: The url to track the Tez AM: http://inok-c0:2019/proxy/application_1415843888183_0002/
14/11/13 09:58:58 INFO client.RMProxy: Connecting to ResourceManager at /192.168.0.131:2015
14/11/13 09:58:58 INFO client.RMProxy: Connecting to ResourceManager at /192.168.0.131:2015
-----------------
When I submit a common mapreduce job in hue through oozie, there is an error
Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses
#oozie job --oozie http://localhost:11000/oozie --config examples/apps/java-main/job.properties -run
job: 0000003-141113143744303-oozie-inok-W
http://qnalist.com/questions/5187496/failing-to-run-hive-action-using-mapreduce-framework-name-yarn-tez
<property>
<name>mapreduce.framework.name</name>
<value>yarn-tez</value>
</property>
but in mapred-default.xml in hadoop-mapreduce-client-core-2.3.0.jar
<property>
<name>mapreduce.framework.name</name>
<value>local</value>
<description>The runtime framework for executing MapReduce jobs.
Can be one of local, classic or yarn.
</description>
</property>
----
try these mothods
1. cp tez-site.xml to etc/hadoop
2. cp all tez relavant jars to oozie share lib 's oozie dir and update to hdfs
3. check tez's example
References
http://blog.woopi.org/wordpress/?p=96
http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.2/bk_installing_manually_book/content/rpm-chap-tez.html