编译 apache Oozie 4.3.0

文章开始前先吐槽下Oozie居然要自己编译。官方文档

1.环境准备

• Unix box

• Java JDK 1.7+（本文用的是1.8）

• maven (现在一般是3.3.9)

• apache Oozie 4.3.0

  官方文档有要求安装pig，其实没什么卵用。

2.MAVEN配置

maven安装过程这里就省略了。接下来配置maven mirror，可以配置成阿里云的。

     nexus
     * 
     http://maven.aliyun.com/nexus/content/groups/public/

3.编译

3.1.解压Oozie

tar -xzvf oozie-4.3.0.tar.gz

3.2.修改pom.xml

我的java版本是1.8的，Oozie默认的java版本是1.7

cd oozie-4.3.0
vi pom.xml
1.7      ->      1.8
1.7      ->      1.8 

3.3.准备缺少的dependency

${mvn_repo}代表本地maven仓库地址

3.3.1 pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar

mkdir -p ${mvn_repo}/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde

cd ${mvn_repo}/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde

wget http://repo.spring.io/plugins-release/org/pentaho/pentaho-aggdesigner-\
algorithm/5.1.5-jhyde/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar

3.3.2 doxia-module-twiki-1.0-alpha-9.2y.jar

mkdir -p ${mvn_repo}/org/apache/maven/doxia/doxia-module-twiki/1.0-alpha-9.2y
                         
cd ${mvn_repo}/org/apache/maven/doxia/doxia-module-twiki/1.0-alpha-9.2y
                         
wget https://repository.cloudera.com/content/repositories/releases/org/\
apache/maven/doxia/doxia-module-twiki/1.0-alpha-9.2y/doxia-module-twiki-1.0-alpha-9.2y.jar --no-check-certificate

3.3.3 doxia-core-1.0-alpha-9.2y.jar

mkdir -p ${mvn_repo}/org/apache/maven/doxia/doxia-module-twiki

cd ${mvn_repo}/org/apache/maven/doxia/doxia-module-twiki

wget https://repository.cloudera.com/content/repositories\
/releases/org/apache/maven/doxia/doxia-core/1.0-alpha-9.2y/doxia-core-1.0-alpha-9.2y.jar --no-check-certificate

3.4.编译Oozie

cd oozie-4.3.0
bin/mkdistro.sh  -DskipTests -Puber -Phadoop-2 \
-Dhadoop.version=2.7.2 \
-Dhadoop.auth.version=2.7.2 \
-Ddistcp.version=2.7.2 \
-Dsqoop.version=1.4.3 \
-Dhive.version=1.2.0 \
-Dhbase.version=0.94.27

hadoop version需要改成自己安装hadoop version，其他的也是，没安装也没关系。

经过漫长的等待。。。

[INFO] Apache Oozie Main .................................. SUCCESS [  0.601 s]
[INFO] Apache Oozie Hadoop Utils hadoop-2-4.3.0 ........... SUCCESS [  1.371 s]
[INFO] Apache Oozie Hadoop Distcp hadoop-2-4.3.0 .......... SUCCESS [  0.061 s]
[INFO] Apache Oozie Hadoop Auth hadoop-2-4.3.0 Test ....... SUCCESS [  0.191 s]
[INFO] Apache Oozie Hadoop Libs ........................... SUCCESS [  0.011 s]
[INFO] Apache Oozie Client ................................ SUCCESS [  8.851 s]
[INFO] Apache Oozie Share Lib Oozie ....................... SUCCESS [  1.710 s]
[INFO] Apache Oozie Share Lib HCatalog .................... SUCCESS [  1.460 s]
[INFO] Apache Oozie Share Lib Distcp ...................... SUCCESS [  0.443 s]
[INFO] Apache Oozie Core .................................. SUCCESS [ 29.701 s]
[INFO] Apache Oozie Share Lib Streaming ................... SUCCESS [  2.492 s]
[INFO] Apache Oozie Share Lib Pig ......................... SUCCESS [  2.425 s]
[INFO] Apache Oozie Share Lib Hive ........................ SUCCESS [  3.291 s]
[INFO] Apache Oozie Share Lib Hive 2 ...................... SUCCESS [  2.734 s]
[INFO] Apache Oozie Share Lib Sqoop ....................... SUCCESS [  1.063 s]
[INFO] Apache Oozie Examples .............................. SUCCESS [  2.499 s]
[INFO] Apache Oozie Share Lib Spark ....................... SUCCESS [  3.877 s]
[INFO] Apache Oozie Share Lib ............................. SUCCESS [ 12.781 s]
[INFO] Apache Oozie Docs .................................. SUCCESS [  5.929 s]
[INFO] Apache Oozie WebApp ................................ SUCCESS [ 23.219 s]
[INFO] Apache Oozie Tools ................................. SUCCESS [  1.896 s]
[INFO] Apache Oozie MiniOozie ............................. SUCCESS [  0.693 s]
[INFO] Apache Oozie Distro ................................ SUCCESS [ 17.542 s]
[INFO] Apache Oozie ZooKeeper Security Tests .............. SUCCESS [  1.894 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS

编译好的包在 oozie-4.3.0/distro/target/oozie-4.3.0-distro.tar.gz

3.5.可能出现的错误

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (default) on project\
 oozie-distro: An Ant BuildException has occured: java.net.ConnectException: 连接超时 (Connection timed out) -> [Help]

这里需要下载tomcat，可能因为公司的网有代理导致java连接超时，那就手动下载一下吧。。。

cd oozie-4.3.0/distro/downloads
wget http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.47/bin/apache-tomcat-6.0.47.tar.gz
mv apache-tomcat-6.0.47.tar.gz tomcat-6.0.47.tar.gz

需要重命名下文件名，然后修改oozie-distro.xml,删了下面三行代码，然后再编译一次吧。

vi oozie-4.3.0/distro/oozie-distro.xml