oozie安装部署在hadoop2.50.0-CDH5.3.6

最近在这几天在搞伪分布式安装hue,在安装oozie的时候总是启动不了web界面,花了很长时间搞定,下面是安装oozie的步骤总结一下:

安装环境:

  • centOS6.4
  • Java JDK 1.7
  • Maven
  • hadoop-2.5.0-cdh5.3.6
  • oozie-4.0.0-cdh5.3.6
  • ext-2.2.zip

安装前注意事项

Download ExtJS library (it must be version 2.2)。

  ExtJS版本必须是2.2版本
  推荐使用Oozie这个Linux用户。这个暂时不需要管

解压oozie

配置hadoop的core-site.xml

我们的Hadoop平台使用的是用户xusy,这里配置Hadoop代理用户也使用该用户,部署Oozie的主机名为xuxudede.com。修改Hadoop的配置文件core-site.xml,增加如下配置内容:


    hadoop.proxyuser.root.hosts
    cloud61


    hadoop.proxyuser.root.groups
    root

解压oozie-hadooplibs-4.0.0-cdh5.3.6.tar.gz

Expand the Oozie hadooplibs tar.gz in the same location
Oozie distribution tar.gz was expanded
特别注意:解压的目录要与oozie解压的路径相同,解压完成后会在oozie根目录下生成一个hadooplibs文件!!!
如图:
oozie安装部署在hadoop2.50.0-CDH5.3.6_第1张图片

创建libext目录

Create a libext/ directory in the directory where Oozie was expanded.
在oozie的根目录中创建libext目录。
如图:
oozie安装部署在hadoop2.50.0-CDH5.3.6_第2张图片

将hadoop相关的jar库文件拷贝到libext下面

拷贝hadooplibs/hadooplib-2.5.0-cdh5.3.6.oozie-4.0.0-cdh5.3.6下的jar包到libext/目录下

拷贝ext-2.2.zip到libext/目录下

如图:
oozie安装部署在hadoop2.50.0-CDH5.3.6_第3张图片

创建war包并加到tomcat里

$ bin/oozie-setup.sh prepare-war

重启hadoop进成,直到脱离安全模式

先stop namenode、datanode、resourcemanager、nodemanager、historyserver,然后再分别start。这里不再讲述!

把oozie根目录下oozie-sharelib-4.0.0-cdh5.3.6-yarn.tar.gz包解压并且放到hdfs上

运行命令如下:hdfs://xuxudede.com:9000是HDFS主机名加端口号!

$bin/oozie-setup.sh sharelib create -fs hdfs://xuxudede.com:9000 -locallib oozie-sharelib-4.0.0-cdh5.3.6-yarn.tar.gz



1. Oozie install


   /etc/profile:
   OOZIE_HOME=/home/hadoop/oozie-3.2.0-cdh4.1.2


   $OOZIE_HOME//oozie-server/bin/catalina.sh:
   JAVA_HOME=/usr/jdk6/jdk1.6.0_32
   CATALINA_HOME=/home/cup/oozie-3.3.0-cdh4.2.1/oozie-server

   $OOZIE_HOME/bin/oozie-setup.sh:
   $oozie-setup.sh -extjs /home/hadoop/ext-2.2.zip -hadoop 0.20.200 $HADOOP_HOME

   $oozie-setup.sh -extjs /home/hadoop/ext-2.2.zip -hadoop 2.0 $HADOOP_HOME

2. $OOZIE_HOME/bin/oozie-run.sh 启动oozie


5. oozie启动报找不到org/apache/hadoop/utils/ReflectionUtils类
   将/home/hadoop/oozie-3.2.0-cdh4.1.2/libtools/*.jar copy to /home/hadoop/oozie-3.2.0-cdh4.1.2/oozie-server/webapps/oozie/WEB-INF/lib下

6. oozie启动报
REASON: org.apache.oozie.service.ServiceException: E0103: Could not load service classes, Schema 'SA' does not exist {SELECT t0.bean_type, t0.conf, t0.console_url, t0.cred, t0.data, t0.error_code, t0.error_message, t0.external_child_ids, t0.external_id, t0.external_status, t0.name, t0.retries, t0.stats, t0.tracker_uri, t0.transition, t0.type, t0.user_retry_count, t0.user_retry_interval, t0.user_retry_max, t0.end_time, t0.execution_path, t0.last_check_time, t0.log_token, t0.pending, t0.pending_age, t0.signal_value, t0.sla_xml, t0.start_time, t0.status, t0.wf_id FROM WF_ACTIONS t0 WHERE t0.bean_type = ? AND t0.id = ?} [code=30000, state=42Y07]

7. $OOZIE_HOME/bin/ooziedb.sh create -sqlfile oozie.sql -run

Validate DB Connection
DONE
Check DB schema does not exist
DONE
Check OOZIE_SYS table does not exist
DONE
Create SQL schema
DONE
Create OOZIE_SYS table
DONE

Oozie DB has been created for Oozie version '3.2.0-cdh4.1.2'


The SQL commands have been written to: oozie.sql

sql脚本保存到$OOZIE_HOME/bin/oozie.sql文件中.

8. oozie-site.xml:
   
   
        oozie.service.ProxyUserService.proxyuser.hue.hosts
        *
   

   
        oozie.service.ProxyUserService.proxyuser.hue.groups
        *
   

   
        oozie.service.ProxyUserService.proxyuser.cup.hosts
        *
   

   
        oozie.service.ProxyUserService.proxyuser.cup.groups
        *
   



8.
Error occurred during initialization of VM
Incompatible minimum and maximum heap sizes specified

oozie-env.sh:
export CATALINA_OPTS="$CATALINA_OPTS -Xms2g -Xmx4g"


8. $OOZIE_HOME/bin/oozie-run.sh 启动oozie   
   $OOZIE_HOME/bin/oozie-run.sh & 后台启动oozie

   最新:
   $oozied.sh run

   $ jps
   28945 Bootstrap

9. $OOZIE_HOME/bin/oozie admin -oozie http://192.168.101.122:11000/oozie -status
   System mode: NORMAL 则表示已经成功
   http://192.168.101.122:11000/oozie就能看到Oozie的管理界面



你可能感兴趣的:(cloudera)