HDP3 集成 TEZ_UI

HDP3以上的版本移除了TEZ view，收集网上的资料，自行部署集成

tez_ui 页面部署

1. 准备一个tomcat

mkdir tez_ui
cd tez_ui
wget https://mirrors.tuna.tsinghua.edu.cn/apache/tomcat/tomcat-9/v9.0.37/bin/apache-tomcat-9.0.37.tar.gz
tar -zxvf apache-tomcat-9.0.37.tar.gz
cd apache-tomcat-9.0.37/webapps
mkdir back
mv * back

2. 修改tomcat默认端口为9999（官网推荐的tez_ui端口，具体配置略）

3. 根据你使用的tez版本，下载对应的版本的安装包，我这里使用的时0.9.1 。

wget https://mirrors.tuna.tsinghua.edu.cn/apache/tez/0.9.1/apache-tez-0.9.1-bin.tar.gz
tar -zxvf apache-tez-0.9.1-bin.tar.gz
mkdir tez_ui
cp apache-tez-0.9.1-bin/tez-ui-0.9.1.war tez_ui/
cd tez_ui
jar -xvf tez-ui-0.9.1.war

4. 修改config/configs.env中的内容，主要是timeline和rm的地址

ENV = {
  hosts: {
    /*
     * Timeline Server Address:
     * By default TEZ UI looks for timeline server at http://localhost:8188, uncomment and change
     * the following value for pointing to a different address.
     */
    timeline: "http://10.30.238.12:8188",

    /*
     * Resource Manager Address:
     * By default RM REST APIs are expected to be at http://localhost:8088, uncomment and change
     * the following value to point to a different address.
     */
    rm: "http://10.30.238.11:8088",

    /*
     * Resource Manager Web Proxy Address:
     * Optional - By default, value configured as RM host will be taken as proxy address
     * Use this configuration when RM web proxy is configured at a different address than RM.
     */
    //rmProxy: "http://localhost:8088",
  },

  /*
   * Time Zone in which dates are displayed in the UI:
   * If not set, local time zone will be used.
   * Refer http://momentjs.com/timezone/docs/ for valid entries.
   */
  //timeZone: "UTC",

  /*
   * yarnProtocol:
   * If specified, this protocol would be used to construct node manager log links.
   * Possible values: http, https
   * Default value: If not specified, protocol of hosts.rm will be used
   */
  //yarnProtocol: "",
};

5. 启动tomcat即可

cd ${TOMCAT_HOME}/bin
sh startup.sh

tez-site.xml 配置

tez.am.view-acls: *
yarn.timeline-service.enabled: true
tez.tez-ui.history-url.base http://your-host/tez-ui/
tez.am.tez-ui.history-url.template __HISTORY_URL_BASE__?viewPath=/#/tez-app/__APPLICATION_ID__
tez.history.logging.service.class org.apache.tez.dag.history.logging.ats.ATSV15HistoryLoggingService

hive-site.xml配置

hive.server2.enable.doAs: true
hive.exec.failure.hooks: org.apache.hadoop.hive.ql.hooks.ATSHook
hive.exec.post.hooks: org.apache.hadoop.hive.ql.hooks.ATSHook,org.apache.atlas.hive.hook.HiveHook
hive.exec.pre.hooks: org.apache.hadoop.hive.ql.hooks.ATSHook
hive_timeline_logging_enabled: true

yarn-site.xml配置

hadoop.http.cross-origin.allowed-origins: *
yarn.nodemanager.webapp.cross-origin.enabled: true
yarn.resourcemanager.webapp.cross-origin.enabled: true
yarn.timeline-service.http-cross-origin.enabled: true