Azkaban安装及分布式部署（multiple-executor）

  参考文章：https://blog.csdn.net/weixin_35852328/article/details/79327996

 官网：https://azkaban.readthedocs.io/en/latest/

　　Azkaban是由Linkedin开源的一个批量工作流任务调度器。用于在一个工作流内以一个特定的顺序运行一组工作和流程。Azkaban定义了一种KV文件格式来建立任务之间的依赖关系，并提供一个易于使用的web用户界面维护和跟踪你的工作流。
它有如下功能特点：Web用户界面，方便上传工作流，方便设置任务之间的关系，调度工作流，认证/授权(权限的工作)，能够杀死并重新启动工作流，模块化和可插拔的插件机制，项目工作区，工作流和任务的日志记录和审计。azkaban工作流调度器及相关工具对比。

1.准备资源

 

注：因公司主机无法连外网，所以先用本地虚拟机执行下载编译后，将相关文件打包，再上传到公司服务器进行安装

1.1下载

如果没有git可以（yum -y git）,建议把yum源设置为阿里源，否则各种资源找不到

git clone -b 3.12.0 https://github.com/azkaban/azkaban.git

1.2编译

进入azkaban，执行编译,此过程异常痛苦，可以多执行几次（20次后才build success）注：使用gradle进行安装，若没有gradle会自行安装

cd azkaban
./gradlew clean
./gradlew distTar

编译完成后，结果在/root/azkaban/azkaban-exec-server/build/distributions下

[root@localhost distributions]# pwd
/root/azkaban/azkaban-exec-server/build/distributions
[root@localhost distributions]# ls
azkaban-exec-server-3.12.0.tar.gz
[root@localhost distributions]#  

包说明

azkaban-common : 常用工具类。

azkaban-db : 对应的sql脚本

azkaban-hadoop-secutity-plugin : hadoop 有关kerberos插件

azkaban-solo-server: web和executor 一起的项目。

azkaban-web/executor-server:azkaban的 web和executor的server信息

azkaban-spi: azkaban存储接口以及exception类

1.3打包上传

打包所有文件

[root@localhost distributions]# cd /root
[root@localhost ~]# ls
anaconda-ks.cfg  ap  AutoConfig.sh  azkaban  pullK8sImages.sh
[root@localhost ~]# tar -cvf azkaban.tar azkaban

再将azkaban.tar上传至公司服务器

2.安装

root@hadoop01:/iot>mkdir azkaban-3.12.0
root@hadoop01:/iot>cd azkaban-3.12.0/
root@hadoop01:/iot/azkaban-3.12.0>ls
root@hadoop01:/iot/azkaban-3.12.0>cp /iot/azkaban/azkaban-exec-server/build/distributions/azkaban-exec-server-3.12.0.tar.gz  .
root@hadoop01:/iot/azkaban-3.12.0>cp /iot/azkaban/azkaban-web-server/build/distributions/azkaban-web-server-3.12.0.tar.gz  .
root@hadoop01:/iot/azkaban-3.12.0>cp /iot/azkaban/azkaban-sql/build/distributions/azkaban-sql-3.12.0.tar.gz .

2.1解压

解压并重命名

root@hadoop01:/iot/azkaban-3.12.0>tar -xvf azkaban-exec-server-3.12.0.tar.gz
root@hadoop01:/iot/azkaban-3.12.0>tar -xvf azkaban-sql-3.12.0.tar.gz
root@hadoop01:/iot/azkaban-3.12.0>tar -xvf azkaban-web-server-3.12.0.tar.gz
root@hadoop01:/iot/azkaban-3.12.0>mv azkaban-exec-server-3.12.0 azkaban-exec-server
root@hadoop01:/iot/azkaban-3.12.0>mv azkaban-sql-3.12.0 azkaban-sql
root@hadoop01:/iot/azkaban-3.12.0>mv azkaban-web-server-3.12.0 azkaban-web-server
root@hadoop01:/iot/azkaban-3.12.0>ll
总用量 31336
drwxr-xr-x 4 root root     4096 4月  25 10:17 azkaban-exec-server
-rw------- 1 root root 14084197 4月  25 12:07 azkaban-exec-server-3.12.0.tar.gz
drwxr-xr-x 2 root root     4096 4月  25 10:54 azkaban-sql
-rw------- 1 root root     2046 4月  25 12:09 azkaban-sql-3.12.0.tar.gz
drwxr-xr-x 5 root root     4096 4月  25 10:54 azkaban-web-server
-rw------- 1 root root 17985060 4月  25 12:08 azkaban-web-server-3.12.0.tar.gz
root@hadoop01:/iot/azkaban-3.12.0>

2.2 建表

>mysql -h10.10.10.10 -P3306 -uroot -p

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
7 rows in set (0.00 sec)

mysql> create database azkaban;
Query OK, 1 row affected (0.01 sec)


mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| azkaban            |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
8 rows in set (0.00 sec)

mysql> use azkaban;
Database changed
mysql> show tables;
Empty set (0.00 sec)
mysql> source /iot/azkaban-3.12.0/azkaban-sql/create-all-sql-3.12.0.sql;
mysql> show tables;
+------------------------+
| Tables_in_azkaban      |
+------------------------+
| active_executing_flows |
| active_sla             |
| execution_flows        |
| execution_jobs         |
| execution_logs         |
| executor_events        |
| executors              |
| project_events         |
| project_files          |
| project_flows          |
| project_permissions    |
| project_properties     |
| project_versions       |
| projects               |
| properties             |
| schedules              |
| triggers               |
+------------------------+
17 rows in set (0.00 sec)

2.3配置keystore

root@hadoop01:/iot>find -name keytool
./old-bak/jdk1.8.0_162/bin/keytool
./old-bak/jdk1.8.0_162/jre/bin/keytool
root@hadoop01:/iot>keytool -keystore keystore -alias jetty -genkey -keyalg RSA
输入密钥库口令:  
密钥库口令太短 - 至少必须为 6 个字符
输入密钥库口令:  
再次输入新口令: 
您的名字与姓氏是什么?
  [Unknown]:  pupu
您的组织单位名称是什么?
  [Unknown]:  pupu
您的组织名称是什么?
  [Unknown]:  pupu
您所在的城市或区域名称是什么?
  [Unknown]:  beijing
您所在的省/市/自治区名称是什么?
  [Unknown]:  beijing                                    
该单位的双字母国家/地区代码是什么?
  [Unknown]:  CN
CN=pupu, OU=pupu, O=pupu, L=beijing, ST=beijing, C=CN是否正确?
  [否]:  y

输入  的密钥口令
        (如果和密钥库口令相同, 按回车):  

Warning:
JKS 密钥库使用专用格式。建议使用 "keytool -importkeystore -srckeystore keystore -destkeystore keystore -deststoretype pkcs12" 迁移到行业标准格式 PKCS12。
root@hadoop01:/iot>cp keystore /iot/azkaban-3.12.0/azkaban-web-server/bin

完成上述工作后,将在当前目录生成 keystore 证书文件,将keystore 考贝到 azkaban web服务器根目录中的bin目录下.如:cp keystore /iot/azkaban-3.12.0/azkaban-web-server/bin

 

2.4修改配置

配置web-server

# 因为web-server 和 exec-server都没有默认配置文件我们需要复制solo的配置文件

修改azkaban.properties文件 

root@hadoop01:/iot/azkaban-3.12.0>cp /iot/azkaban/azkaban-solo-server/build/distributions/azkaban-solo-server-3.12.0.tar.gz .
root@hadoop01:/iot/azkaban-3.12.0>ls
azkaban-exec-server  azkaban-exec-server-3.12.0.tar.gz  azkaban-solo-server-3.12.0.tar.gz  azkaban-sql  azkaban-sql-3.12.0.tar.gz  azkaban-web-server  azkaban-web-server-3.12.0.tar.gz
root@hadoop01:/iot/azkaban-3.12.0>tar -xzvf azkaban-solo-server-3.12.0.tar.gz
root@hadoop01:/iot/azkaban-3.12.0>mv azkaban-solo-server-3.12.0 azkaban-solo-server
root@hadoop01:/iot/azkaban-3.12.0>cp -r /iot/azkaban-3.12.0/azkaban-solo-server/conf azkaban-web-server
root@hadoop01:/iot/azkaban-3.12.0>cp /iot/azkaban/azkaban-web-server/src/test/resources/log4j.properties azkaban-web-server/conf/
root@hadoop01:/iot/azkaban-3.12.0>cp -r azkaban-solo-server/plugins/ azkaban-web-server

 

root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/conf>pwd
/iot/azkaban-3.12.0/azkaban-web-server/conf
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/conf>cat azkaban.properties 
# Azkaban Personalization Settings
azkaban.name=Test
azkaban.label=My Local Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
#default.timezone.id=America/Los_Angeles
default.timezone.id=Asia/Shanghai

# Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=conf/azkaban-users.xml

# Loader for projects
executor.global.properties=conf/global.properties
azkaban.project.dir=projects

#database.type=h2
#h2.path=./h2
#h2.create.tables=true
database.type=mysql
mysql.port=3306
mysql.host=your host IP
mysql.database=azkaban
mysql.user=root
mysql.password=your password
mysql.numconnections=100


# Velocity dev mode
velocity.dev.mode=false

# Azkaban Jetty server properties.
jetty.use.ssl=false
jetty.maxThreads=25
jetty.port=8081

# Azkaban Executor settings
executor.port=12321

# mail settings
mail.sender=
mail.host=
job.failure.email=
job.success.email=

lockdown.create.projects=false

cache.directory=cache

# JMX stats
jetty.connector.stats=true
executor.connector.stats=true

# Azkaban plugin settings
azkaban.jobtype.plugin.dir=plugins/jobtypes
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/conf>

root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/plugins/jobtypes>pwd
/iot/azkaban-3.12.0/azkaban-web-server/plugins/jobtypes
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/plugins/jobtypes>cat commonprivate.properties
# set execute-as-user
azkaban.native.lib=false
execute.as.user=false

2.5 web-server启动验证及问题修正 

2.5.1启动web-serrver并验证

root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>sh /iot/azkaban-3.12.0/azkaban-solo-server/bin/azkaban-solo-shutdown.sh 
cat: /iot/azkaban-3.12.0/azkaban-solo-server/bin/../currentpid: 没有那个文件或目录
killing AzkabanSingleServer
kill: 用法:kill [-s 信号声明 | -n 信号编号 | -信号声明] 进程号 | 任务声明 ... 或 kill -l [信号声明]
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>sh start-web.sh 
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>start-web.sh:行5: logs/webServerLog_2019-04-25+16:52:39.out: 没有那个文件或目录

root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin/mkdir logs
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>ls
azkaban-web-shutdown.sh azkaban-web-start.sh keystore logs start-web.sh
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>cat start-web.sh
#!/bin/bash

base_dir=$(dirname $0)/..

#bin/azkaban-web-start.sh $base_dir >logs/webServerLog_`date +%F+%T`.out 2>&1 &
nohup sh azkaban-web-start.sh $base_dir >logs/webServerLog_`date +%F+%T`.out 2>&1 &
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>cd logs

root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin/logs>ll

总用量 12
-rw------- 1 root root 73 4月 25 17:15 webServerLog_2019-04-25+17:15:15.out
-rw------- 1 root root 5040 4月 25 17:18 webServerLog_2019-04-25+17:18:53.out
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin/logs>cat webServerLog_2019-04-25+17:18:53.out
Error: HADOOP_HOME is not set. Hadoop job types will not run properly.
./..
.:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7.2.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-web-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7.0.jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/commons-validator-1.3.1.jar:./../lib/data-1.15.7.jar:./../lib/data-transform-1.15.7.jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2.2.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/javax.inject-1.jar:./../lib/javax.servlet-api-3.0.1.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-20070829.jar:./../lib/li-jersey-uri-1.15.7.jar:./../lib/log4j-1.2.16.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mina-core-1.1.7.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/netty-3.2.3.Final.jar:./../lib/oro-2.0.8.jar:./../lib/parseq-1.3.6.jar:./../lib/pegasus-common-1.15.7.jar:./../lib/quartz-2.2.1.jar:./../lib/r2-1.15.7.jar:./../lib/restli-common-1.15.7.jar:./../lib/restli-server-1.15.7.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.7.jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-0.jar:./../lib/struts-core-1.3.8.jar:./../lib/struts-taglib-1.3.8.jar:./../lib/struts-tiles-1.3.8.jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar
2019/04/25 17:18:53.533 +0800 INFO [AzkabanWebServer] Starting Jetty Azkaban Web Server...
2019/04/25 17:18:53.547 +0800 INFO [AzkabanServer] Loading azkaban settings file from ./../conf
2019/04/25 17:18:53.548 +0800 INFO [AzkabanServer] Loading azkaban properties file
2019/04/25 17:18:53.549 +0800 INFO [AzkabanWebServer] Setting up connector with stats on: true
2019/04/25 17:18:53.569 +0800 ERROR [StdOutErrRedirect] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
2019/04/25 17:18:53.569 +0800 ERROR [StdOutErrRedirect] SLF4J: Defaulting to no-operation (NOP) logger implementation
2019/04/25 17:18:53.569 +0800 ERROR [StdOutErrRedirect] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
2019/04/25 17:18:53.653 +0800 INFO [AzkabanWebServer] Loading user manager class azkaban.user.XmlUserManager
2019/04/25 17:18:53.653 +0800 ERROR [AzkabanWebServer] Could not instantiate UserManager azkaban.user.XmlUserManager
2019/04/25 17:18:53.653 +0800 ERROR [StdOutErrRedirect] Exception in thread "main"
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] at azkaban.webapp.AzkabanWebServer.loadUserManager(AzkabanWebServer.java:264)
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] at azkaban.webapp.AzkabanWebServer.(AzkabanWebServer.java:198)
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] at azkaban.webapp.AzkabanWebServer.main(AzkabanWebServer.java:762)
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] Caused by: java.lang.reflect.InvocationTargetException
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2019/04/25 17:18:53.655 +0800 ERROR [StdOutErrRedirect] at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
2019/04/25 17:18:53.655 +0800 ERROR [StdOutErrRedirect] at azkaban.webapp.AzkabanWebServer.loadUserManager(AzkabanWebServer.java:260)
2019/04/25 17:18:53.655 +0800 ERROR [StdOutErrRedirect] ... 2 more
2019/04/25 17:18:53.655 +0800 ERROR [StdOutErrRedirect] Caused by: java.lang.IllegalArgumentException: User xml file conf/azkaban-users.xml doesn't exist.
2019/04/25 17:18:53.655 +0800 ERROR [StdOutErrRedirect] at azkaban.user.XmlUserManager.parseXMLFile(XmlUserManager.java:90)
2019/04/25 17:18:53.655 +0800 ERROR [StdOutErrRedirect] at azkaban.user.XmlUserManager.(XmlUserManager.java:84)
2019/04/25 17:18:53.655 +0800 ERROR [StdOutErrRedirect] ... 7 more

 

2.5.2 报错：Failed to load class "org.slf4j.impl.StaticLoggerBinder"

解决上面问题，看起来是缺少slf4j包

比较azkaban-web-server/lib 和azkaban-solo-server/lib下slf4j相关的包，发现少的包应该是slf4j-log4j12-1.7.7.jar

root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server>ls lib
activation-1.1.jar             commons-codec-1.6.jar          commons-jexl-2.1.1.jar       dom4j-1.1.jar                 javax.servlet-api-3.0.1.jar  mail-1.4.5.jar                   pegasus-common-1.15.7.jar     
antlr-2.7.2.jar                commons-collections-3.2.2.jar  commons-lang-2.6.jar         guava-13.0.1.jar              jetty-6.1.26.jar             metrics-core-3.1.0.jar           quartz-2.2.1.jar              snappy-0.3.jar
azkaban-common-3.12.0.jar      commons-compress-1.2.jar       commons-logging-1.1.3.jar    httpclient-4.3.1.jar          jetty-util-6.1.26.jar        metrics-jvm-3.1.0.jar            r2-1.15.7.jar                 sslext-1.2-0.jar
azkaban-web-server-3.12.0.jar  commons-dbcp-1.4.jar           commons-math3-3.0.jar        httpcore-4.3.jar              joda-time-2.0.jar            mina-core-1.1.7.jar              restli-common-1.15.7.jar      struts-core-1.3.8.jar
c3p0-0.9.1.1.jar               commons-dbutils-1.5.jar        commons-pool-1.5.4.jar       jackson-core-2.2.2.jar        jopt-simple-4.3.jar          mysql-connector-java-5.1.28.jar  restli-server-1.15.7.jar      struts-taglib-1.3.8.jar
cglib-nodep-2.2.jar            commons-digester-1.8.jar       commons-validator-1.3.1.jar  jackson-core-asl-1.9.5.jar    json-20070829.jar            netty-3.2.3.Final.jar            servlet-api-2.5-20081211.jar  struts-tiles-1.3.8.jar
commons-beanutils-1.7.0.jar    commons-fileupload-1.2.1.jar   data-1.15.7.jar              jackson-mapper-asl-1.9.5.jar  li-jersey-uri-1.15.7.jar     oro-2.0.8.jar                    servlet-api-2.5.jar           velocity-1.7.jar
commons-chain-1.1.jar          commons-io-2.4.jar             data-transform-1.15.7.jar    javax.inject-1.jar            log4j-1.2.16.jar             parseq-1.3.6.jar                 slf4j-api-1.7.7.jar           velocity-tools-2.0.jar
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server>ls ../azkaban-solo-server/lib
activation-1.1.jar              commons-collections-3.2.2.jar  commons-pool-1.5.4.jar       jackson-core-asl-1.9.5.jar    kafka-clients-0.10.0.0.jar         oro-2.0.8.jar                 snappy-0.3.jar
antlr-2.7.2.jar                 commons-compress-1.2.jar       commons-validator-1.3.1.jar  jackson-mapper-asl-1.9.5.jar  kafka-log4j-appender-0.10.0.0.jar  parseq-1.3.6.jar              snappy-java-1.1.2.4.jar
azkaban-common-3.12.0.jar       commons-dbcp-1.4.jar           data-1.15.7.jar              javax.inject-1.jar            li-jersey-uri-1.15.7.jar           pegasus-common-1.15.7.jar     sslext-1.2-0.jar
azkaban-exec-server-3.12.0.jar  commons-dbutils-1.5.jar        data-transform-1.15.7.jar    javax.servlet-api-3.0.1.jar   log4j-1.2.17.jar                   quartz-2.2.1.jar              struts-core-1.3.8.jar
azkaban-solo-server-3.12.0.jar  commons-digester-1.8.jar       dom4j-1.1.jar                jetty-6.1.26.jar              lz4-1.3.0.jar                      r2-1.15.7.jar                 struts-taglib-1.3.8.jar
azkaban-web-server-3.12.0.jar   commons-fileupload-1.2.1.jar   guava-13.0.1.jar             jetty-util-6.1.26.jar         mail-1.4.5.jar                     restli-common-1.15.7.jar      struts-tiles-1.3.8.jar
c3p0-0.9.1.1.jar                commons-io-2.4.jar             h2-1.4.193.jar               joda-time-2.0.jar             metrics-core-3.1.0.jar             restli-server-1.15.7.jar      velocity-1.7.jar
cglib-nodep-2.2.jar             commons-jexl-2.1.1.jar         hamcrest-core-1.1.jar        jopt-simple-4.3.jar           metrics-jvm-3.1.0.jar              servlet-api-2.5-20081211.jar  velocity-tools-2.0.jar
commons-beanutils-1.7.0.jar     commons-lang-2.6.jar           httpclient-4.3.1.jar         json-20070829.jar             mina-core-1.1.7.jar                servlet-api-2.5.jar
commons-chain-1.1.jar           commons-logging-1.1.3.jar      httpcore-4.3.jar             json-simple-1.1.1.jar         mysql-connector-java-5.1.28.jar    slf4j-api-1.7.21.jar
commons-codec-1.6.jar           commons-math3-3.0.jar          jackson-core-2.2.2.jar       junit-4.10.jar                netty-3.2.3.Final.jar              slf4j-log4j12-1.7.21.jar
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server>

然后到https://mvnrepository.com/artifact/org.slf4j/slf4j-log4j12/1.7.7，上将下面代码copy到idea中任一项目的pom.xml，后会自动下载到本地仓库，找到后将其传入上面azkaban-web-server/lib下，此问题解决

<dependency>
    <groupId>org.slf4jgroupId>
    <artifactId>slf4j-log4j12artifactId>
    <version>1.7.7version>
    <scope>testscope>
dependency>

 

后再次重启azkaban-web-server/bin下sh start-web.sh

2.5.3 报错：Could not instantiate UserManager azkaban.user.XmlUserManager

检查/iot/azkaban-3.12.0/azkaban-web-server/conf/下有此文件azkaban-users.xml ，可将azkaban.properties中相关配置设置为绝对路径

oot@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin/logs>cat webServerLog_2019-04-25+18:12:00.out
Error: HADOOP_HOME is not set. Hadoop job types will not run properly.
./..
.:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7.2.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-web-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7.0.jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/commons-validator-1.3.1.jar:./../lib/data-1.15.7.jar:./../lib/data-transform-1.15.7.jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2.2.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/javax.inject-1.jar:./../lib/javax.servlet-api-3.0.1.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-20070829.jar:./../lib/li-jersey-uri-1.15.7.jar:./../lib/log4j-1.2.16.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mina-core-1.1.7.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/netty-3.2.3.Final.jar:./../lib/oro-2.0.8.jar:./../lib/parseq-1.3.6.jar:./../lib/pegasus-common-1.15.7.jar:./../lib/quartz-2.2.1.jar:./../lib/r2-1.15.7.jar:./../lib/restli-common-1.15.7.jar:./../lib/restli-server-1.15.7.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.7.jar:./../lib/slf4j-log4j12-1.7.7.jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-0.jar:./../lib/struts-core-1.3.8.jar:./../lib/struts-taglib-1.3.8.jar:./../lib/struts-tiles-1.3.8.jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar
2019/04/25 18:12:00.678 +0800 INFO [AzkabanWebServer] Starting Jetty Azkaban Web Server...
2019/04/25 18:12:00.692 +0800 INFO [AzkabanServer] Loading azkaban settings file from ./../conf
2019/04/25 18:12:00.692 +0800 INFO [AzkabanServer] Loading azkaban properties file
2019/04/25 18:12:00.694 +0800 INFO [AzkabanWebServer] Setting up connector with stats on: true
2019/04/25 18:12:00.715 +0800 INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2019/04/25 18:12:00.762 +0800 INFO [AzkabanWebServer] Loading user manager class azkaban.user.XmlUserManager
2019/04/25 18:12:00.762 +0800 ERROR [AzkabanWebServer] Could not instantiate UserManager azkaban.user.XmlUserManager
2019/04/25 18:12:00.763 +0800 ERROR [StdOutErrRedirect] Exception in thread "main" 
2019/04/25 18:12:00.763 +0800 ERROR [StdOutErrRedirect] java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
2019/04/25 18:12:00.763 +0800 ERROR [StdOutErrRedirect]         at azkaban.webapp.AzkabanWebServer.loadUserManager(AzkabanWebServer.java:264)
2019/04/25 18:12:00.763 +0800 ERROR [StdOutErrRedirect]         at azkaban.webapp.AzkabanWebServer.(AzkabanWebServer.java:198)
2019/04/25 18:12:00.763 +0800 ERROR [StdOutErrRedirect]         at azkaban.webapp.AzkabanWebServer.main(AzkabanWebServer.java:762)
2019/04/25 18:12:00.764 +0800 ERROR [StdOutErrRedirect] Caused by: java.lang.reflect.InvocationTargetException
2019/04/25 18:12:00.764 +0800 ERROR [StdOutErrRedirect]         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2019/04/25 18:12:00.764 +0800 ERROR [StdOutErrRedirect]         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2019/04/25 18:12:00.764 +0800 ERROR [StdOutErrRedirect]         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2019/04/25 18:12:00.764 +0800 ERROR [StdOutErrRedirect]         at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
2019/04/25 18:12:00.764 +0800 ERROR [StdOutErrRedirect]         at azkaban.webapp.AzkabanWebServer.loadUserManager(AzkabanWebServer.java:260)
2019/04/25 18:12:00.764 +0800 ERROR [StdOutErrRedirect]         ... 2 more
2019/04/25 18:12:00.764 +0800 ERROR [StdOutErrRedirect] Caused by: java.lang.IllegalArgumentException: User xml file conf/azkaban-users.xml doesn't exist.
2019/04/25 18:12:00.764 +0800 ERROR [StdOutErrRedirect]         at azkaban.user.XmlUserManager.parseXMLFile(XmlUserManager.java:90)
2019/04/25 18:12:00.764 +0800 ERROR [StdOutErrRedirect]         at azkaban.user.XmlUserManager.(XmlUserManager.java:84)
2019/04/25 18:12:00.764 +0800 ERROR [StdOutErrRedirect]         ... 7 more
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin/logs>

更改过后的配置文件

root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/conf>ls
azkaban.properties  azkaban-users.xml  global.properties  log4j.properties
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/conf>pwd
/iot/azkaban-3.12.0/azkaban-web-server/conf
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/conf>cat azkaban.properties 
# Azkaban Personalization Settings
azkaban.name=Test
azkaban.label=My Local Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
#default.timezone.id=America/Los_Angeles
default.timezone.id=Asia/Shanghai

# Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=/iot/azkaban-3.12.0/azkaban-web-server/conf/azkaban-users.xml

# Loader for projects
executor.global.properties=/iot/azkaban-3.12.0/azkaban-web-server/conf/global.properties
azkaban.project.dir=projects

#database.type=h2
#h2.path=./h2
#h2.create.tables=true
database.type=mysql
mysql.port=3306
mysql.host=youknow
mysql.database=azkaban
mysql.user=root
mysql.password=pwd
mysql.numconnections=100


# Velocity dev mode
velocity.dev.mode=false

# Azkaban Jetty server properties.
jetty.use.ssl=false
jetty.maxThreads=25
jetty.port=8081

# Azkaban Executor settings
executor.port=12321

# mail settings
mail.sender=
mail.host=
job.failure.email=
job.success.email=

lockdown.create.projects=false

cache.directory=cache

# JMX stats
jetty.connector.stats=true
executor.connector.stats=true

# Azkaban plugin settings
azkaban.jobtype.plugin.dir=/iot/azkaban-3.12.0/azkaban-web-server/plugins/jobtypes
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/conf>

再次重启，进程已经启动起来了

root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>sh start-web.sh 
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>ps -ef|grep az
root      20995      1 14 18:22 pts/2    00:00:01 java -Xmx4G -Dlog4j.configuration=file:./../conf/log4j.properties -server -Dcom.sun.management.jmxremote -Djava.io.tmpdir=/tmp -Dexecutorport=12321 -Dserverpath=/iot/azkaban-3.12.0/azkaban-web-server/bin -Dlog4j.log.dir=./../logs -cp .:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7.2.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-web-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7.0.jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/commons-validator-1.3.1.jar:./../lib/data-1.15.7.jar:./../lib/data-transform-1.15.7.jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2.2.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/javax.inject-1.jar:./../lib/javax.servlet-api-3.0.1.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-20070829.jar:./../lib/li-jersey-uri-1.15.7.jar:./../lib/log4j-1.2.16.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mina-core-1.1.7.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/netty-3.2.3.Final.jar:./../lib/oro-2.0.8.jar:./../lib/parseq-1.3.6.jar:./../lib/pegasus-common-1.15.7.jar:./../lib/quartz-2.2.1.jar:./../lib/r2-1.15.7.jar:./../lib/restli-common-1.15.7.jar:./../lib/restli-server-1.15.7.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.7.jar:./../lib/slf4j-log4j12-1.7.7.jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-0.jar:./../lib/struts-core-1.3.8.jar:./../lib/struts-taglib-1.3.8.jar:./../lib/struts-tiles-1.3.8.jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar azkaban.webapp.AzkabanWebServer -conf ./../conf ./..
root      21031  18067  0 18:23 pts/2    00:00:00 grep --color=auto az
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>

2.5.4 报错：java.sql.SQLException: Unknown column 'ex.executor_id'

root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>cd logs
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin/logs>ls
webServerLog_2019-04-25+17:15:15.out  webServerLog_2019-04-25+17:18:53.out  webServerLog_2019-04-25+17:24:54.out  webServerLog_2019-04-25+18:03:10.out  webServerLog_2019-04-25+18:12:00.out  webServerLog_2019-04-25+18:22:59.out
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin/logs>ll
总用量 44
-rw------- 1 root root   73 4月  25 17:15 webServerLog_2019-04-25+17:15:15.out
-rw------- 1 root root 5040 4月  25 17:18 webServerLog_2019-04-25+17:18:53.out
-rw------- 1 root root 5040 4月  25 17:24 webServerLog_2019-04-25+17:24:54.out
-rw------- 1 root root 4827 4月  25 18:03 webServerLog_2019-04-25+18:03:10.out
-rw------- 1 root root 4827 4月  25 18:12 webServerLog_2019-04-25+18:12:00.out
-rw------- 1 root root 5092 4月  25 18:23 webServerLog_2019-04-25+18:22:59.out
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin/logs>cat webServerLog_2019-04-25+18:22:59.out
Error: HADOOP_HOME is not set. Hadoop job types will not run properly.
./..
.:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7.2.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-web-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7.0.jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/commons-validator-1.3.1.jar:./../lib/data-1.15.7.jar:./../lib/data-transform-1.15.7.jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2.2.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/javax.inject-1.jar:./../lib/javax.servlet-api-3.0.1.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-20070829.jar:./../lib/li-jersey-uri-1.15.7.jar:./../lib/log4j-1.2.16.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mina-core-1.1.7.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/netty-3.2.3.Final.jar:./../lib/oro-2.0.8.jar:./../lib/parseq-1.3.6.jar:./../lib/pegasus-common-1.15.7.jar:./../lib/quartz-2.2.1.jar:./../lib/r2-1.15.7.jar:./../lib/restli-common-1.15.7.jar:./../lib/restli-server-1.15.7.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.7.jar:./../lib/slf4j-log4j12-1.7.7.jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-0.jar:./../lib/struts-core-1.3.8.jar:./../lib/struts-taglib-1.3.8.jar:./../lib/struts-tiles-1.3.8.jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar
2019/04/25 18:23:00.230 +0800 INFO [AzkabanWebServer] Starting Jetty Azkaban Web Server...
2019/04/25 18:23:00.247 +0800 INFO [AzkabanServer] Loading azkaban settings file from ./../conf
2019/04/25 18:23:00.247 +0800 INFO [AzkabanServer] Loading azkaban properties file
2019/04/25 18:23:00.249 +0800 INFO [AzkabanWebServer] Setting up connector with stats on: true
2019/04/25 18:23:00.270 +0800 INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2019/04/25 18:23:00.319 +0800 INFO [AzkabanWebServer] Loading user manager class azkaban.user.XmlUserManager
2019/04/25 18:23:00.456 +0800 INFO [XmlUserManager] Loading user azkaban
2019/04/25 18:23:00.456 +0800 INFO [XmlUserManager] Loading user metrics
2019/04/25 18:23:00.486 +0800 INFO [ExecutorManager] Initializing local executor localhost:12321
2019/04/25 18:23:00.785 +0800 ERROR [StdOutErrRedirect] Exception in thread "main" 
2019/04/25 18:23:00.785 +0800 ERROR [StdOutErrRedirect] azkaban.executor.ExecutorManagerException: Error fetching active flows
2019/04/25 18:23:00.785 +0800 ERROR [StdOutErrRedirect]         at azkaban.executor.JdbcExecutorLoader.fetchActiveFlows(JdbcExecutorLoader.java:216)
2019/04/25 18:23:00.785 +0800 ERROR [StdOutErrRedirect]         at azkaban.executor.ExecutorManager.loadRunningFlows(ExecutorManager.java:432)
2019/04/25 18:23:00.785 +0800 ERROR [StdOutErrRedirect]         at azkaban.executor.ExecutorManager.(ExecutorManager.java:127)
2019/04/25 18:23:00.786 +0800 ERROR [StdOutErrRedirect]         at azkaban.webapp.AzkabanWebServer.loadExecutorManager(AzkabanWebServer.java:281)
2019/04/25 18:23:00.786 +0800 ERROR [StdOutErrRedirect]         at azkaban.webapp.AzkabanWebServer.(AzkabanWebServer.java:202)
2019/04/25 18:23:00.786 +0800 ERROR [StdOutErrRedirect]         at azkaban.webapp.AzkabanWebServer.main(AzkabanWebServer.java:762)
2019/04/25 18:23:00.786 +0800 ERROR [StdOutErrRedirect] Caused by: java.sql.SQLException: Unknown column 'ex.executor_id' in 'on clause' Query: SELECT ex.exec_id exec_id, ex.enc_type enc_type, ex.flow_data flow_data, et.host host, et.port port, ax.update_time axUpdateTime, et.id executorId, et.active executorStatus FROM execution_flows ex INNER JOIN  active_executing_flows ax ON ex.exec_id = ax.exec_id INNER JOIN  executors et ON ex.executor_id = et.id Parameters: []
2019/04/25 18:23:00.786 +0800 ERROR [StdOutErrRedirect]         at org.apache.commons.dbutils.AbstractQueryRunner.rethrow(AbstractQueryRunner.java:363)
2019/04/25 18:23:00.786 +0800 ERROR [StdOutErrRedirect]         at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:350)
2019/04/25 18:23:00.786 +0800 ERROR [StdOutErrRedirect]         at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:306)
2019/04/25 18:23:00.786 +0800 ERROR [StdOutErrRedirect]         at azkaban.executor.JdbcExecutorLoader.fetchActiveFlows(JdbcExecutorLoader.java:212)
2019/04/25 18:23:00.786 +0800 ERROR [StdOutErrRedirect]         ... 5 more
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin/logs>

浏览器http://yourip:8443/，依然打不开,推测可能是sql语句没有执行完

root@hadoop01:/iot/azkaban-3.12.0/azkaban-sql>cat update.active_executing_flows.3.0.sql
ALTER TABLE active_executing_flows DROP COLUMN host;
ALTER TABLE active_executing_flows DROP COLUMN port;root@hadoop01:/iot/azkaban-3.12.0/azkaban-sql>
root@hadoop01:/iot/azkaban-3.12.0/azkaban-sql>cat update.execution_flows.3.0.sql
ALTER TABLE execution_flows ADD COLUMN executor_id INT DEFAULT NULL;
CREATE INDEX executor_id ON execution_flows(executor_id);root@hadoop01:/iot/azkaban-3.12.0/azkaban-sql>
root@hadoop01:/iot/azkaban-3.12.0/azkaban-sql>mysql -h10.10.10.10 -P3306 -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 572327
Server version: 5.7.24-log Source distribution

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> soure /iot/azkaban-3.12.0/azkaban-sql/update.active_executing_flows.3.0.sql;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'soure /iot/azkaban-3.12.0/azkaban-sql/update.active_executing_flows.3.0.sql' at line 1
mysql> ALTER TABLE active_executing_flows DROP COLUMN host;
ERROR 1046 (3D000): No database selected
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| azkaban            |
| mysql              |
| performance_schema |
| sys                |
| ums                |
+--------------------+
8 rows in set (0.00 sec)

mysql> use azkaban
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> soure /iot/azkaban-3.12.0/azkaban-sql/update.active_executing_flows.3.0.sql;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'soure /iot/azkaban-3.12.0/azkaban-sql/update.active_executing_flows.3.0.sql' at line 1
mysql> ALTER TABLE active_executing_flows DROP COLUMN host;
Query OK, 0 rows affected (0.05 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> ALTER TABLE active_executing_flows DROP COLUMN port;
Query OK, 0 rows affected (0.13 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> ALTER TABLE execution_flows ADD COLUMN executor_id INT DEFAULT NULL;
Query OK, 0 rows affected (0.06 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> CREATE INDEX executor_id ON execution_flows(executor_id);
Query OK, 0 rows affected (0.03 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> desc execution_flows
    -> ;
+-------------+--------------+------+-----+---------+----------------+
| Field       | Type         | Null | Key | Default | Extra          |
+-------------+--------------+------+-----+---------+----------------+
| exec_id     | int(11)      | NO   | PRI | NULL    | auto_increment |
| project_id  | int(11)      | NO   | MUL | NULL    |                |
| version     | int(11)      | NO   |     | NULL    |                |
| flow_id     | varchar(128) | NO   |     | NULL    |                |
| status      | tinyint(4)   | YES  |     | NULL    |                |
| submit_user | varchar(64)  | YES  |     | NULL    |                |
| submit_time | bigint(20)   | YES  |     | NULL    |                |
| update_time | bigint(20)   | YES  |     | NULL    |                |
| start_time  | bigint(20)   | YES  | MUL | NULL    |                |
| end_time    | bigint(20)   | YES  | MUL | NULL    |                |
| enc_type    | tinyint(4)   | YES  |     | NULL    |                |
| flow_data   | longblob     | YES  |     | NULL    |                |
| executor_id | int(11)      | YES  | MUL | NULL    |                |
+-------------+--------------+------+-----+---------+----------------+
13 rows in set (0.00 sec)

mysql> desc active_executing_flows
    -> ;
+-------------+------------+------+-----+---------+-------+
| Field       | Type       | Null | Key | Default | Extra |
+-------------+------------+------+-----+---------+-------+
| exec_id     | int(11)    | NO   | PRI | NULL    |       |
| update_time | bigint(20) | YES  |     | NULL    |       |
+-------------+------------+------+-----+---------+-------+
2 rows in set (0.00 sec)

mysql> 

停止进程后重启

root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>sh azkaban-web-shutdown.sh
Killing Web Server. [pid: 20995]
done.
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>ps -ef|grep az
root      27055  23775  0 18:48 pts/2    00:00:00 grep --color=auto az
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>sh start-web.sh 
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>ps -ef|grep az
root      27218      1 29 18:48 pts/2    00:00:02 java -Xmx4G -Dlog4j.configuration=file:./../conf/log4j.properties -server -Dcom.sun.management.jmxremote -Djava.io.tmpdir=/tmp -Dexecutorport=12321 -Dserverpath=/iot/azkaban-3.12.0/azkaban-web-server/bin -Dlog4j.log.dir=./../logs -cp .:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7.2.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-web-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7.0.jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/commons-validator-1.3.1.jar:./../lib/data-1.15.7.jar:./../lib/data-transform-1.15.7.jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2.2.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/javax.inject-1.jar:./../lib/javax.servlet-api-3.0.1.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-20070829.jar:./../lib/li-jersey-uri-1.15.7.jar:./../lib/log4j-1.2.16.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mina-core-1.1.7.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/netty-3.2.3.Final.jar:./../lib/oro-2.0.8.jar:./../lib/parseq-1.3.6.jar:./../lib/pegasus-common-1.15.7.jar:./../lib/quartz-2.2.1.jar:./../lib/r2-1.15.7.jar:./../lib/restli-common-1.15.7.jar:./../lib/restli-server-1.15.7.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.7.jar:./../lib/slf4j-log4j12-1.7.7.jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-0.jar:./../lib/struts-core-1.3.8.jar:./../lib/struts-taglib-1.3.8.jar:./../lib/struts-tiles-1.3.8.jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar azkaban.webapp.AzkabanWebServer -conf ./../conf ./..
root      27306  23775  0 18:48 pts/2    00:00:00 grep --color=auto az
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>

2.5.5报错：plugin path plugins/triggers doesn't exist!

查看日志，报错plugin path plugins/triggers doesn't exist!，并提示服务运行端口8081

root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin/logs>cat webServerLog_2019-04-25+18:48:35.out
Error: HADOOP_HOME is not set. Hadoop job types will not run properly.
./..
.:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7.2.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-web-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7.0.jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/commons-validator-1.3.1.jar:./../lib/data-1.15.7.jar:./../lib/data-transform-1.15.7.jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2.2.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/javax.inject-1.jar:./../lib/javax.servlet-api-3.0.1.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-20070829.jar:./../lib/li-jersey-uri-1.15.7.jar:./../lib/log4j-1.2.16.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mina-core-1.1.7.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/netty-3.2.3.Final.jar:./../lib/oro-2.0.8.jar:./../lib/parseq-1.3.6.jar:./../lib/pegasus-common-1.15.7.jar:./../lib/quartz-2.2.1.jar:./../lib/r2-1.15.7.jar:./../lib/restli-common-1.15.7.jar:./../lib/restli-server-1.15.7.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.7.jar:./../lib/slf4j-log4j12-1.7.7.jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-0.jar:./../lib/struts-core-1.3.8.jar:./../lib/struts-taglib-1.3.8.jar:./../lib/struts-tiles-1.3.8.jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar
2019/04/25 18:48:36.006 +0800 INFO [AzkabanWebServer] Starting Jetty Azkaban Web Server...
2019/04/25 18:48:36.020 +0800 INFO [AzkabanServer] Loading azkaban settings file from ./../conf
2019/04/25 18:48:36.020 +0800 INFO [AzkabanServer] Loading azkaban properties file
2019/04/25 18:48:36.022 +0800 INFO [AzkabanWebServer] Setting up connector with stats on: true
2019/04/25 18:48:36.043 +0800 INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2019/04/25 18:48:36.091 +0800 INFO [AzkabanWebServer] Loading user manager class azkaban.user.XmlUserManager
2019/04/25 18:48:36.119 +0800 INFO [XmlUserManager] Loading user azkaban
2019/04/25 18:48:36.120 +0800 INFO [XmlUserManager] Loading user metrics
2019/04/25 18:48:36.151 +0800 INFO [ExecutorManager] Initializing local executor localhost:12321
2019/04/25 18:48:36.464 +0800 INFO [AzkabanWebServer] Loading JDBC for project management
2019/04/25 18:48:36.465 +0800 INFO [ExecutorManager] Cleaning old logs from execution_logs
2019/04/25 18:48:36.468 +0800 INFO [ProjectManager] Project version retention is set to 3
2019/04/25 18:48:36.470 +0800 WARN [XmlValidatorManager] Validator directory validators does not exist or is not a directory.
2019/04/25 18:48:36.471 +0800 WARN [XmlValidatorManager] Azkaban properties file does not contain the key project.validators.xml.file
2019/04/25 18:48:36.515 +0800 INFO [ExecutorManager] Cleaning old log files before 2019-01-31T18:48:36.471+08:00
2019/04/25 18:48:36.523 +0800 INFO [ExecutorManager] Cleaned up 0 log entries.
2019/04/25 18:48:36.523 +0800 INFO [ExecutorManager] log clean up time: 0 seconds.
2019/04/25 18:48:36.550 +0800 INFO [TriggerManager] TriggerManager loaded.
2019/04/25 18:48:36.550 +0800 INFO [AzkabanWebServer] Loading built-in checker and action types
2019/04/25 18:48:36.554 +0800 INFO [CheckerTypeLoader] Registering checker BasicTimeChecker
2019/04/25 18:48:36.554 +0800 INFO [CheckerTypeLoader] Registering checker SlaChecker
2019/04/25 18:48:36.554 +0800 INFO [CheckerTypeLoader] Registering checker ExecutionChecker
2019/04/25 18:48:36.554 +0800 INFO [ActionTypeLoader] Registering action ExecuteFlowAction
2019/04/25 18:48:36.554 +0800 INFO [ActionTypeLoader] Registering action KillExecutionAction
2019/04/25 18:48:36.554 +0800 INFO [ActionTypeLoader] Registering action AlertAction
2019/04/25 18:48:36.554 +0800 INFO [ActionTypeLoader] Registering action CreateTriggerAction
2019/04/25 18:48:36.554 +0800 INFO [AzkabanWebServer] Loading trigger based scheduler
2019/04/25 18:48:36.556 +0800 INFO [AzkabanWebServer] Loading plug-in checker and action types
2019/04/25 18:48:36.556 +0800 ERROR [AzkabanWebServer] plugin path plugins/triggers doesn't exist!
2019/04/25 18:48:36.557 +0800 INFO [AzkabanWebServer] Setting timezone to Asia/Shanghai
2019/04/25 18:48:36.557 +0800 INFO [AzkabanWebServer] Registering MBeans...
2019/04/25 18:48:36.567 +0800 INFO [AzkabanWebServer] Bean azkaban.jmx.JmxJettyServer registered.
2019/04/25 18:48:36.569 +0800 INFO [AzkabanWebServer] Bean azkaban.jmx.JmxTriggerManager registered.
2019/04/25 18:48:36.571 +0800 INFO [AzkabanWebServer] Bean azkaban.jmx.JmxExecutorManager registered.
2019/04/25 18:48:36.582 +0800 INFO [AzkabanWebServer] Bean org.apache.log4j.jmx.HierarchyDynamicMBean registered.
2019/04/25 18:48:36.582 +0800 INFO [StdOutErrRedirect] ************* loginLoggerObjName is null, make sure there is a logger with name azkaban.webapp.servlet.LoginAbstractAzkabanServlet
2019/04/25 18:48:36.584 +0800 INFO [AzkabanWebServer] Setting up web resource dir web/
2019/04/25 18:48:36.636 +0800 INFO [JdbcTriggerLoader] Loading all triggers from db.
2019/04/25 18:48:36.654 +0800 INFO [JdbcTriggerLoader] Loaded 0 triggers.
2019/04/25 18:48:36.654 +0800 INFO [log] jetty-6.1.26
2019/04/25 18:48:36.680 +0800 INFO [ProjectManagerServlet] downloadBufferSize: 8192
2019/04/25 18:48:36.768 +0800 INFO [RestLiAnnotationReader] Processed actions resource 'azkaban.restli.ProjectManagerResource'
2019/04/25 18:48:36.788 +0800 INFO [RestLiAnnotationReader] Processed actions resource 'azkaban.restli.UserManagerResource'
2019/04/25 18:48:36.804 +0800 INFO [log] Started SocketConnector@0.0.0.0:8081
2019/04/25 18:48:36.804 +0800 INFO [AzkabanWebServer] Server running on  port 8081.
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin/logs>

2.6浏览器登陆

浏览器http://ip:8081,

 

 看到的页面奇丑无比，哈哈，解决方法如下 
修改azkaban.properties文件中的web.resource.dir 

同时修改以下项，在最后多加了个/

# Azkaban plugin settings
azkaban.jobtype.plugin.dir=/iot/azkaban-3.12.0/azkaban-web-server/plugins/jobtypes/

重启服务，再重新在浏览器输入http://ip:8081, 登陆信息azkaban/azkaban

 

启动日志中仍然有ERROR [AzkabanWebServer] plugin path plugins/triggers doesn't exist!

 2.7启动exec

root@hadoop01:/iot/azkaban-3.12.0>cd azkaban-exec-server
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server>ls
bin  lib
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server>cp -r ../azkaban-web-server/conf .
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server>ls
bin  conf  lib
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server>cp -r ../azkaban-web-server/plugins .
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server>ls
bin  conf  lib  plugins
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server>
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server/bin>mkdir logs
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server/bin>cat start-exec.sh 
#!/bin/bash

# pass along command line arguments to azkaban-executor-start.sh script
nohup sh azkaban-executor-start.sh "$@" >logs/executorServerLog__`date +%F+%T`.out 2>&1 &

root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server/bin>
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server/bin>sh start-exec.sh 
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server/bin>ps -ef|grep az
root      86579      1  0 09:24 ?        00:00:07 java -Xmx4G -Dlog4j.configuration=file:./../conf/log4j.properties -server -Dcom.sun.management.jmxremote -Djava.io.tmpdir=/tmp -Dexecutorport=12321 -Dserverpath=/iot/azkaban-3.12.0/azkaban-web-server/bin -Dlog4j.log.dir=./../logs -cp .:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7.2.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-web-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7.0.jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/commons-validator-1.3.1.jar:./../lib/data-1.15.7.jar:./../lib/data-transform-1.15.7.jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2.2.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/javax.inject-1.jar:./../lib/javax.servlet-api-3.0.1.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-20070829.jar:./../lib/li-jersey-uri-1.15.7.jar:./../lib/log4j-1.2.16.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mina-core-1.1.7.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/netty-3.2.3.Final.jar:./../lib/oro-2.0.8.jar:./../lib/parseq-1.3.6.jar:./../lib/pegasus-common-1.15.7.jar:./../lib/quartz-2.2.1.jar:./../lib/r2-1.15.7.jar:./../lib/restli-common-1.15.7.jar:./../lib/restli-server-1.15.7.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.7.jar:./../lib/slf4j-log4j12-1.7.7.jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-0.jar:./../lib/struts-core-1.3.8.jar:./../lib/struts-taglib-1.3.8.jar:./../lib/struts-tiles-1.3.8.jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar azkaban.webapp.AzkabanWebServer -conf ./../conf ./..
root     100716      1  2 10:18 pts/2    00:00:02 java -Xmx3G -Dlog4j.configuration=file:./../conf/log4j.properties -server -Dcom.sun.management.jmxremote -Djava.io.tmpdir=/tmp -Dexecutorport=12321 -Dserverpath=/iot/azkaban-3.12.0/azkaban-exec-server/bin -Dlog4j.log.dir=./../logs -cp .:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-exec-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/guava-13.0.1.jar:./../lib/hamcrest-core-1.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-simple-1.1.1.jar:./../lib/junit-4.10.jar:./../lib/kafka-clients-0.10.0.0.jar:./../lib/kafka-log4j-appender-0.10.0.0.jar:./../lib/log4j-1.2.17.jar:./../lib/lz4-1.3.0.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/quartz-2.2.1.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.21.jar:./../lib/slf4j-log4j12-1.7.21.jar:./../lib/snappy-java-1.1.2.4.jar:./../lib/velocity-1.7.jar:./../extlib/*.jar:./../plugins/*/*.jar azkaban.execapp.AzkabanExecutorServer -conf ./../conf
root     101191  99434  0 10:20 pts/2    00:00:00 grep --color=auto az
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server/bin>
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server/bin>cd logs
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server/bin/logs>ls
executorServerLog__2019-04-26+10:16:33.out  executorServerLog__2019-04-26+10:18:42.out
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server/bin/logs>ll
总用量 12
-rw------- 1 root root   79 4月  26 10:16 executorServerLog__2019-04-26+10:16:33.out
-rw------- 1 root root 6188 4月  26 10:19 executorServerLog__2019-04-26+10:18:42.out
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server/bin/logs>cat executorServerLog__2019-04-26+10:18:42.out
Error: HADOOP_HOME is not set. Hadoop job types will not run properly.
./..
.:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-exec-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/guava-13.0.1.jar:./../lib/hamcrest-core-1.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-simple-1.1.1.jar:./../lib/junit-4.10.jar:./../lib/kafka-clients-0.10.0.0.jar:./../lib/kafka-log4j-appender-0.10.0.0.jar:./../lib/log4j-1.2.17.jar:./../lib/lz4-1.3.0.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/quartz-2.2.1.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.21.jar:./../lib/slf4j-log4j12-1.7.21.jar:./../lib/snappy-java-1.1.2.4.jar:./../lib/velocity-1.7.jar:./../extlib/*.jar:./../plugins/*/*.jar
Starting AzkabanExecutorServer on port 12321 ...
2019/04/26 10:18:42.828 +0800 INFO [AzkabanExecutorServer] Starting Jetty Azkaban Executor...
2019/04/26 10:18:42.845 +0800 INFO [AzkabanServer] Loading azkaban settings file from ./../conf
2019/04/26 10:18:42.845 +0800 INFO [AzkabanServer] Loading azkaban properties file
2019/04/26 10:18:42.862 +0800 INFO [AzkabanExecutorServer] Setting timezone to Asia/Shanghai
2019/04/26 10:18:42.885 +0800 INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2019/04/26 10:18:42.887 +0800 INFO [AzkabanExecutorServer] Setting up connector with stats on: true
2019/04/26 10:18:42.889 +0800 INFO [AzkabanExecutorServer] Jetty connector name: 0.0.0.0:12321, default header buffer size: 4096
2019/04/26 10:18:42.889 +0800 INFO [AzkabanExecutorServer] Jetty connector name: 0.0.0.0:12321, (if) new header buffer size: 4096
2019/04/26 10:18:42.943 +0800 INFO [FlowRunnerManager] Execution dir retention set to 86400000 ms
2019/04/26 10:18:42.944 +0800 INFO [FlowRunnerManager] useNewThreadPool: false
2019/04/26 10:18:42.946 +0800 INFO [FlowRunnerManager] Cleaning recently finished
2019/04/26 10:18:42.946 +0800 INFO [FlowRunnerManager] Cleaning old projects
2019/04/26 10:18:42.946 +0800 INFO [FlowRunnerManager] Cleaning old execution dirs
2019/04/26 10:18:42.948 +0800 INFO [JobTypeManager] Loading plugin default job types
2019/04/26 10:18:42.951 +0800 INFO [JobTypeManager] Job type plugin directory set. Loading extra job types from /iot/azkaban-3.12.0/azkaban-web-server/plugins/jobtypes
2019/04/26 10:18:42.951 +0800 INFO [JobTypeManager] Common plugin job props file /iot/azkaban-3.12.0/azkaban-web-server/plugins/jobtypes/common.properties not found. Using empty props.
2019/04/26 10:18:42.951 +0800 INFO [JobTypeManager] Common plugin load props file /iot/azkaban-3.12.0/azkaban-web-server/plugins/jobtypes/commonprivate.properties found. Attempt to load.
2019/04/26 10:18:42.952 +0800 INFO [JmxJobMBeanManager] Initializing azkaban.execapp.jmx.JmxJobMBeanManager
2019/04/26 10:18:42.952 +0800 INFO [AzkabanExecutorServer] Job callback enabled? true
2019/04/26 10:18:42.954 +0800 INFO [JobCallbackManager] Initializing
2019/04/26 10:18:42.957 +0800 INFO [JobCallbackRequestMaker] responseWaitTimeoutMS: 5000
2019/04/26 10:18:42.958 +0800 INFO [JobCallbackRequestMaker] Global request configuration , expectContinueEnabled=false, proxy=null, localAddress=null, staleConnectionCheckEnabled=true, cookieSpec=null, redirectsEnabled=true, relativeRedirectsAllowed=true, maxRedirects=50, circularRedirectsAllowed=false, authenticationEnabled=true, targetPreferredAuthSchemes=null, proxyPreferredAuthSchemes=null, connectionRequestTimeout=3000, connectTimeout=3000, socketTimeout=3000]
2019/04/26 10:18:43.199 +0800 INFO [JobCallbackRequestMaker] Jobcall thread pool size: 10
2019/04/26 10:18:43.201 +0800 INFO [JobCallbackRequestMaker] Initialization for azkaban.execapp.event.JobCallbackRequestMaker is completed
2019/04/26 10:18:43.202 +0800 INFO [JobCallbackManager] Initialization completed azkaban.execapp.event.JobCallbackManager
2019/04/26 10:18:43.202 +0800 INFO [JobCallbackManager] azkabanHostName hadoop01
2019/04/26 10:18:43.202 +0800 INFO [AzkabanExecutorServer] Registering MBeans...
2019/04/26 10:18:43.211 +0800 INFO [AzkabanExecutorServer] Bean azkaban.jmx.JmxJettyServer registered.
2019/04/26 10:18:43.212 +0800 INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxFlowRunnerManager registered.
2019/04/26 10:18:43.214 +0800 INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxJobMBeanManager registered.
2019/04/26 10:18:43.214 +0800 INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxJobCallback registered.
2019/04/26 10:18:43.216 +0800 INFO [SystemMemoryInfo] Current system free memory is 16159652 kb (MemFree 237400, Buffers 193984, Cached 15722952, SwapCached 5316)
2019/04/26 10:18:43.216 +0800 INFO [SystemMemoryInfo] Scheduled thread to read /proc/meminfo every 30 seconds
2019/04/26 10:18:43.218 +0800 INFO [AzkabanExecutorServer] No value for property: jmx.attribute.processor.class was found
2019/04/26 10:18:43.218 +0800 INFO [log] jetty-6.1.26
2019/04/26 10:18:43.219 +0800 INFO [SystemMemoryInfo] Current system free memory is 16159280 kb (MemFree 237028, Buffers 193984, Cached 15722952, SwapCached 5316)
2019/04/26 10:18:43.240 +0800 INFO [log] Started SocketConnector@0.0.0.0:12321
2019/04/26 10:18:43.263 +0800 INFO [AzkabanExecutorServer] Started Executor Server on hadoop01:12321
2019/04/26 10:19:13.218 +0800 INFO [SystemMemoryInfo] Current system free memory is 16154456 kb (MemFree 231688, Buffers 193984, Cached 15723468, SwapCached 5316)

3.简单使用

详细请看这里：azkaban的简单使用

查看flow log

 4.分布式部署（multiple-executor）

以上是two-server 方式运行，即web-service和exec-server运行在同一台机器上，且使用数据库。

分布式，即multiple-executor，即将exec-server运行在更多的节点上。

我们先关闭本机上的exec

root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server/bin>sh azkaban-executor-s
azkaban-executor-shutdown.sh  azkaban-executor-start.sh     
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server/bin>sh azkaban-executor-shutdown.sh 
cat: ./../executor.port: 没有那个文件或目录
Killing Executor. [pid: 100716, port: ]
rm: 无法删除"./../executor.port": 没有那个文件或目录
done.
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server/bin>ps -ef|grep az
root      43139  42477  0 15:12 pts/2    00:00:00 grep --color=auto az
root      86579      1  0 09:24 ?        00:00:20 java -Xmx4G -Dlog4j.configuration=file:./../conf/log4j.properties -server -Dcom.sun.management.jmxremote -Djava.io.tmpdir=/tmp -Dexecutorport=12321 -Dserverpath=/iot/azkaban-3.12.0/azkaban-web-server/bin -Dlog4j.log.dir=./../logs -cp .:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7.2.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-web-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7.0.jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/commons-validator-1.3.1.jar:./../lib/data-1.15.7.jar:./../lib/data-transform-1.15.7.jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2.2.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/javax.inject-1.jar:./../lib/javax.servlet-api-3.0.1.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-20070829.jar:./../lib/li-jersey-uri-1.15.7.jar:./../lib/log4j-1.2.16.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mina-core-1.1.7.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/netty-3.2.3.Final.jar:./../lib/oro-2.0.8.jar:./../lib/parseq-1.3.6.jar:./../lib/pegasus-common-1.15.7.jar:./../lib/quartz-2.2.1.jar:./../lib/r2-1.15.7.jar:./../lib/restli-common-1.15.7.jar:./../lib/restli-server-1.15.7.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.7.jar:./../lib/slf4j-log4j12-1.7.7.jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-0.jar:./../lib/struts-core-1.3.8.jar:./../lib/struts-taglib-1.3.8.jar:./../lib/struts-tiles-1.3.8.jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar azkaban.webapp.AzkabanWebServer -conf ./../conf ./..
root@hadoop01:/iot/azkaban-3.12.0/azkaban-exec-server/bin>

在azkaban服务器(本机hadoop01)上配置好exec节点ip，这里cdh已经配置好了每台的hosts都配置成一样的，也可以

iotapp@hadoop01:/iot>cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.10.10.70  hadoop01
10.10.10.71  hadoop02
10.10.10.72  hadoop03
10.10.10.84  hadoop04
10.10.10.85  hadoop05

拷贝exec到hadoop02和hadoop03节点

这里hadoop02，hadoop03无法直接sftp root,可以在hadoop01将目录压缩成tar包，用普通用户sftp user后再chown root后再tar -xvf即可，保留azkaban-exec,删除其它文件

hadoop02，hadoop03目录结构

root@hadoop03:/iot/azkaban-3.12.0>ls
azkaban-exec-server
root@hadoop03:/iot/azkaban-3.12.0>

hadoop01关闭web-server修改配置文件

root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>sh azkaban-web-shutdown.sh
Killing Web Server. [pid: 86579]
done.
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>ps -ef|grep az
root      93612  92827  0 18:19 pts/3    00:00:00 grep --color=auto az
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/conf> vi azkaban.properties ##新增下面内容
#启用multiple-executor模式
azkaban.use.multiple.executors=true

#在每次分发job时，先过滤出满足条件的executor，然后再做比较筛选
#如最小剩余内存,MinimumFreeMemory,过滤器会检查executor空余内存是否会大于6G，如果不足6G，则web-server不会将任务交由该executor执行。可参考Azkaban Github源码
#如CpuStatus，过滤器会检查executor的cpu占用率是否达到95%，若达到95%，web-server也不会将任务交给该executor执行。可参考Azkaban Github源码。
#参数含义参考官网说明http://azkaban.github.io/azkaban/docs/latest/#configuration
#azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus
# 由于是虚拟机，不需要过滤，只需要比较即可
# 某个任务是否指定了executor id
azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1
# 是否比较内存
azkaban.executorselector.comparator.Memory=1
# 是否最后一次被分发
azkaban.executorselector.comparator.LastDispatched=1
# 是否比较CPU
azkaban.executorselector.comparator.CpuUsage=1

在mysql中增加executors表中添加executor

主机上再次验证

mysql> use azkaban 
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+------------------------+
| Tables_in_azkaban      |
+------------------------+
| active_executing_flows |
| active_sla             |
| execution_flows        |
| execution_jobs         |
| execution_logs         |
| executor_events        |
| executors              |
| project_events         |
| project_files          |
| project_flows          |
| project_permissions    |
| project_properties     |
| project_versions       |
| projects               |
| properties             |
| schedules              |
| triggers               |
+------------------------+
17 rows in set (0.00 sec)

mysql> select * from executors 
    -> ;
+----+-----------+-------+--------+
| id | host      | port  | active |
+----+-----------+-------+--------+
|  1 | localhost | 12321 |      1 |
|  3 | hadoop01  | 12321 |      0 |
|  4 | hadoop02  | 12321 |      1 |
|  5 | hadoop03  | 12321 |      1 |
+----+-----------+-------+--------+
4 rows in set (0.00 sec)

mysql> 

 然后可以启动程序了,顺序为web-server -> hadoop02 exec-server -> hadoop03 exec-server

root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>sh start-web.sh
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>ps -ef|grep az
root     104317      1 30 19:04 pts/2    00:00:02 java -Xmx4G -Dlog4j.configuration=file:./../conf/log4j.properties -server -Dcom.sun.management.jmxremote -Djava.io.tmpdir=/tmp -Dexecutorport=12321 -Dserverpath=/iot/azkaban-3.12.0/azkaban-web-server/bin -Dlog4j.log.dir=./../logs -cp .:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7.2.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-web-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7.0.jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/commons-validator-1.3.1.jar:./../lib/data-1.15.7.jar:./../lib/data-transform-1.15.7.jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2.2.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/javax.inject-1.jar:./../lib/javax.servlet-api-3.0.1.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-20070829.jar:./../lib/li-jersey-uri-1.15.7.jar:./../lib/log4j-1.2.16.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mina-core-1.1.7.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/netty-3.2.3.Final.jar:./../lib/oro-2.0.8.jar:./../lib/parseq-1.3.6.jar:./../lib/pegasus-common-1.15.7.jar:./../lib/quartz-2.2.1.jar:./../lib/r2-1.15.7.jar:./../lib/restli-common-1.15.7.jar:./../lib/restli-server-1.15.7.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.7.jar:./../lib/slf4j-log4j12-1.7.7.jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-0.jar:./../lib/struts-core-1.3.8.jar:./../lib/struts-taglib-1.3.8.jar:./../lib/struts-tiles-1.3.8.jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar azkaban.webapp.AzkabanWebServer -conf ./../conf ./..
root     104365 100529  0 19:04 pts/2    00:00:00 grep --color=auto az
root@hadoop01:/iot/azkaban-3.12.0/azkaban-web-server/bin>

root@hadoop02:/iot/azkaban-3.12.0/azkaban-exec-server/bin>sh start-exec.sh
root@hadoop02:/iot/azkaban-3.12.0/azkaban-exec-server/bin>ps -ef|grep az
root      77640      1  9 19:05 pts/1    00:00:01 java -Xmx3G -Dlog4j.configuration=file:./../conf/log4j.properties -server -Dcom.sun.management.jmxremote -Djava.io.tmpdir=/tmp -Dexecutorport=12321 -Dserverpath=/iot/azkaban-3.12.0/azkaban-exec-server/bin -Dlog4j.log.dir=./../logs -cp .:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-exec-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/guava-13.0.1.jar:./../lib/hamcrest-core-1.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-simple-1.1.1.jar:./../lib/junit-4.10.jar:./../lib/kafka-clients-0.10.0.0.jar:./../lib/kafka-log4j-appender-0.10.0.0.jar:./../lib/log4j-1.2.17.jar:./../lib/lz4-1.3.0.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/quartz-2.2.1.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.21.jar:./../lib/slf4j-log4j12-1.7.21.jar:./../lib/snappy-java-1.1.2.4.jar:./../lib/velocity-1.7.jar:./../extlib/*.jar:./../plugins/*/*.jar azkaban.execapp.AzkabanExecutorServer -conf ./../conf
root      77724  52993  0 19:05 pts/1    00:00:00 grep --color=auto az
root@hadoop02:/iot/azkaban-3.12.0/azkaban-exec-server/bin>

root@hadoop03:/iot/azkaban-3.12.0/azkaban-exec-server/bin/logs>ll
总用量 144
-rw------- 1 root root 79 4月 26 10:16 executorServerLog__2019-04-26+10:16:33.out
-rw------- 1 root root 135628 4月 26 15:12 executorServerLog__2019-04-26+10:18:42.out
-rw------- 1 root root 4031 4月 26 19:06 executorServerLog__2019-04-26+19:06:54.out
root@hadoop03:/iot/azkaban-3.12.0/azkaban-exec-server/bin/logs>cat executorServerLog__2019-04-26+19:06:54.out
Error: HADOOP_HOME is not set. Hadoop job types will not run properly.
./..
.:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-exec-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/guava-13.0.1.jar:./../lib/hamcrest-core-1.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-simple-1.1.1.jar:./../lib/junit-4.10.jar:./../lib/kafka-clients-0.10.0.0.jar:./../lib/kafka-log4j-appender-0.10.0.0.jar:./../lib/log4j-1.2.17.jar:./../lib/lz4-1.3.0.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/quartz-2.2.1.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.21.jar:./../lib/slf4j-log4j12-1.7.21.jar:./../lib/snappy-java-1.1.2.4.jar:./../lib/velocity-1.7.jar:./../extlib/*.jar:./../plugins/*/*.jar
Starting AzkabanExecutorServer on port 12321 ...
2019/04/26 19:06:54.827 +0800 INFO [AzkabanExecutorServer] Starting Jetty Azkaban Executor...
2019/04/26 19:06:54.841 +0800 INFO [AzkabanServer] Loading azkaban settings file from ./../conf
2019/04/26 19:06:54.842 +0800 INFO [AzkabanServer] Loading azkaban properties file
2019/04/26 19:06:54.862 +0800 INFO [AzkabanExecutorServer] Setting timezone to Asia/Shanghai
2019/04/26 19:06:54.883 +0800 INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2019/04/26 19:06:54.885 +0800 INFO [AzkabanExecutorServer] Setting up connector with stats on: true
2019/04/26 19:06:54.887 +0800 INFO [AzkabanExecutorServer] Jetty connector name: 0.0.0.0:12321, default header buffer size: 4096
2019/04/26 19:06:54.887 +0800 INFO [AzkabanExecutorServer] Jetty connector name: 0.0.0.0:12321, (if) new header buffer size: 4096
2019/04/26 19:06:54.944 +0800 INFO [FlowRunnerManager] Execution dir retention set to 86400000 ms
2019/04/26 19:06:54.945 +0800 INFO [FlowRunnerManager] useNewThreadPool: false
2019/04/26 19:06:54.949 +0800 INFO [FlowRunnerManager] Cleaning recently finished
2019/04/26 19:06:54.949 +0800 ERROR [StdOutErrRedirect] Exception in thread "main"
2019/04/26 19:06:54.949 +0800 INFO [FlowRunnerManager] Cleaning old projects
2019/04/26 19:06:54.949 +0800 INFO [FlowRunnerManager] Cleaning old execution dirs
2019/04/26 19:06:54.949 +0800 ERROR [StdOutErrRedirect] java.io.FileNotFoundException: /iot/azkaban-3.12.0/azkaban-web-server/conf/global.properties (没有那个文件或目录)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at java.io.FileInputStream.open0(Native Method)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at java.io.FileInputStream.open(FileInputStream.java:195)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at java.io.FileInputStream.(FileInputStream.java:138)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at azkaban.utils.Props.(Props.java:89)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at azkaban.utils.Props.(Props.java:75)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at azkaban.execapp.FlowRunnerManager.(FlowRunnerManager.java:203)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at azkaban.execapp.AzkabanExecutorServer.(AzkabanExecutorServer.java:119)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at azkaban.execapp.AzkabanExecutorServer.main(AzkabanExecutorServer.java:382)
root@hadoop03:/iot/azkaban-3.12.0/azkaban-exec-server/bin/logs>ls

root@hadoop03:/iot/azkaban-3.12.0/azkaban-exec-server/conf>vi azkaban.properties 

...

# Loader for projects
executor.global.properties=/iot/azkaban-3.12.0/azkaban-exec-server/conf/global.properties
azkaban.project.dir=projects

....

# Azkaban plugin settings
azkaban.jobtype.plugin.dir=/iot/azkaban-3.12.0/azkaban-exec-server/plugins/jobtypes/

...

root@hadoop03:/iot/azkaban-3.12.0/azkaban-exec-server/bin>sh azkaban-executor-shutdown.sh
cat: ./../executor.port: 没有那个文件或目录
Killing Executor. [pid: 69178, port: ]
rm: 无法删除"./../executor.port": 没有那个文件或目录
done.
root@hadoop03:/iot/azkaban-3.12.0/azkaban-exec-server/bin>sh start-exec.sh
root@hadoop03:/iot/azkaban-3.12.0/azkaban-exec-server/bin>cd ../logs
-bash: cd: ../logs: 没有那个文件或目录
root@hadoop03:/iot/azkaban-3.12.0/azkaban-exec-server/bin>ls
azkaban-executor-shutdown.sh azkaban-executor-start.sh executions executor.port logs projects start-exec.sh temp
root@hadoop03:/iot/azkaban-3.12.0/azkaban-exec-server/bin>cd logs
root@hadoop03:/iot/azkaban-3.12.0/azkaban-exec-server/bin/logs>ls
executorServerLog__2019-04-26+10:16:33.out executorServerLog__2019-04-26+10:18:42.out executorServerLog__2019-04-26+19:06:54.out executorServerLog__2019-04-26+19:11:25.out
root@hadoop03:/iot/azkaban-3.12.0/azkaban-exec-server/bin/logs>ll
总用量 156
-rw------- 1 root root 79 4月 26 10:16 executorServerLog__2019-04-26+10:16:33.out
-rw------- 1 root root 135628 4月 26 15:12 executorServerLog__2019-04-26+10:18:42.out
-rw------- 1 root root 4195 4月 26 19:10 executorServerLog__2019-04-26+19:06:54.out
-rw------- 1 root root 5479 4月 26 19:11 executorServerLog__2019-04-26+19:11:25.out
root@hadoop03:/iot/azkaban-3.12.0/azkaban-exec-server/bin/logs>cat executorServerLog__2019-04-26+19:11:25.out
Error: HADOOP_HOME is not set. Hadoop job types will not run properly.
./..
.:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-exec-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/guava-13.0.1.jar:./../lib/hamcrest-core-1.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-simple-1.1.1.jar:./../lib/junit-4.10.jar:./../lib/kafka-clients-0.10.0.0.jar:./../lib/kafka-log4j-appender-0.10.0.0.jar:./../lib/log4j-1.2.17.jar:./../lib/lz4-1.3.0.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/quartz-2.2.1.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.21.jar:./../lib/slf4j-log4j12-1.7.21.jar:./../lib/snappy-java-1.1.2.4.jar:./../lib/velocity-1.7.jar:./../extlib/*.jar:./../plugins/*/*.jar
Starting AzkabanExecutorServer on port 12321 ...
2019/04/26 19:11:25.928 +0800 INFO [AzkabanExecutorServer] Starting Jetty Azkaban Executor...
2019/04/26 19:11:25.943 +0800 INFO [AzkabanServer] Loading azkaban settings file from ./../conf
2019/04/26 19:11:25.943 +0800 INFO [AzkabanServer] Loading azkaban properties file
2019/04/26 19:11:25.961 +0800 INFO [AzkabanExecutorServer] Setting timezone to Asia/Shanghai
2019/04/26 19:11:25.984 +0800 INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2019/04/26 19:11:25.985 +0800 INFO [AzkabanExecutorServer] Setting up connector with stats on: true
2019/04/26 19:11:25.987 +0800 INFO [AzkabanExecutorServer] Jetty connector name: 0.0.0.0:12321, default header buffer size: 4096
2019/04/26 19:11:25.987 +0800 INFO [AzkabanExecutorServer] Jetty connector name: 0.0.0.0:12321, (if) new header buffer size: 4096
2019/04/26 19:11:26.039 +0800 INFO [FlowRunnerManager] Execution dir retention set to 86400000 ms
2019/04/26 19:11:26.040 +0800 INFO [FlowRunnerManager] useNewThreadPool: false
2019/04/26 19:11:26.042 +0800 INFO [FlowRunnerManager] Cleaning recently finished
2019/04/26 19:11:26.043 +0800 INFO [FlowRunnerManager] Cleaning old projects
2019/04/26 19:11:26.043 +0800 INFO [FlowRunnerManager] Cleaning old execution dirs
2019/04/26 19:11:26.044 +0800 INFO [JobTypeManager] Loading plugin default job types
2019/04/26 19:11:26.047 +0800 INFO [JmxJobMBeanManager] Initializing azkaban.execapp.jmx.JmxJobMBeanManager
2019/04/26 19:11:26.047 +0800 INFO [AzkabanExecutorServer] Job callback enabled? true
2019/04/26 19:11:26.049 +0800 INFO [JobCallbackManager] Initializing
2019/04/26 19:11:26.052 +0800 INFO [JobCallbackRequestMaker] responseWaitTimeoutMS: 5000
2019/04/26 19:11:26.053 +0800 INFO [JobCallbackRequestMaker] Global request configuration , expectContinueEnabled=false, proxy=null, localAddress=null, staleConnectionCheckEnabled=true, cookieSpec=null, redirectsEnabled=true, relativeRedirectsAllowed=true, maxRedirects=50, circularRedirectsAllowed=false, authenticationEnabled=true, targetPreferredAuthSchemes=null, proxyPreferredAuthSchemes=null, connectionRequestTimeout=3000, connectTimeout=3000, socketTimeout=3000]
2019/04/26 19:11:26.261 +0800 INFO [JobCallbackRequestMaker] Jobcall thread pool size: 10
2019/04/26 19:11:26.262 +0800 INFO [JobCallbackRequestMaker] Initialization for azkaban.execapp.event.JobCallbackRequestMaker is completed
2019/04/26 19:11:26.262 +0800 INFO [JobCallbackManager] Initialization completed azkaban.execapp.event.JobCallbackManager
2019/04/26 19:11:26.263 +0800 INFO [JobCallbackManager] azkabanHostName hadoop03
2019/04/26 19:11:26.263 +0800 INFO [AzkabanExecutorServer] Registering MBeans...
2019/04/26 19:11:26.271 +0800 INFO [AzkabanExecutorServer] Bean azkaban.jmx.JmxJettyServer registered.
2019/04/26 19:11:26.272 +0800 INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxFlowRunnerManager registered.
2019/04/26 19:11:26.273 +0800 INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxJobMBeanManager registered.
2019/04/26 19:11:26.274 +0800 INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxJobCallback registered.
2019/04/26 19:11:26.275 +0800 INFO [SystemMemoryInfo] Current system free memory is 29289720 kb (MemFree 417156, Buffers 178268, Cached 28694296, SwapCached 0)
2019/04/26 19:11:26.276 +0800 INFO [SystemMemoryInfo] Scheduled thread to read /proc/meminfo every 30 seconds
2019/04/26 19:11:26.277 +0800 INFO [AzkabanExecutorServer] No value for property: jmx.attribute.processor.class was found
2019/04/26 19:11:26.277 +0800 INFO [log] jetty-6.1.26
2019/04/26 19:11:26.278 +0800 INFO [SystemMemoryInfo] Current system free memory is 29289720 kb (MemFree 417156, Buffers 178268, Cached 28694296, SwapCached 0)
2019/04/26 19:11:26.305 +0800 INFO [log] Started [email protected]:12321
2019/04/26 19:11:26.332 +0800 INFO [AzkabanExecutorServer] Started Executor Server on hadoop03:12321

报错：/iot/azkaban-3.12.0/azkaban-web-server/conf/global.properties (没有那个文件或目录)，将配置文件中该项改为/iot/azkaban-3.12.0/azkaban-exec-server/conf/global.properties 

重启即可

 

单击execution id为23，发现在hadoop03上执行了

至此，分布式azkaban安装完毕。

 

转载于:https://www.cnblogs.com/pu20065226/p/10767446.html