Azkaban配置

1.下载Azkaban
1.1登陆Azkaban的官网：https://azkaban.github.io/ 点击Downloads，如图示：

1.2点击之后，在跳转的页面中选择Releases，进入页面选择相应的版本下载，这里选择的版本是3.70.0版本，点击“Source code(tar.gz)”下载。

1.3 选择自己要下载的源码，下载

2.环境准备
2.1 在安装之前要安装jdk,MySQL
2.2 安装git，git是一个开源的分布式版本控制系统，一般在项目版本控制中会使用git控制，这里安装Azkaban需要git是因为需要通过git构建依赖的包。git安装https://mp.csdn.net/mdeditor/91409996#
3.安装azkaban
3.1 上传下载好的azkaban，解压到/azkaban-temp文件夹
3.2 进入azkaban-temp目录，进行编译

1.[root@mynode5 azkaban]# ./gradlew distTar
2.Downloading https://services.gradle.org/distributions/gradle-4.6-all.zip
3......................................................................................................
4.Unzipping /root/.gradle/wrapper/dists/gradle-4.6-all/bcst21l2brirad8k2ben1letg/gradle-4.6-all.zip to /root/.gradle/wrapper/dists/gradle-4.6-all/bcst21l2brirad8k2ben1letg
5.Set executable permissions for: /root/.gradle/wrapper/dists/gradle-4.6-all/bcst21l2brirad8k2ben1letg/gradle-4.6/bin/gradle
6.Starting a Gradle Daemon (subsequent builds will be faster)
7.Parallel execution with configuration on demand is an incubating feature.
8.Download https://plugins.gradle.org/m2/com/gradle/build-scan/com.gradle.build-scan.gradle.plugin/1.9/com.gradle.build-scan.gradle.plugin-1.9.pom
9.... ...
10.Download https://repo.maven.apache.org/maven2/org/codehaus/jackson/jackson-core-asl/1.8.8/jackson-core-asl-1.8.8.pom
11.Download https://repo.maven.apache.org/maven2/org/codehaus/jackson/jackson-mapper-asl/1.8.8/jackson-mapper-asl-1.8.8.pom
12.Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.7/commons-codec-1.7.jar8.8.pom
13.Download https://repo.maven.apache.org/maven2/com/twitter/parquet-hadoop-bundle/1.3.2/parquet-hadoop-bundle-1.3.2.jar
14.Download https://repo.maven.apache.org/maven2/org/codehaus/jackson/jackson-core-asl/1.8.8/jackson-core-asl-1.8.8.jar
15.Download https://repo.maven.apache.org/maven2/org/codehaus/jackson/jackson-mapper-asl/1.8.8/jackson-mapper-asl-1.8.8.jar
16.
17.BUILD SUCCESSFUL in 4m 6s
18.54 actionable tasks: 40 executed, 14 from cache

注意：编译过程中有可能由于网络延时造成编译时失败，可以多重试几次解决此问题

3.3新建azkaban目录，将编译好的文件复制到此目录下

1.mkdir /azkaban
2.cd azkaban
3.cp /azkaban-temp/azkaban-db/build/distributions/azkaban-db-0.1.0-SNAPSHOT.tar.gz /sazkaban
4.cp /azkaban-temp/azkaban-web-server/build/distributions/azkaban-web-server-0.1.0-SNAPSHOT.tar.gz /azkaban
5.cp /azkaban-temp/azkaban-exec-server/build/distributions/azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz /azkaban

3.4将文件解压重命名

1. tar -zxvf azkaban-db-0.1.0-SNAPSHOT.tar.gz 
2. tar -zxvf azkaban-web-server-0.1.0-SNAPSHOT.tar.gz
3. tar -zxvf azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz 
4. mv azkaban-db-0.1.0-SNAPSHOT azkaban-db
5. mv azkaban-web-server-0.1.0-SNAPSHOT azkaban-web
6. mv azkaban-exec-server-0.1.0-SNAPSHOT azkaban-exec

4.导入数据库

4.1 登录mysql数据库，创建azkaban数据库
	mysql -u root -p123456
	create database azkaban；
4.2 准备sql文件，azkaban-db目录下的create-all-sql-0.1.0-SNAPSHOT.sql下的sql文件放到数据库所在的节点，我的azkaban和mysql在同一节点，不用移动
4.3 导入sql文件
	use azkaban;
	source /azkaban/azkaban-db/create-all-sql-0.1.0-SNAPSHOT.sql
4.4 可以查看一下表，一共29个
	show tables;
	mysql> show tables;
	+--------------------------+
	| Tables_in_azkaban        |
	+--------------------------+
	| QRTZ_BLOB_TRIGGERS       |
	| QRTZ_CALENDARS           |
	| QRTZ_CRON_TRIGGERS       |
	| QRTZ_FIRED_TRIGGERS      |
	| QRTZ_JOB_DETAILS         |
	| QRTZ_LOCKS               |
	| QRTZ_PAUSED_TRIGGER_GRPS |
	| QRTZ_SCHEDULER_STATE     |
	| QRTZ_SIMPLE_TRIGGERS     |
	| QRTZ_SIMPROP_TRIGGERS    |
	| QRTZ_TRIGGERS            |
	| active_executing_flows   |
	| active_sla               |
	| execution_dependencies   |
	| execution_flows          |
	| execution_jobs           |
	| execution_logs           |
	| executor_events          |
	| executors                |
	| project_events           |
	| project_files            |
	| project_flow_files       |
	| project_flows            |
	| project_permissions      |
	| project_properties       |
	| project_versions         |
	| projects                 |
	| properties               |
	| triggers                 |
	+--------------------------+
	29 rows in set (0.00 sec)

5.配置azkaban
5.1 /azkaban目录下执行命令：keytool -keystore keystore -alias jetty -genkey -keyalg RSA 创建ssl配置。

keytool -keystore keystore -alias jetty -genkey -keyalg RSA
2.输入密钥库口令: 
3.再次输入新口令: 
4.您的名字与姓氏是什么?
5.[Unknown]: azkaban
6.您的组织单位名称是什么?
7.[Unknown]: azkaban
8.您的组织名称是什么?
9.[Unknown]: azkaban
10.您所在的城市或区域名称是什么?
11.[Unknown]: beijing
12.您所在的省/市/自治区名称是什么?
13.[Unknown]: beijing
14.该单位的双字母国家/地区代码是什么?
15.[Unknown]: CN
16.CN=azkaban, OU=azkaban, O=azkaban, L=beijing, ST=beijing, C=CN是否正确?
17.[否]: Y 
18.
19.输入  的密钥口令
20.(如果和密钥库口令相同, 按回车):
输入完信息，执行完以上命令之后，在当前目录下生成一个keystore文件，将此文件复制到azkaban web服务器根目录下。
	mv /azkaban/keystore /azkaban/azkaban-web

5.2 Azkaban web 服务器配置

进入/azkaban/azkaban-web/conf目录下，编辑azkaban.properties文件：
1.cd /azkaban/azkaban-web/conf/
	vim azkaban.properties
		# Azkaban Personalization Settings
		azkaban.name=Test
		azkaban.label=My Local Azkaban
		azkaban.color=#FF3601
		azkaban.default.servlet.path=/index
		web.resource.dir=/azkaban/azkaban-web/web
		default.timezone.id=Asia/Shanghai
		# Azkaban UserManager class
		user.manager.class=azkaban.user.XmlUserManager
		user.manager.xml.file=/azkaban/azkaban-web/conf/azkaban-users.xml
		# Loader for projects
		executor.global.properties=/azkaban/azkaban-web/conf/global.properties
		azkaban.project.dir=projects
		# Velocity dev mode
		velocity.dev.mode=false
		# Azkaban Jetty server properties.
		jetty.use.ssl=false
		jetty.maxThreads=25
		jetty.port=8081
		jetty.keystore=/azkaban/azkaban-web/keystore #SSL 文件名
		jetty.password=123456 #SSL 文件密码
		jetty.keypassword=123456  #Jetty 主密码 与 keystore 文件相同
		jetty.truststore=/azkaban/azkaban-web/keystore #SSL 文件名
		jetty.trustpassword=123456 # SSL 文件密码
		# Azkaban Executor settings
		# mail settings
		mail.sender=
		mail.host=			
		# User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users.
		# enduser -> myazkabanhost:443 -> proxy -> localhost:8081
		# when this parameters set then these parameters are used to generate email links.
		# if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.
		# azkaban.webserver.external_hostname=myazkabanhost.com
		# azkaban.webserver.external_ssl_port=443
		# azkaban.webserver.external_port=8081
		job.failure.email=
		job.success.email=
		lockdown.create.projects=false
		cache.directory=cache
		# JMX stats
		jetty.connector.stats=true
		executor.connector.stats=true
		executor.port=12312
		# Azkaban mysql settings by default. Users should configure their own username and password.
		database.type=mysql
		mysql.port=3306
		mysql.host=node1
		mysql.database=azkaban
		mysql.user=root
		mysql.password=123456
		mysql.numconnections=100
		#Multiple Executor
		azkaban.use.multiple.executors=true			
		azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus
		azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1
		azkaban.executorselector.comparator.Memory=1
		azkaban.executorselector.comparator.LastDispatched=1
		azkaban.executorselector.comparator.CpuUsage=1

5.3 Azkaban executor服务器配置

进入/azkaban/azkaban-exec/conf目录下，编辑azkaban.properties文件
	vim azkaban.properties
		# Azkaban Personalization Settings
		azkaban.name=Test
		azkaban.label=My Local Azkaban
		azkaban.color=#FF3601
		azkaban.default.servlet.path=/index
		web.resource.dir=/azkaban/azkaban-web/web/
		default.timezone.id=Asia/Shanghai
		# Azkaban UserManager class
		user.manager.class=azkaban.user.XmlUserManager
		user.manager.xml.file=/azkaban/azkaban-web/conf/azkaban-users.xml
		# Loader for projects
		executor.global.properties=/azkaban/azkaban-web/conf/global.properties
		azkaban.project.dir=projects
		# Velocity dev mode
		velocity.dev.mode=false
		# Azkaban Jetty server properties.
		jetty.use.ssl=false
		jetty.maxThreads=25
		jetty.port=8081
		# Where the Azkaban web server is located
		azkaban.webserver.url=http://localhost:8081
		# mail settings
		mail.sender=
		mail.host=
		# User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users.
		# enduser -> myazkabanhost:443 -> proxy -> localhost:8081
		# when this parameters set then these parameters are used to generate email links.
		# if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.
		# azkaban.webserver.external_hostname=myazkabanhost.com
		# azkaban.webserver.external_ssl_port=443
		# azkaban.webserver.external_port=8081
		job.failure.email=
		job.success.email=
		lockdown.create.projects=false
		cache.directory=cache
		# JMX stats
		jetty.connector.stats=true
		executor.connector.stats=true
		# Azkaban plugin settings
		azkaban.jobtype.plugin.dir=/azkaban/azkaban-exec/plugins/jobtypes
		# Azkaban mysql settings by default. Users should configure their own username and password.
		database.type=mysql
		mysql.port=3306
		mysql.host=node1
		mysql.database=azkaban
		mysql.user=root
		mysql.password=123456
		mysql.numconnections=100
		# Azkaban Executor settings
		executor.maxThreads=50
		executor.flow.threads=30
		executor.port=12321

6.启动Azkaban
6.1 启动AzkabanExecutorServer

cd /azkaban/azkaban-exec/bin             
./start-exec.sh 

6.2 通过jps检查是否正常运行

jps
21373 AzkabanExecutorServer

6.3 激活AzkabanExecutor

浏览器：http://node1:12321/executor?action=activate

6.4 启动AzkabanWebServer

cd /azkaban/azkaban-web/bin
./start-web.sh

6.5 jps检查

21419 AzkabanWebServer
21373 AzkabanExecutorServer	                                                                                                                                                                                            

7.验证Azkaban运行情况

验证Azkaban是否启动成功，可以访问Azkaban的WebUI界面，检查是否启动成功。
在浏览器输入http://node1:8081，检查是否安装成功
账号密码默认：azkaban，在/azkaban/azkaban-web/conf/azkaban-users.xml 文件中

登录进入后