azkaban集群多节点模式配置

配置多节点执行服务器的时候，需要在AzkabanWebServer的配置文件azkaban.properties里添加

azkaban.use.multiple.executors=true
azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus
azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1
azkaban.executorselector.comparator.Memory=1
azkaban.executorselector.comparator.LastDispatched=1
azkaban.executorselector.comparator.CpuUsage=1

以确认使用的是分布式方式，随后提交的job会根据情况自行选择执行服务器，否则默认只使用本地执行服务器，切记！

之后需要在mysql的azkaban库中添加各个执行服务器的ip/域名和端口：

insert into executors(host,port) values("EXECUTOR_PORT",EXECUTOR_PORT);

至此，azkaban的分布式配置才算成功。

配置多节点执行服务器的时候，需要在AzkabanWebServer的配置文件azkaban.properties里添加

azkaban.use.multiple.executors=true
azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus
azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1
azkaban.executorselector.comparator.Memory=1
azkaban.executorselector.comparator.LastDispatched=1
azkaban.executorselector.comparator.CpuUsage=1

以确认使用的是分布式方式，随后提交的job会根据情况自行选择执行服务器，否则默认只使用本地执行服务器，切记！

之后需要在mysql的azkaban库中添加各个执行服务器的ip/域名和端口：

insert into executors(host,port) values("EXECUTOR_PORT",EXECUTOR_PORT);

至此，azkaban的分布式配置才算成功。

以下来源官网文档：  http://azkaban.github.io/azkaban/docs/latest/

Configuring AzabanWebServer and AzkabanExecutorServer clients

The Executor server needs to be setup with a port, and the AzabanWebServer will need to know what this port is.

The following properties need to be set on AzkabanExecutorServer's azkaban.properties.

# Azkaban Executor settings
executor.maxThreads=50
executor.port=12321
executor.flow.threads=30

Single Executor Mode

By default the executor.port is set to 12321. The AzkabanWebServer will have to point to this port as well.

This is done by setting the following property in AzkabanWebServer's azkaban.properties.

executor.port=12321

Multiple Executor Mode

We need to enable multiple executor mode in webserver configuration if we want to run in multiple executor mode. Please verify that you have following properties in azkaban.properties. azkaban.use.multiple.executors and azkaban.executorselector.comparator.* are required properties. Please note that azkaban.use.multiple.executors is not honoured in multiple exeutor mode.

azkaban.use.multiple.executors=true
azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus
azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1
azkaban.executorselector.comparator.Memory=1
azkaban.executorselector.comparator.LastDispatched=1
azkaban.executorselector.comparator.CpuUsage=1

These changes are only picked up after restarting the servers.

Configure executors for Multiple Executor Mode

We don't have a executor management UI at this point of Time. Executors need to be configured in database. For examples:-

Insert all executors into the mysql DB for executor setups. Verify that correct executors are active in executors table.insert into executors(host,port) values("EXECUTOR_PORT",EXECUTOR_PORT);