本文基于 Elastic-Job V2.1.5 版本分享
1. 概述
本文主要分享 Elastic-Job-Cloud 作业配置。
如果你阅读过以下文章,有助于对本文的理解:
另外,笔者假设你已经对 《Elastic-Job-Lite 源码分析系列》 有一定的了解。
2. 云作业App
首先,理解下 云作业App 的定义:
简单来说,一个云作业App可以理解成由多个作业打在一起的 jar。
2.1 云作业App配置类
CloudAppConfiguration,云作业App配置。实现代码如下:
public final class CloudAppConfiguration {
/**
* 应用名
*/
private final String appName;
/**
* 应用包地址
*/
private final String appURL;
/**
* 应用启动脚本
*/
private final String bootstrapScript;
/**
* cpu 数量
*/
private double cpuCount = 1;
/**
* 内存 大小
*/
private double memoryMB = 128;
/**
* 每次执行作业时是否从缓存中读取应用。禁用则每次执行任务均从应用仓库下载应用至本地
*/
private boolean appCacheEnable = true;
/**
* 常驻作业事件采样率统计条数,默认不采样全部记录。
* 为避免数据量过大,可对频繁调度的常驻作业配置采样率,即作业每执行N次,才会记录作业执行及追踪相关数据
*/
private int eventTraceSamplingCount;
}
2.2 操作云作业App配置
云作业App配置有多种操作:
有两种方式进行操作,以添加举例子:
curl -l -H "Content-type: application/json" -X POST -d '{"appName":"foo_app","appURL":"http://app_host:8080/yourJobs.gz","cpuCount":0.1,"memoryMB":64.0,"bootstrapScript":"bin/start.sh","appCacheEnable":true,"eventTraceSamplingCount":0}' http://elastic_job_cloud_host:8899/api/app
// CloudAppRestfulApi
@Path("/app")
public final class CloudAppRestfulApi {
/**
* 注册应用配置.
*
* @param appConfig 应用配置
*/
@POST
@Consumes(MediaType.APPLICATION_JSON)
public void register(final CloudAppConfiguration appConfig) {
Optional appConfigFromZk = appConfigService.load(appConfig.getAppName());
if (appConfigFromZk.isPresent()) {
throw new AppConfigurationException("app '%s' already existed.", appConfig.getAppName());
}
appConfigService.add(appConfig);
}
}
// CloudAppConfigurationService.java
/**
* 添加云作业APP配置.
*
* @param appConfig 云作业App配置对象
*/
public void add(final CloudAppConfiguration appConfig) {
regCenter.persist(CloudAppConfigurationNode.getRootNodePath(appConfig.getAppName()), CloudAppConfigurationGsonFactory.toJson(appConfig));
}
// CloudAppConfigurationNode.JAVA
public final class CloudAppConfigurationNode {
public static final String ROOT = "/config/app";
private static final String APP_CONFIG = ROOT + "/%s"; // %s = ${APP_NAME}
}
[zk: localhost:2181(CONNECTED) 1] get /elastic-job-cloud/config/app/exampleApp
{"appName":"exampleApp","appURL":"http://785j8w.com1.z0.glb.clouddn.com/elastic-job-example-cloud-2.1.5.tar.gz","bootstrapScript":"bin/start.sh","cpuCount":1.0,"memoryMB":128.0,"appCacheEnable":true,"eventTraceSamplingCount":0}
3. 云作业
一个云作业应用可以包含一个或多个云作业。云作业有两种作业配置:云作业配置、本地云作业配置。下面来分别分享它们。
3.1 云作业配置
CloudJobConfiguration,云作业配置。实现代码如下:
public final class CloudJobConfiguration implements JobRootConfiguration {
/**
* 作业应用名称 {@link com.dangdang.ddframe.job.cloud.scheduler.config.app.CloudAppConfiguration}
*/
private final String appName;
/**
* 作业类型配置
*/
private final JobTypeConfiguration typeConfig;
/**
* 单片作业所需要的CPU数量,最小值为0.001
*/
private final double cpuCount;
/**
* 单片作业所需要的内存MB,最小值为1
*/
private final double memoryMB;
/**
* 作业执行类型
*/
private final CloudJobExecutionType jobExecutionType;
/**
* Spring容器中配置的bean名称
*/
private String beanName;
/**
* Spring方式配置Spring配置文件相对路径以及名称,如:META-INF\applicationContext.xml
*/
private String applicationContext;
3.1.1 操作云作业配置
云作业配置有多种操作:
有两种方式进行操作,以添加举例子:
// Java启动方式作业注册
curl -l -H "Content-type: application/json" -X POST -d '{"jobName":"foo_job","appName":"foo_app","jobClass":"yourJobClass","jobType":"SIMPLE","jobExecutionType":"TRANSIENT","cron":"0/5 * * * * ?","shardingTotalCount":5,"cpuCount":0.1,"memoryMB":64.0,"appURL":"http://app_host:8080/foo-job.tar.gz","failover":true,"misfire":true,"bootstrapScript":"bin/start.sh"}' http://elastic_job_cloud_host:8899/api/job/register
// Spring启动方式作业注册
curl -l -H "Content-type: application/json" -X POST -d '{"jobName":"foo_job","jobClass":"yourJobClass","beanName":"yourBeanName","applicationContext":"applicationContext.xml","jobType":"SIMPLE","jobExecutionType":"TRANSIENT","cron":"0/5 * * * * ?","shardingTotalCount":5,"cpuCount":0.1,"memoryMB":64.0,"appURL":"http://file_host:8080/foo-job.tar.gz","failover":false,"misfire":true,"bootstrapScript":"bin/start.sh"}' http://elastic_job_cloud_masterhost:8899/api/job/register
// CloudJobRestfulApi.java
public final class CloudJobRestfulApi {
/**
* 注册作业.
*
* @param jobConfig 作业配置
*/
@POST
@Path("/register")
@Consumes(MediaType.APPLICATION_JSON)
public void register(final CloudJobConfiguration jobConfig) {
producerManager.register(jobConfig);
}
}
// ProducerManager.java
public final class ProducerManager {
/**
* 注册作业.
*
* @param jobConfig 作业配置
*/
public void register(final CloudJobConfiguration jobConfig) {
if (disableJobService.isDisabled(jobConfig.getJobName())) {
throw new JobConfigurationException("Job '%s' has been disable.", jobConfig.getJobName());
}
Optional appConfigFromZk = appConfigService.load(jobConfig.getAppName());
if (!appConfigFromZk.isPresent()) {
throw new AppConfigurationException("Register app '%s' firstly.", jobConfig.getAppName());
}
Optional jobConfigFromZk = configService.load(jobConfig.getJobName());
if (jobConfigFromZk.isPresent()) {
throw new JobConfigurationException("Job '%s' already existed.", jobConfig.getJobName());
}
// 添加云作业配置
configService.add(jobConfig);
// 调度作业
schedule(jobConfig);
}
}
// CloudJobConfigurationService.java
/**
* 添加云作业配置.
*
* @param jobConfig 云作业配置对象
*/
public void add(final CloudJobConfiguration jobConfig) {
regCenter.persist(CloudJobConfigurationNode.getRootNodePath(jobConfig.getJobName()), CloudJobConfigurationGsonFactory.toJson(jobConfig));
}
// CloudJobConfigurationNode.java
public final class CloudJobConfigurationNode {
public static final String ROOT = "/config/job";
private static final String JOB_CONFIG = ROOT + "/%s"; // %s = ${JOB_NAME}
}
[zk: localhost:2181(CONNECTED) 3] get /elastic-job-cloud/config/job/test_job_simple
{"jobName":"test_job_simple","jobClass":"com.dangdang.ddframe.job.example.job.simple.JavaSimpleJob","jobType":"SIMPLE","cron":"0/10 * * * * ?","shardingTotalCount":1,"shardingItemParameters":"","jobParameter":"","failover":false,"misfire":false,"description":"","jobProperties":{"job_exception_handler":"com.dangdang.ddframe.job.executor.handler.impl.DefaultJobExceptionHandler","executor_service_handler":"com.dangdang.ddframe.job.executor.handler.impl.DefaultExecutorServiceHandler"},"appName":"exampleApp","cpuCount":0.1,"memoryMB":64.0,"jobExecutionType":"TRANSIENT"}
3.2 本地云作业配置
LocalCloudJobConfiguration,本地云作业配置。实现代码如下:
public final class LocalCloudJobConfiguration implements JobRootConfiguration {
private final JobTypeConfiguration typeConfig;
/**
* 分片作业序号
*/
private final int shardingItem;
private String beanName;
private String applicationContext;
}
到底有什么用呢?
在开发Elastic-Job-Cloud作业时,开发人员可以脱离Mesos环境,在本地运行和调试作业。可以利用本地运行模式充分的调试业务功能以及单元测试,完成之后再部署至Mesos集群。
本地运行作业无需安装Mesos环境。
3.3 云作业配置总结
在此我向大家推荐一个架构学习交流群。交流学习群号:993070439 里面会分享一些资深架构师录制的视频录像:有Spring,MyBatis,Netty源码分析,高并发、高性能、分布式、微服务架构的原理,JVM性能优化、分布式架构等这些成为架构师必备的知识体系,还能领取免费的学习资源。