深入理解XXL-JOB执行器注册原理

一、执行器注册到调度中心

执行器相当于一个应用服务,需要注册到调度中心这样才能由调度中心统一管理。有点像SpringCloud微服务中的Eureka或者Nacos注册中心,xxl-job没有选择他们,而是直接用调度中心作为注册中心。

1.1 XxlJobConfig 执行器配置

执行器自动注册应当Spring在初始化相关bean的时候完成自动注册,因此从XxlJobConfig这个配置类开始了解。

在此新增XxlJobSpringExecutor执行器,这个执行器会在Spring Bean容器中进行管理,和普通bean的生命周期一致。

@Configuration
@Slf4j
public class XxlJobConfig {
    //调度中心地址
    @Value("${xxl.job.admin.addresses}")
    private String adminAddresses;

    //访问token
    @Value("${xxl.job.accessToken}")
    private String accessToken;

    //执行器应用名
    @Value("${xxl.job.executor.appname}")
    private String appname;

  
    @Value("${xxl.job.executor.address}")
    private String address;

    @Value("${xxl.job.executor.ip}")
    private String ip;

    @Value("${xxl.job.executor.port}")
    private int port;

    //日志路径
    @Value("${xxl.job.executor.logpath}")
    private String logPath;

    //日志保存天数
    @Value("${xxl.job.executor.logretentiondays}")
    private int logRetentionDays;

    @Bean
    public XxlJobSpringExecutor xxlJobExecutor() {
        log.info(">>>>>>>>>>> xxl-job config init.");
        XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
        xxlJobSpringExecutor.setAdminAddresses(adminAddresses);
        xxlJobSpringExecutor.setAppname(appname);
        xxlJobSpringExecutor.setAddress(address);;
        xxlJobSpringExecutor.setIp(ip);
        xxlJobSpringExecutor.setPort(port);
        xxlJobSpringExecutor.setAccessToken(accessToken);
        xxlJobSpringExecutor.setLogPath(logPath);
        xxlJobSpringExecutor.setLogRetentionDays(logRetentionDays);

        return xxlJobSpringExecutor;
    }
}

2.2 XxlJobSpringExecutor 执行器

XxlJobSpringExecutor 继承了 XxlJobExecutor 基础类,实现了ApplicationContextAware,SmartInitializingSingleton,DisposableBean等接口。

  • XxlJobExecutor : 执行器的基本实现,定义了执行器的启动和停止方法,包含生成调度中心代理集合,初始化回调线程,初始化执行器内置容器等。
  • ApplicationContextAware:在Spring生命周期中,ApplicationContextAware是容器刷新的钩子函数,通过此接口可以获取ApplicationContext的应用上下文
  • SmartInitializingSingleton:这个接口可以在初始化单例bean进行一些后置操作,因此将bean注册到Spring容器时,会调用afterSingletonsInstantiated方法进行一些初始化操作
  • DisposableBean:在bean被容器销毁时会调用的钩子函数,进行一些销毁操作

在旧版本中,执行器需要继承IJobhandler,实现execute方法,在新版本中只需要在方法中添加@xxljob注解即可,因此一个类可以注册多个任务,具体实现是在initJobHandlerMethodRepository方法中执行的。initJobHandlerMethodRepository的执行流程如下:

  1. 从applicationContext中获取所有有注解的bean
  2. 获取每个bean的所有的带有XxlJob注解的方法,被XxlJob注解修饰的的方法是一个任务
  3. 遍历所有的任务
  4. 注册任务处理器,registJobHandler调用的是父类XxlJobExecutor的registJobHandler
/**
 * xxl-job executor (for spring)
 *
 * @author xuxueli 2018-11-01 09:24:52
 */
public class XxlJobSpringExecutor extends XxlJobExecutor implements ApplicationContextAware, SmartInitializingSingleton, DisposableBean {
    private static final Logger logger = LoggerFactory.getLogger(XxlJobSpringExecutor.class);


    // start
    @Override
    public void afterSingletonsInstantiated() {

        // init JobHandler Repository
        /*initJobHandlerRepository(applicationContext);*/

        // init JobHandler Repository (for method)
        // 初始化任务处理器仓库 (针对于方法处理器)
        initJobHandlerMethodRepository(applicationContext);

        // refresh GlueFactory
        GlueFactory.refreshInstance(1);

        // super start
        try {
            // 调用 XxlJobExecutor 的start方法
            super.start();
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    // destroy
    @Override
    public void destroy() {
        super.destroy();
    }


    /*private void initJobHandlerRepository(ApplicationContext applicationContext) {
        if (applicationContext == null) {
            return;
        }

        // init job handler action
        Map serviceBeanMap = applicationContext.getBeansWithAnnotation(JobHandler.class);

        if (serviceBeanMap != null && serviceBeanMap.size() > 0) {
            for (Object serviceBean : serviceBeanMap.values()) {
                if (serviceBean instanceof IJobHandler) {
                    String name = serviceBean.getClass().getAnnotation(JobHandler.class).value();
                    IJobHandler handler = (IJobHandler) serviceBean;
                    if (loadJobHandler(name) != null) {
                        throw new RuntimeException("xxl-job jobhandler[" + name + "] naming conflicts.");
                    }
                    registJobHandler(name, handler);
                }
            }
        }
    }*/

    private void initJobHandlerMethodRepository(ApplicationContext applicationContext) {
        if (applicationContext == null) {
            return;
        }
        // init job handler from method
         // 1. 从applicationContext中获取所有有注解的bean
        String[] beanDefinitionNames = applicationContext.getBeanNamesForType(Object.class, false, true);
        for (String beanDefinitionName : beanDefinitionNames) {
            Object bean = applicationContext.getBean(beanDefinitionName);

            // 2. 获取每个bean的所有的带有XxlJob注解的方法,被XxlJob注解修饰的的方法是一个任务
            Map<Method, XxlJob> annotatedMethods = null;   // referred to :org.springframework.context.event.EventListenerMethodProcessor.processBean
            try {
                annotatedMethods = MethodIntrospector.selectMethods(bean.getClass(),
                        new MethodIntrospector.MetadataLookup<XxlJob>() {
                            @Override
                            public XxlJob inspect(Method method) {
                                return AnnotatedElementUtils.findMergedAnnotation(method, XxlJob.class);
                            }
                        });
            } catch (Throwable ex) {
                logger.error("xxl-job method-jobhandler resolve error for bean[" + beanDefinitionName + "].", ex);
            }
            if (annotatedMethods==null || annotatedMethods.isEmpty()) {
                continue;
            }
            // 3.遍历所有的任务
            for (Map.Entry<Method, XxlJob> methodXxlJobEntry : annotatedMethods.entrySet()) {
                Method executeMethod = methodXxlJobEntry.getKey();
                XxlJob xxlJob = methodXxlJobEntry.getValue();
                if (xxlJob == null) {
                    continue;
                }

                String name = xxlJob.value();
                if (name.trim().length() == 0) {
                    throw new RuntimeException("xxl-job method-jobhandler name invalid, for[" + bean.getClass() + "#" + executeMethod.getName() + "] .");
                }
                if (loadJobHandler(name) != null) {
                    throw new RuntimeException("xxl-job jobhandler[" + name + "] naming conflicts.");
                }

                // execute method
                /*if (!(method.getParameterTypes().length == 1 && method.getParameterTypes()[0].isAssignableFrom(String.class))) {
                    throw new RuntimeException("xxl-job method-jobhandler param-classtype invalid, for[" + bean.getClass() + "#" + method.getName() + "] , " +
                            "The correct method format like " public ReturnT execute(String param) " .");
                }
                if (!method.getReturnType().isAssignableFrom(ReturnT.class)) {
                    throw new RuntimeException("xxl-job method-jobhandler return-classtype invalid, for[" + bean.getClass() + "#" + method.getName() + "] , " +
                            "The correct method format like " public ReturnT execute(String param) " .");
                }*/

                executeMethod.setAccessible(true);

                // init and destory
                Method initMethod = null;
                Method destroyMethod = null;

                if (xxlJob.init().trim().length() > 0) {
                    try {
                        initMethod = bean.getClass().getDeclaredMethod(xxlJob.init());
                        initMethod.setAccessible(true);
                    } catch (NoSuchMethodException e) {
                        throw new RuntimeException("xxl-job method-jobhandler initMethod invalid, for[" + bean.getClass() + "#" + executeMethod.getName() + "] .");
                    }
                }
                if (xxlJob.destroy().trim().length() > 0) {
                    try {
                        destroyMethod = bean.getClass().getDeclaredMethod(xxlJob.destroy());
                        destroyMethod.setAccessible(true);
                    } catch (NoSuchMethodException e) {
                        throw new RuntimeException("xxl-job method-jobhandler destroyMethod invalid, for[" + bean.getClass() + "#" + executeMethod.getName() + "] .");
                    }
                }

                // registry jobhandler
                // 4. 注册任务处理器
                registJobHandler(name, new MethodJobHandler(bean, executeMethod, initMethod, destroyMethod));
            }
        }

    }

    // ---------------------- applicationContext ----------------------
    private static ApplicationContext applicationContext;

    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        this.applicationContext = applicationContext;
    }

    public static ApplicationContext getApplicationContext() {
        return applicationContext;
    }

}

2.3 MethodJobHandler 方法代理

MethodJobHandler 继承了 IJobHandler,通过反射的方式调用目标对象的execute、initMethod、destroyMethod操作。

/**
* @author xuxueli 2019-12-11 21:12:18
*/
public class MethodJobHandler extends IJobHandler {

   // 被代理的目标对象
   private final Object target;
   // 任务执行方法
   private final Method method;
   // 初始化方法
   private Method initMethod;
   // 销毁方法
   private Method destroyMethod;

   public MethodJobHandler(Object target, Method method, Method initMethod, Method destroyMethod) {
       this.target = target;
       this.method = method;

       this.initMethod = initMethod;
       this.destroyMethod = destroyMethod;
   }

   @Override
   public void execute() throws Exception {
       // 通过反射方法调用
       Class<?>[] paramTypes = method.getParameterTypes();
       if (paramTypes.length > 0) {
           method.invoke(target, new Object[paramTypes.length]);       // method-param can not be primitive-types
       } else {
           method.invoke(target);
       }
   }

   @Override
   public void init() throws Exception {
       if(initMethod != null) {
           initMethod.invoke(target);
       }
   }

   @Override
   public void destroy() throws Exception {
       if(destroyMethod != null) {
           destroyMethod.invoke(target);
       }
   }

   @Override
   public String toString() {
       return super.toString()+"["+ target.getClass() + "#" + method.getName() +"]";
   }
}

2.4 XxlJobExecutor注册

XxlJobExecutor 的 registJobHandler 方法就是将 每个jobHandler的名字和MethodJobHandler 以key-value的形式放在一个线程安全的map里面,缓存在内存中。

// ---------------------- job handler repository ----------------------
private static ConcurrentMap<String, IJobHandler> jobHandlerRepository = new ConcurrentHashMap<String, IJobHandler>();
public static IJobHandler loadJobHandler(String name){
    return jobHandlerRepository.get(name);
}
public static IJobHandler registJobHandler(String name, IJobHandler jobHandler){
    logger.info(">>>>>>>>>>> xxl-job register jobhandler success, name:{}, jobHandler:{}", name, jobHandler);
    return jobHandlerRepository.put(name, jobHandler);
}

2.5 XxlJobExecutor 初始化

XxlJobSpringExecutor 中会调用 XxlJobExecutor 中的start方法,start方法主要做了以下内容:

  1. 初始化日志路径
  2. 初始化admin调度中心代理,通过配置的adminAdress,为每个调度中心生成一个AdminBiz代理类,并放到adminBizList中作为调度中心集合
  3. 初始化日志文件清理线程,这个清理线程会在后台定期清理那些过期的日志文件
  4. 初始化触发回调线程
  5. 初始化执行器内置容器

关于注册相关代码如下:

AdminBiz 定义了回调,注册,移除注册等方法,定义了执行器和调度中心的交互方式

  • 执行器启动调用注册接口注册到调度中心
  • 调度中心回调执行器的任务方法
  • 执行器退出,通知调度中心移除注册信息

initAdminBizList 方法就是识别配置的调度中心地址,生成AdminBizClient代理类,并用一个List集合进行缓存。

// ---------------------- admin-client (rpc invoker) ----------------------
private static List<AdminBiz> adminBizList;
private void initAdminBizList(String adminAddresses, String accessToken) throws Exception {
    if (adminAddresses!=null && adminAddresses.trim().length()>0) {
        for (String address: adminAddresses.trim().split(",")) {
            if (address!=null && address.trim().length()>0) {

                AdminBiz adminBiz = new AdminBizClient(address.trim(), accessToken);

                if (adminBizList == null) {
                    adminBizList = new ArrayList<AdminBiz>();
                }
                adminBizList.add(adminBiz);
            }
        }
    }
}
public static List<AdminBiz> getAdminBizList(){
    return adminBizList;
}

重点来了,什么时候调用adminBiz的registry方法呢。在XxlJobExecutor的initEmbedServer 方法中创建了内置容器,EmbedServer的start方法又会启动容器,并且会开始注册执行器。

// ---------------------- executor-server (rpc provider) ----------------------
private EmbedServer embedServer = null;

private void initEmbedServer(String address, String ip, int port, String appname, String accessToken) throws Exception {

    // fill ip port
    port = port>0?port: NetUtil.findAvailablePort(9999);
    ip = (ip!=null&&ip.trim().length()>0)?ip: IpUtil.getIp();

    // generate address
    if (address==null || address.trim().length()==0) {
        String ip_port_address = IpUtil.getIpPort(ip, port);   // registry-address:default use address to registry , otherwise use ip:port if address is null
        address = "http://{ip_port}/".replace("{ip_port}", ip_port_address);
    }

    // accessToken
    if (accessToken==null || accessToken.trim().length()==0) {
        logger.warn(">>>>>>>>>>> xxl-job accessToken is empty. To ensure system security, please set the accessToken.");
    }

    // start
    embedServer = new EmbedServer();
    embedServer.start(address, port, appname, accessToken);
}

在这里只贴出 EmbedServer 中和注册相关的部分代码,ExecutorRegistryThread 是一个执行器注册线程

// ---------------------- registry ----------------------

public void startRegistry(final String appname, final String address) {
    // start registry
    ExecutorRegistryThread.getInstance().start(appname, address);
}

public void stopRegistry() {
    // stop registry
    ExecutorRegistryThread.getInstance().toStop();
}

2.6 ExecutorRegistryThread 执行器注册线程

ExecutorRegistryThread 采用懒加载的方式实现单利模式,并且启动一个后台线程,每隔30s的心跳时间,不断地将执行器注册到调度中心。并在内置容器停止后,会调用取消注册方法

public class ExecutorRegistryThread {
    private static Logger logger = LoggerFactory.getLogger(ExecutorRegistryThread.class);

    private static ExecutorRegistryThread instance = new ExecutorRegistryThread();
    public static ExecutorRegistryThread getInstance(){
        return instance;
    }

    private Thread registryThread;
    private volatile boolean toStop = false;
    public void start(final String appname, final String address){

        // valid
        if (appname==null || appname.trim().length()==0) {
            logger.warn(">>>>>>>>>>> xxl-job, executor registry config fail, appname is null.");
            return;
        }
        if (XxlJobExecutor.getAdminBizList() == null) {
            logger.warn(">>>>>>>>>>> xxl-job, executor registry config fail, adminAddresses is null.");
            return;
        }

        registryThread = new Thread(new Runnable() {
            @Override
            public void run() {

                // registry
                while (!toStop) {
                    try {
                        RegistryParam registryParam = new RegistryParam(RegistryConfig.RegistType.EXECUTOR.name(), appname, address);
                        for (AdminBiz adminBiz: XxlJobExecutor.getAdminBizList()) {
                            try {
                                //进行注册
                                ReturnT<String> registryResult = adminBiz.registry(registryParam);
                                if (registryResult!=null && ReturnT.SUCCESS_CODE == registryResult.getCode()) {
                                    registryResult = ReturnT.SUCCESS;
                                    logger.debug(">>>>>>>>>>> xxl-job registry success, registryParam:{}, registryResult:{}", new Object[]{registryParam, registryResult});
                                    break;
                                } else {
                                    logger.info(">>>>>>>>>>> xxl-job registry fail, registryParam:{}, registryResult:{}", new Object[]{registryParam, registryResult});
                                }
                            } catch (Exception e) {
                                logger.info(">>>>>>>>>>> xxl-job registry error, registryParam:{}", registryParam, e);
                            }

                        }
                    } catch (Exception e) {
                        if (!toStop) {
                            logger.error(e.getMessage(), e);
                        }

                    }

                    try {
                        if (!toStop) {
                            TimeUnit.SECONDS.sleep(RegistryConfig.BEAT_TIMEOUT);
                        }
                    } catch (InterruptedException e) {
                        if (!toStop) {
                            logger.warn(">>>>>>>>>>> xxl-job, executor registry thread interrupted, error msg:{}", e.getMessage());
                        }
                    }
                }

                // registry remove
                try {
                    RegistryParam registryParam = new RegistryParam(RegistryConfig.RegistType.EXECUTOR.name(), appname, address);
                    for (AdminBiz adminBiz: XxlJobExecutor.getAdminBizList()) {
                        try {
                            ReturnT<String> registryResult = adminBiz.registryRemove(registryParam);
                            if (registryResult!=null && ReturnT.SUCCESS_CODE == registryResult.getCode()) {
                                registryResult = ReturnT.SUCCESS;
                                logger.info(">>>>>>>>>>> xxl-job registry-remove success, registryParam:{}, registryResult:{}", new Object[]{registryParam, registryResult});
                                break;
                            } else {
                                logger.info(">>>>>>>>>>> xxl-job registry-remove fail, registryParam:{}, registryResult:{}", new Object[]{registryParam, registryResult});
                            }
                        } catch (Exception e) {
                            if (!toStop) {
                                logger.info(">>>>>>>>>>> xxl-job registry-remove error, registryParam:{}", registryParam, e);
                            }

                        }

                    }
                } catch (Exception e) {
                    if (!toStop) {
                        logger.error(e.getMessage(), e);
                    }
                }
                logger.info(">>>>>>>>>>> xxl-job, executor registry thread destory.");

            }
        });
        registryThread.setDaemon(true);
        registryThread.setName("xxl-job, executor ExecutorRegistryThread");
        registryThread.start();
    }

    public void toStop() {
        toStop = true;

        // interrupt and wait
        if (registryThread != null) {
            registryThread.interrupt();
            try {
                registryThread.join();
            } catch (InterruptedException e) {
                logger.error(e.getMessage(), e);
            }
        }

    }

}

2.7 总结

执行器注册到调度中心的流程如下:

  1. XxlJobConfig 配置类向Spring bean容器中注册了XxlJobSpringExecutor,XxlJobSpringExecutor这个bean实现了SmartInitializingSingleton接口,会在实例化bean后进行初始化操作。
  2. XxlJobSpringExecutor 会调用 XxlJobExecutor 的 start 方法,进行一些初始化操作。其中包括将配置的调度中心包装成一个AdminBiz代理类,这个代理类包含回调、注册、取消注册等方法,执行器就是通过这个代理类进行注册的。
  3. XxlJobExecutor 的 start 方法还会初始化一个内置容器,这个内置容器在启动时会启动一个ExecutorRegistryThread后台注册线程,以默认30s的心跳时间,不断的通过AdminBiz代理向调度中心注册当前执行器的信息。

二、调度中心管理注册信息

在上一节中讲述了执行器注册到调度中心的流程,最终是通过AdminBiz代理来向调度中心注册。xxl-job-core中采用内部的xxl-rpc来实现远程方法调用,执行器作为一个客户端,通过XxlJobRemotingUtil向调度中心发起远程调用,接口路由是api/registry

com.xxl.job.core.biz.client.AdminBizClient#registry

@Override
public ReturnT<String> registry(RegistryParam registryParam) {
    return XxlJobRemotingUtil.postBody(addressUrl + "api/registry", accessToken, timeout, registryParam, String.class);
}

2.1 JobApiController 远程方法

执行器注册最终会调用到xxl-job-admin的JobApiController暴露的Http接口,如果是回调、注册、注册移除等操作则会调用AdminBiz的调度中心端实现 AdminBizImpl。

/**
 * Created by xuxueli on 17/5/10.
 */
@Controller
@RequestMapping("/api")
public class JobApiController {

    @Resource
    private AdminBiz adminBiz;

    /**
     * api
     *
     * @param uri
     * @param data
     * @return
     */
    @RequestMapping("/{uri}")
    @ResponseBody
    @PermissionLimit(limit=false)
    public ReturnT<String> api(HttpServletRequest request, @PathVariable("uri") String uri, @RequestBody(required = false) String data) {

        // valid
        if (!"POST".equalsIgnoreCase(request.getMethod())) {
            return new ReturnT<String>(ReturnT.FAIL_CODE, "invalid request, HttpMethod not support.");
        }
        if (uri==null || uri.trim().length()==0) {
            return new ReturnT<String>(ReturnT.FAIL_CODE, "invalid request, uri-mapping empty.");
        }
        if (XxlJobAdminConfig.getAdminConfig().getAccessToken()!=null
                && XxlJobAdminConfig.getAdminConfig().getAccessToken().trim().length()>0
                && !XxlJobAdminConfig.getAdminConfig().getAccessToken().equals(request.getHeader(XxlJobRemotingUtil.XXL_JOB_ACCESS_TOKEN))) {
            return new ReturnT<String>(ReturnT.FAIL_CODE, "The access token is wrong.");
        }
        
        // 如果是调用对应的回调,注册或者注册移除
        // services mapping
        if ("callback".equals(uri)) {
            List<HandleCallbackParam> callbackParamList = GsonTool.fromJson(data, List.class, HandleCallbackParam.class);
            return adminBiz.callback(callbackParamList);
        } else if ("registry".equals(uri)) {
            RegistryParam registryParam = GsonTool.fromJson(data, RegistryParam.class);
            return adminBiz.registry(registryParam);
        } else if ("registryRemove".equals(uri)) {
            RegistryParam registryParam = GsonTool.fromJson(data, RegistryParam.class);
            return adminBiz.registryRemove(registryParam);
        } else {
            return new ReturnT<String>(ReturnT.FAIL_CODE, "invalid request, uri-mapping("+ uri +") not found.");
        }

    }

}

2.2 AdminBizImpl

AdminBizImpl的registry会使用JobRegistryHelper来实现注册信息的管理。

@Service
public class AdminBizImpl implements AdminBiz {


    @Override
    public ReturnT<String> callback(List<HandleCallbackParam> callbackParamList) {
        return JobCompleteHelper.getInstance().callback(callbackParamList);
    }

    @Override
    public ReturnT<String> registry(RegistryParam registryParam) {
        // 注册
        return JobRegistryHelper.getInstance().registry(registryParam);
    }

    @Override
    public ReturnT<String> registryRemove(RegistryParam registryParam) {
        return JobRegistryHelper.getInstance().registryRemove(registryParam);
    }

}

2.3 JobRegistryHelper 注册

public ReturnT<String> registry(RegistryParam registryParam) {
   // 调用基础的参数
   // valid
   if (!StringUtils.hasText(registryParam.getRegistryGroup())
         || !StringUtils.hasText(registryParam.getRegistryKey())
         || !StringUtils.hasText(registryParam.getRegistryValue())) {
      return new ReturnT<String>(ReturnT.FAIL_CODE, "Illegal Argument.");
   }

   // 异步注册
   // async execute
   registryOrRemoveThreadPool.execute(new Runnable() {
      @Override
      public void run() {
         // 将注册信息保存在xxl_job_registry表中
         int ret = XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().registryUpdate(registryParam.getRegistryGroup(), registryParam.getRegistryKey(), registryParam.getRegistryValue(), new Date());
         if (ret < 1) {
            XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().registrySave(registryParam.getRegistryGroup(), registryParam.getRegistryKey(), registryParam.getRegistryValue(), new Date());

            // fresh
            freshGroupRegistryInfo(registryParam);
         }
      }
   });

   return ReturnT.SUCCESS;
}

xxl_job_registry表结构如下,主要存储注册的执行器以及对应的执行器地址,以及最新的心跳续期时间。

CREATE TABLE `xxl_job_registry` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `registry_group` varchar(50) NOT NULL,
  `registry_key` varchar(255) NOT NULL,
  `registry_value` varchar(255) NOT NULL,
  `update_time` datetime DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `i_g_k_v` (`registry_group`,`registry_key`,`registry_value`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

2.4 JobRegistryHelper 注册信息更新

com.xxl.job.admin.core.thread.JobRegistryHelper#start 会启动注册线程池,并会开启一个监控线程,扫描那些已经失效的注册信息并进行清理。start方法中的监控线程主要做了以下内容:

  1. 获取自动注册的执行器
  2. 根据注册信息的更新时间判断是否超过了续期超时时间3*30s,清除已经失效的注册信息
  3. 更新每个执行器的注册信息列表,修改xxl_job_group的注册列表字段
public void start(){

   // for registry or remove
   registryOrRemoveThreadPool = new ThreadPoolExecutor(
         2,
         10,
         30L,
         TimeUnit.SECONDS,
         new LinkedBlockingQueue<Runnable>(2000),
         new ThreadFactory() {
            @Override
            public Thread newThread(Runnable r) {
               return new Thread(r, "xxl-job, admin JobRegistryMonitorHelper-registryOrRemoveThreadPool-" + r.hashCode());
            }
         },
         new RejectedExecutionHandler() {
            @Override
            public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
               r.run();
               logger.warn(">>>>>>>>>>> xxl-job, registry or remove too fast, match threadpool rejected handler(run now).");
            }
         });

   // for monitor
   registryMonitorThread = new Thread(new Runnable() {
      @Override
      public void run() {
         while (!toStop) {
            try {
               // 1. 获取自动注册的执行器
               // auto registry group
               List<XxlJobGroup> groupList = XxlJobAdminConfig.getAdminConfig().getXxlJobGroupDao().findByAddressType(0);
               if (groupList!=null && !groupList.isEmpty()) {

                  // 2. 根据注册信息的更新时间判断是否超过了续期超时时间3*30s,清除已经失效的注册信息
                  // remove dead address (admin/executor)
                  List<Integer> ids = XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().findDead(RegistryConfig.DEAD_TIMEOUT, new Date());
                  if (ids!=null && ids.size()>0) {
                     XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().removeDead(ids);
                  }

                  // 3. 更新每个应用注册的信息,以appName为key,List为value缓存起来
                  // fresh online address (admin/executor)
                  HashMap<String, List<String>> appAddressMap = new HashMap<String, List<String>>();
                  List<XxlJobRegistry> list = XxlJobAdminConfig.getAdminConfig().getXxlJobRegistryDao().findAll(RegistryConfig.DEAD_TIMEOUT, new Date());
                  if (list != null) {
                     for (XxlJobRegistry item: list) {
                        if (RegistryConfig.RegistType.EXECUTOR.name().equals(item.getRegistryGroup())) {
                           String appname = item.getRegistryKey();
                           List<String> registryList = appAddressMap.get(appname);
                           if (registryList == null) {
                              registryList = new ArrayList<String>();
                           }

                           if (!registryList.contains(item.getRegistryValue())) {
                              registryList.add(item.getRegistryValue());
                           }
                           appAddressMap.put(appname, registryList);
                        }
                     }
                  }

                  // 4. 更新每个app的注册信息,修改xxl_job_group的注册列表字段
                  // fresh group address
                  for (XxlJobGroup group: groupList) {
                     List<String> registryList = appAddressMap.get(group.getAppname());
                     String addressListStr = null;
                     if (registryList!=null && !registryList.isEmpty()) {
                        Collections.sort(registryList);
                        StringBuilder addressListSB = new StringBuilder();
                        for (String item:registryList) {
                           addressListSB.append(item).append(",");
                        }
                        addressListStr = addressListSB.toString();
                        addressListStr = addressListStr.substring(0, addressListStr.length()-1);
                     }
                     group.setAddressList(addressListStr);
                     group.setUpdateTime(new Date());

                     XxlJobAdminConfig.getAdminConfig().getXxlJobGroupDao().update(group);
                  }
               }
            } catch (Exception e) {
               if (!toStop) {
                  logger.error(">>>>>>>>>>> xxl-job, job registry monitor thread error:{}", e);
               }
            }
            try {
               TimeUnit.SECONDS.sleep(RegistryConfig.BEAT_TIMEOUT);
            } catch (InterruptedException e) {
               if (!toStop) {
                  logger.error(">>>>>>>>>>> xxl-job, job registry monitor thread error:{}", e);
               }
            }
         }
         logger.info(">>>>>>>>>>> xxl-job, job registry monitor thread stop");
      }
   });
   registryMonitorThread.setDaemon(true);
   registryMonitorThread.setName("xxl-job, admin JobRegistryMonitorHelper-registryMonitorThread");
   registryMonitorThread.start();
}

public void toStop(){
   toStop = true;

   // stop registryOrRemoveThreadPool
   registryOrRemoveThreadPool.shutdownNow();

   // stop monitir (interrupt and wait)
   registryMonitorThread.interrupt();
   try {
      registryMonitorThread.join();
   } catch (InterruptedException e) {
      logger.error(e.getMessage(), e);
   }
}

2.5 总结

1、调度中心对外暴露一个api HTTP接口,执行器在注册的时候调用这个接口进行信息的注册

2、调度中心将注册信息保存在xxl_job_registry表中,该表只要存储执行器以及对应的地址信息。

3、调度中心JobRegistry会启动一个后台监控线程,及时从注册表中清理那些已经失效的注册信息,并同步到执行器表中的注册列表里面

转载:https://juejin.cn/post/7068635930705264647

你可能感兴趣的:(中间件,java,微服务,spring,cloud)