ExpiringMap 来过期 元素，防止频繁更新数据库， 缓存雪崩

先介绍下 ExpiringMap 一个高性能、低开销、零依赖、线程安全的ConcurrentMap实现，它可以使单独元素过期。并且监听过期。
git 地址：https://github.com/jhalterman/expiringmap

先介绍下项目功能，功能很简单， 利用阿里 canal 来监控数据库变动 推送MQ，异步任务 多个线程循环 拉取MQ 消息 来 处理业务逻辑。
在1.0版本中 每一个企业的 联系人都会操作一次数据库， 大大增加数据库压力， 与MQ 压力(这里为何会增加MQ 压力， 由于每个联系人都会更新数据库消费速度，很慢 赶不上生产速度，导致消息大量积压，资源有限，积压几百万MQ 就快蹦了， 见笑， 服务器资源有限)。怎么办呢， 要考虑 相同企业 联系人在特定情况下只更新一次数据 。
找到问题， 就开干， 开始考虑是redis的布隆过滤， 考虑到需要重启生产redis 更改redis配置，而且可能会导致大量IO。最后选择 ExpiringMap。经过测试，完全OJBK。
不多说直接 上关键代码 ：

package com.aspire.eab.async.framework.listener;

import com.aspire.eab.async.business.biz.IContactBusiness;
import com.aspire.eab.async.business.biz.IDepartmentBusiness;
import com.aspire.eab.async.business.biz.IEnterpriseBusiness;
import com.aspire.eab.async.framework.constant.AsyncConstants;
import com.aspire.eab.async.framework.param.AsyncTaskParam;
import net.jodah.expiringmap.ExpirationPolicy;
import net.jodah.expiringmap.ExpiringMap;
import org.redisson.api.RMapCache;
import org.redisson.api.RedissonClient;
import org.redisson.api.map.event.EntryEvent;
import org.redisson.api.map.event.EntryExpiredListener;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.stereotype.Component;
import org.springframework.util.StringUtils;

import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.LongAdder;

@Component
public class ExpiringMapCacheListener {

    private final Logger logger = LoggerFactory.getLogger(this.getClass());

    @Autowired
    private IDepartmentBusiness departmentBusiness;
    @Autowired
    private  IEnterpriseBusiness enterpriseBusiness;
    @Autowired
    private  IContactBusiness contactBusiness;
    @Autowired
    private AsyncTaskParam taskParam;

    ExpiringMap<String, LongAdder> getStringLongAdderExpiringMap() {
        return ExpiringMap.builder()
                .variableExpiration().expirationListener((theKey, theValue) -> {
                    logger.info("key {}执行过期监听器=======",theKey);
                    String[] str = theKey.toString().split("-");
                    String entId = str[1];
                    String depId = str[2];
                    this.updateEmployees(entId,depId);
                })
                .expiration(taskParam.getExpirationTime(), TimeUnit.SECONDS)
                .expirationPolicy(ExpirationPolicy.CREATED)
                .build();
    }
    private void updateEmployees(String enterpriseId, String departmentId) {
        //非0部门才去修改
        if (!AsyncConstants.DEP_ROOT_ID.equals(departmentId)) {
            logger.info("processUpdateEmployees->修改企业:[{}]部门[{}]人数", enterpriseId, departmentId);
            departmentBusiness.updateCurrentDeptEmployees(enterpriseId, departmentId);
        }
        int employees = contactBusiness.getEntEmployees(enterpriseId);
        enterpriseBusiness.updateCurrentEntEmployees(enterpriseId, employees);
    }
}

监听过期更新。

   String mapKey = AsyncConstants.EAB_CONTACT_COUNT +"-"+ enterpriseId + "-" + departmentId;
        if(employeesMap.containsKey(mapKey)){
            LongAdder adder = employeesMap.get(mapKey);
            adder.increment();
        }else{
            LongAdder adder = new LongAdder();
            adder.increment();
            employeesMap.putIfAbsent(mapKey,adder);
         }
        LongAdder adder = employeesMap.get(mapKey);
        if(null == adder){
            return false;
        }
        if(taskParam.getMaxEmployeesCount() > adder.intValue()){
            logger.info("expiringMapFilter 跳过>>>>>企业{}部门{}，跳过更新人数指标为：{}",enterpriseId,departmentId,adder.intValue());
            return true;
        }
        if (logger.isInfoEnabled()){
            logger.info("企业 {}，部门：{}达到人数更新指标 {}",enterpriseId,departmentId,adder.intValue());
        }
        //超过指标直接 remove
        employeesMap.remove(mapKey);
        return false;

这里设置一个 指标只要在当前线程 没有超过 配置的数值，都跳过更新DB， 过期 再去更新DB 这样， 大大减少操作数据库的次数。

上一下任务执行线程

public class TaskFetchThread extends BaseThread {

    private final Logger logger = LoggerFactory.getLogger(this.getClass());

    private ThreadPoolExecutor threadPool;

    private AsyncTaskParam taskParam;

    private List<ITaskExecuteStragety> stragetyList;

    private Map<TaskType, List<ITaskExecuteStragety>> stragetyMap = new HashMap<TaskType, List<ITaskExecuteStragety>>();

    private static final String MSG_LOCAL_CACHE = "MSG_LOCAL_CACHE";

    private static final String THREADPOOL = "THREADPOOL";
        
    private Set<String> deniedStragetyNameSet = new HashSet<>();

    public TaskFetchThread(BlockingQueue<AsyncTask<JSONObject>> localMsgCache, ThreadPoolExecutor threadPool,
                           AsyncTaskParam taskParam, List<ITaskExecuteStragety> stragetyList,ExpiringMap<String,LongAdder> employeesMap) {
        this.localMsgCache = localMsgCache;
        this.threadPool = threadPool;
        this.taskParam = taskParam;
        this.stragetyList = stragetyList;
        this.employeesMap = employeesMap;
        this.initCheckMap(Arrays.asList(MSG_LOCAL_CACHE, THREADPOOL));
        this.initDeniedStragetyNameSet();
    }

    @Override
    public void run() {
//    boolean loop = true;

        while (loop) {
            if (!monitorFlag) {
                ThreadSleep.sleep(1000 * 60);
                continue;
            }

            if (localMsgCache.size() == 0) {
                this.printCheckLog(MSG_LOCAL_CACHE, "MSG LOCAL CACHE is empty..", 1000 * 60);
                this.sleep(10);
                continue;
            }

            if (this.isPoolBusy()) {
                this.printCheckLog(THREADPOOL, "Threadpool is busy..", 100);
                this.sleep(10);
                continue;
            }

            AsyncTask<JSONObject> task = localMsgCache.poll();
            if (task == null) {
                continue;
            }

            List<ITaskExecuteStragety> stragetyRList = this.getStragety(task);
            
            if(stragetyRList.isEmpty()) {
              //任务未找到对应的执行策略，无法执行
              logger.error("Corresponding ITaskExecuteStragety has not been founded, tasks will be discarded." + task.toString());
              continue;
            }
            for (ITaskExecuteStragety stragety : stragetyRList) {
              if("EmployeesContactUpdateStragety".equals(stragety.getStragetyName())){
                  task.setEmployeesMap(this.employeesMap);
              }
              threadPool.execute(new TaskExecuteWrapper(task, stragety));
            }

        }

    }

此线程 根据 拉取MQ 线程的数据 ，然后给定 指定的策略 来处理业务。

关键点在于：

 ExpiringMap<String, LongAdder> getStringLongAdderExpiringMap() {
        return ExpiringMap.builder()
                .variableExpiration().expirationListener((theKey, theValue) -> {
                    logger.info("key {}执行过期监听器=======",theKey);
                    String[] str = theKey.toString().split("-");
                    String entId = str[1];
                    String depId = str[2];
                    this.updateEmployees(entId,depId);
                })
                .expiration(taskParam.getExpirationTime(), TimeUnit.SECONDS)
                .expirationPolicy(ExpirationPolicy.CREATED)
                .build();
    }

ExpiringMap 此处配合监听器 expirationListener ，
expiration(taskParam.getExpirationTime(), TimeUnit.SECONDS) 设置 过期时间，与单位。
expirationPolicy(ExpirationPolicy.CREATED) 设置过期策略。
更多 ExpiringMap 功能请参考 git 的test