多级缓存框架<一>

目录

一、概览

二、缓存封装

1. 依赖jar包

2. 定义配置

3. 自定义Cache实现类

4. 自定义redis缓存实现类

5. CacheManager管理Cache

6. 加载Redisson和Ehcache配置文件

三、缓存实战

1. 依赖cache-core包

2. 添加配置

3. Redisson和Ehcache配置文件

4. cacheName过期时间配置

5. 创建CacheManager的bean

6. @Cacheable使用

7. 测试


一、概览

        缓存是高并发下的必备技术之一。本章节使用ehcache作为一级缓存、redis集群作为二级缓存。如下图所示,获取缓存数据流程图。自己封装缓存jar包cache-core,redisson访问redis集群。

多级缓存框架<一>_第1张图片

        ehcache缓存是应用内存级别缓存,也是本地缓存。而redis集群为内存缓存,也是分布式缓存。DB数据存入磁盘,其查询速率远不及ehcache、redis,但DB是数据的源头。所以缓存中数据同步,即:及时更新缓存数据尤为重要。上图绿色部分”同步到redis“是为了降低数据延时同步。

         redis缓存中会出现"雪崩"、"穿透"、"击穿"。本章节也对此做了相应考虑,3种具体处理方式不在本节详细介绍。

        如下图所示是封装缓存的类图。

多级缓存框架<一>_第2张图片

二、缓存封装

1. 依赖jar包



    org.redisson
    redisson-spring-boot-starter
    3.11.6
    provided



    net.sf.ehcache
    ehcache
    2.9.1
    provided

2. 定义配置

cache-core:
  global:
    redisEnable: true        # redis缓存是否可用
    expiration: 120          # 全局过期时间(单位s)
    random-expiration: 60    # 全局过期随机时间(单位s),防止redis大量key同一时间过期
    null-expiration:
      enable: true           # null值缓存过期时间是否开启
      expiration: 5          # null值缓存过期时间,防止redis穿透
  redis:
    config-path: classpath:config\redisson-cluster-config.yml # redisson集群配置文件路径
  ehcache:
    config-path: config/ehcache.xml # ehcache配置文件路径

3. 自定义Cache实现类

RedisAndEhcacheCache类实现Cache接口,该类主要完成put、get、evict等具体方法。需注意:

a. 自定义MyRedisCache类:实现redis具体的put方法,尤其是设置过期随机时间设置

b. 获取缓存数据get方法:先获取ehcache、再获取redis集群

c. null缓存:put时,ehcache不缓存null,只有redis缓存null,防止缓存穿透

package com.common.cache.core.config;

import com.log.util.LogUtil;
import org.springframework.cache.Cache;
import org.springframework.cache.ehcache.EhCacheCache;
import org.springframework.data.redis.cache.RedisCache;

import java.util.concurrent.Callable;

/**
 * @description redis + ehcache的Cache - 核心方法
 * @author TCM
 * @version 1.0
 * @date 2021/7/10 14:01
 **/
public class RedisAndEhcacheCache implements Cache {

    // redis缓存
    private MyRedisCache myRedisCache;
    // ehcache缓存
    private EhCacheCache ehCacheCache;
    // redis缓存可用
    private boolean redisEnable;

    // 构造函数
    public RedisAndEhcacheCache(RedisCache redisCache, EhCacheCache ehCacheCache, boolean redisEnable){
        this.myRedisCache = (MyRedisCache) redisCache;
        this.ehCacheCache = ehCacheCache;
        this.redisEnable = redisEnable;
    }

    // redis是否可用
    private boolean checkRedisEnable() {
        return this.redisEnable;
    }

    @Override
    public String getName() {
        return this.myRedisCache.getName();
    }

    @Override
    public Object getNativeCache() {
        return null;
    }

    // 先获取ehcache缓存、再获取redis缓存
    @Override
    public ValueWrapper get(Object key) {
        // 获取ehcache缓存
        ValueWrapper value = this.ehCacheCache.get(key);
        if (value != null) {
            return value;
        }

        try {
            // redis是否可用
            if (checkRedisEnable()) {
                // 获取redis缓存
                value = this.myRedisCache.get(key);
                // redis数据缓存到ehcache
                if (null != value) {
                    this.ehCacheCache.put(key, value.get());
                }
            }
        } catch (Exception e) {
            LogUtil.error("cache error key: " + key, "RedisAndEhcacheCache");
        }
        return value;
    }

    @Override
    public  T get(Object key, Class type) {
        // 获取ehcache缓存
        Object value = this.ehCacheCache.get(key.toString());
        if (null != value) {
            return (T) value;
        }

        // redis是否可用
        if (checkRedisEnable()) {
            // 获取redis缓存
            value = this.myRedisCache.get(key.toString(), type);
            // redis数据缓存到ehcache
            if (null != value) {
                this.ehCacheCache.put(key, value);
                return (T) value;
            }
        }
        return null;
    }

    @Override
    public  T get(Object key, Callable valueLoader) {
        return (T) this.get(key.toString());
    }

    @Override
    public void put(Object key, Object value) {
        // 如果@Cacheable中null也缓存,则ehcache也会缓存null
        // 此处定义:null值缓存,只在redis中缓存
        // 缓存到ehcache
        if (value != null) {
            this.ehCacheCache.put(key, value);
        }

        // redis可用,则缓存到redis
        if (checkRedisEnable()) {
            this.myRedisCache.put(key, value);
        }
    }

    @Override
    public ValueWrapper putIfAbsent(Object key, Object value) {
        return null;
    }

    @Override
    public void evict(Object key) {
        // 缓存到ehcache
        this.ehCacheCache.evict(key);

        // redis可用,则缓存到redis
        if (checkRedisEnable()) {
            this.myRedisCache.evict(key);
        }
    }

    @Override
    public void clear() {
        this.ehCacheCache.clear();
    }

}

4. 自定义redis缓存实现类

MyRedisCache类处理具体的put方法,其他方法继承父类RedisCache,需注意:

a. 防止"穿透":过期时间为全局null过期时间配置

b. 防止"雪崩":cacheName下缓存时,加上随机过期时间,避免雪崩

d. CacheNameRandomExpirationConfig:存放cacheName与随机过期时间的映射关系

package com.common.cache.core.config;

import org.springframework.data.redis.cache.RedisCache;
import org.springframework.data.redis.cache.RedisCacheConfiguration;
import org.springframework.data.redis.cache.RedisCacheWriter;

import java.time.Duration;

/**
 * @description 处理构造函数 + 过期时间
 * @author TCM
 * @version 1.0
 * @date 2021/7/11 9:16
 **/
public class MyRedisCache extends RedisCache {

    private final String name;
    private final RedisCacheWriter cacheWriter;
    private final RedisCacheConfiguration cacheConfig;

    // null值缓存过期时间是否开启
    private final Boolean enableExpiration;
    // null值缓存过期时间,防止redis穿透
    private final Long nullExpiration;

    public MyRedisCache(String name, RedisCacheWriter cacheWriter, RedisCacheConfiguration cacheConfig,
                        Boolean enableExpiration, Long nullExpiration) {
        super(name, cacheWriter, cacheConfig);
        this.name = name;
        this.cacheWriter = cacheWriter;
        this.cacheConfig = cacheConfig;
        this.enableExpiration = enableExpiration;
        this.nullExpiration = nullExpiration;
    }

    /**
     * 覆写RedisCache中的put
     * 目的:可以在redis的cacheName下所有key添加随机过期时间,防止cacheName下大批量key同一时间失效
     */
    @Override
    public void put(Object key, Object value) {
        Object cacheValue = this.preProcessCacheValue(value);
        if (!this.isAllowNullValues() && cacheValue == null) {
            throw new IllegalArgumentException(String.format("Cache '%s' does not allow 'null' values. Avoid storing null via '@Cacheable(unless=\"#result == null\")' or configure RedisCache to allow 'null' via RedisCacheConfiguration.", this.name));
        } else {
            // 总过期时间
            long totalExpiration;

            // null过期时间开启
            if (enableExpiration && value == null) {
                totalExpiration = nullExpiration;
            } else {
                // 获取cacheName对应的随机过期时间配置
                long randomExpiration = CacheNameRandomExpirationConfig.getInstance().getConcurrentHashMap().get(this.name);
                // 总过期时间 = 过期时间 + 随机过期时间
                totalExpiration = this.cacheConfig.getTtl().getSeconds() + (int) (Math.random() * (randomExpiration + 1));
            }

            // 保存key
            this.cacheWriter.put(this.name, this.createAndConvertCacheKey(key), this.serializeCacheValue(cacheValue), Duration.ofSeconds(totalExpiration));
        }
    }

    // 参考父类RedisCache的方法
    private byte[] createAndConvertCacheKey(Object key) {
        return this.serializeCacheKey(this.createCacheKey(key));
    }

}
package com.common.cache.core.config;

import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

/**
 * @description redis的key的随机过期时间 —— 防止大量key同一时间失效
 * @author TCM
 * @version 1.0
 * @date 2021/8/4 21:57
 **/
public class CacheNameRandomExpirationConfig {

    // 存放redis的cacheName的随机过期时间
    private Map concurrentHashMap;
    // 单例
    private static CacheNameRandomExpirationConfig cacheNameRandomExpirationConfig;

    public CacheNameRandomExpirationConfig(){
        this.concurrentHashMap = new ConcurrentHashMap<>();
    }

    public static synchronized CacheNameRandomExpirationConfig getInstance() {
        if (cacheNameRandomExpirationConfig == null) {
            cacheNameRandomExpirationConfig = new CacheNameRandomExpirationConfig();
        }
        return cacheNameRandomExpirationConfig;
    }

    public Map getConcurrentHashMap() {
        return concurrentHashMap;
    }

}

5. CacheManager管理Cache

SimpleCacheManagerConfig类,创建CacheManagerBean来管理不同的cache对象

package com.common.cache.core.config;

import com.common.cache.core.redisson.FastJson2JsonRedisSerializer;
import net.sf.ehcache.CacheManager;
import org.redisson.spring.data.connection.RedissonConnectionFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cache.ehcache.EhCacheCache;
import org.springframework.cache.support.SimpleCacheManager;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.cache.RedisCache;
import org.springframework.data.redis.cache.RedisCacheConfiguration;
import org.springframework.data.redis.cache.RedisCacheWriter;
import org.springframework.data.redis.serializer.RedisSerializationContext;

import javax.annotation.Resource;
import java.time.Duration;
import java.util.ArrayList;
import java.util.Collection;

/**
 * @description SimpleCacheManager配置
 * @author tcm
 * @version 1.0.0
 * @date 2021/7/8 17:43
 **/
@Configuration
public class SimpleCacheManagerConfig {

    @Resource(name = "cacheRedissonConnectionFactory")
    private RedissonConnectionFactory redissonConnectionFactory;

    private final String redisKeyPrefix = "cache:";

    @Value("${cache-core.global.redisEnable}")
    private boolean redisEnable;
    @Value("${cache-core.global.expiration}")
    private Long expiration;
    @Value("${cache-core.global.random-expiration}")
    private Long randomExpiration;
    @Value("${cache-core.global.null-expiration.enable}")
    private Boolean enableExpiration;
    @Value("${cache-core.global.null-expiration.expiration}")
    private Long nullExpiration;
    @Value("${cache-core.ehcache.config-path:ehcache.xml}")
    private String configPath;

    /**
     * 设置过期时间创建SimpleCacheManager
     * @param cacheName ehcache名称
     * @param expiration redis过期时间
     * @return
     */
    protected SimpleCacheManager create(String cacheName, Long expiration, Long randomExpiration) {
        return createCacheManager(cacheName, expiration, randomExpiration);
    }

    /**
     * 默认过期时间创建SimpleCacheManager
     * @param cacheName ehcache名称
     * @return
     */
    protected SimpleCacheManager create(String cacheName) {
        return createCacheManager(cacheName, expiration, randomExpiration);
    }

    // 创建CacheManager
    private SimpleCacheManager createCacheManager(String cacheName, Long expiration, Long randomExpiration) {
        // 收集配置的cacheName对应的随机过期时间
        CacheNameRandomExpirationConfig.getInstance().getConcurrentHashMap().put(cacheName, randomExpiration);

        // 获取redis + ehcache对象
        RedisAndEhcacheCache redisAndEhcacheCache = getRedisAndEhcacheCache(cacheName, expiration);

        // cache集合
        Collection caches = new ArrayList<>();
        caches.add(redisAndEhcacheCache);

        // redis + ehcache缓存管理器
        SimpleCacheManager simpleCacheManager = new SimpleCacheManager();
        simpleCacheManager.setCaches(caches);

        return simpleCacheManager;
    }

    // 返回redis + ehcache对象
    private RedisAndEhcacheCache getRedisAndEhcacheCache(String cacheName, Long expiration){
        // redis缓存配置
        RedisCacheConfiguration redisCacheConfiguration = RedisCacheConfiguration.defaultCacheConfig()
                // 设置过期时间 = 过期时间 + 随机时间[0,randomExpiration],防止redis大量key同一时间过期
                .entryTtl(Duration.ofSeconds(expiration))
                // 序列化
                .serializeValuesWith(jackson2JsonRedisSerializer())
                // key前缀
                .prefixKeysWith(redisKeyPrefix);

        // 创建redisCache
        RedisCache redisCache = new MyRedisCache(cacheName, RedisCacheWriter.nonLockingRedisCacheWriter(redissonConnectionFactory), redisCacheConfiguration,
                enableExpiration, nullExpiration);

        // 创建ehCacheCache
        CacheManager ehcacheCacheManger = CacheEhcacheConfig.getInstance(configPath).getCacheManager();
        EhCacheCache ehCacheCache = new EhCacheCache(ehcacheCacheManger.getCache(cacheName));

        // 创建redis + ehcache
        return new RedisAndEhcacheCache(redisCache, ehCacheCache, redisEnable);
    }

    // 缓存时序列化value
    private RedisSerializationContext.SerializationPair jackson2JsonRedisSerializer(){
        FastJson2JsonRedisSerializer jackson2JsonRedisSerializer = new FastJson2JsonRedisSerializer<>(Object.class);
        return RedisSerializationContext
                .SerializationPair
                .fromSerializer(jackson2JsonRedisSerializer);
    }

}
 
  

6. 加载Redisson和Ehcache配置文件

注意:a. RedissonClusterConfig:加载redisson集群配置文件

           b. CacheEhcacheConfig:加载ehcache配置文件

           c. FastJson2JsonRedisSerializer:redis保存value时对象序列化

package com.common.cache.core.redisson;

import org.redisson.Redisson;
import org.redisson.api.RedissonClient;
import org.redisson.config.Config;
import org.redisson.spring.data.connection.RedissonConnectionFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.Resource;

import java.io.IOException;

/**
 * @description Redisson集群配置
 * @author tcm
 * @version 1.0.0
 * @date 2021/8/2 17:00
 **/
@Configuration
public class RedissonClusterConfig {

    @Value("${cache-core.redis.config-path:classpath:\\redisson-cluster-config.yml}")
    private Resource configFile;

    @Bean("cacheRedissonConnectionFactory")
    public RedissonConnectionFactory cacheRedissonConnectionFactory() throws IOException {
        // 加载配置文件
        Config config = Config.fromYAML(configFile.getInputStream());
        // 创建RedissonClient
        RedissonClient redissonClient = Redisson.create(config);
        // 创建RedissonConnectionFactory
        return new RedissonConnectionFactory(redissonClient);
    }

}
package com.common.cache.core.config;

import net.sf.ehcache.CacheManager;

import java.net.URL;

/**
 * @description Ehcache配置 - 单例模式
 * @author TCM
 * @version 1.0
 * @date 2021/7/11 10:16
 **/
public class CacheEhcacheConfig {

    // 获取ehcache配置文件路径
    public static String configPath2;

    // Ehcache缓存管理器
    private CacheManager cacheManager;
    private static CacheEhcacheConfig cacheEhcacheConfig;

    private CacheEhcacheConfig() {
        final String configPath = configPath2;
        URL url = CacheEhcacheConfig.class.getClassLoader().getResource(configPath);
        // 创建CacheManager
        this.cacheManager = CacheManager.create(url);
    }

    public static synchronized CacheEhcacheConfig getInstance(String configPath) {
        if (cacheEhcacheConfig == null) {
            configPath2 = configPath;
            cacheEhcacheConfig = new CacheEhcacheConfig();
        }
        return cacheEhcacheConfig;
    }

    public CacheManager getCacheManager() {
        return cacheManager;
    }

    public void setCacheManager(CacheManager cacheManager) {
        this.cacheManager = cacheManager;
    }

}
package com.common.cache.core.redisson;

import com.alibaba.fastjson.JSON;
import org.springframework.data.redis.serializer.RedisSerializer;
import org.springframework.data.redis.serializer.SerializationException;

import java.nio.charset.Charset;

/**
 * @description 自定义redis对象序列化
 * @author tcm
 * @version 1.0.0
 * @date 2021/5/26 17:59
 **/
public class FastJson2JsonRedisSerializer implements RedisSerializer {

    private static final Charset DEFAULT_CHARSET = Charset.forName("UTF-8");

    private Class clazz;

    public FastJson2JsonRedisSerializer(Class clazz) {
        super();
        this.clazz = clazz;
    }

    @Override
    public byte[] serialize(T t) throws SerializationException {
        if (t == null) {
            return new byte[0];
        }
        return JSON.toJSONString(t).getBytes(DEFAULT_CHARSET);
    }

    @Override
    public T deserialize(byte[] bytes) throws SerializationException {
        if (bytes == null || bytes.length <= 0) {
            return null;
        }
        String str = new String(bytes, DEFAULT_CHARSET);
        return (T) JSON.parseObject(str, clazz);
    }

}

三、缓存实战

1. 依赖cache-core包



     com.common
     cache-core
     1.0.0-RELEASE

2. 添加配置

cache-core:
  global:
    redisEnable: true        # redis缓存是否可用
    expiration: 120          # 全局过期时间(单位s)
    random-expiration: 60    # 全局过期随机时间(单位s),防止redis大量key同一时间过期
    null-expiration:
      enable: true           # null值缓存过期时间是否开启
      expiration: 10         # null值缓存过期时间,防止redis穿透
  redis:
    config-path: classpath:config\redisson-cluster-config.yml # redisson集群配置文件路径
  ehcache:
    config-path: config/ehcache.xml # ehcache配置文件路径

3. Redisson和Ehcache配置文件

Redis集群配置Redisson:config/redisson-cluster-config.yml

# Redisson集群配置
clusterServersConfig:
  idleConnectionTimeout: 10000
  pingTimeout: 1000
  connectTimeout: 10000
  timeout: 3000
  retryAttempts: 3
  retryInterval: 1500
  reconnectionTimeout: 3000
  failedAttempts: 3
  subscriptionsPerConnection: 5
  clientName: null
  loadBalancer: ! {}
  slaveSubscriptionConnectionMinimumIdleSize: 1
  slaveSubscriptionConnectionPoolSize: 50
  slaveConnectionMinimumIdleSize: 32
  slaveConnectionPoolSize: 64
  masterConnectionMinimumIdleSize: 32
  masterConnectionPoolSize: 64
  readMode: "SLAVE"
  password: "abcdef"
  nodeAddresses:
    - "redis://192.168.153.94:6382"
    - "redis://192.168.153.94:6383"
    - "redis://192.168.153.94:6384"
    - "redis://192.168.153.94:6385"
    - "redis://192.168.153.94:6386"
    - "redis://192.168.153.94:6387"
  scanInterval: 1000
threads: 0
nettyThreads: 0
codec: ! {}
"transportMode": "NIO"

Ehcache配置文件:config/ehcache.xml



    
    
    
    

    
    

    
    



4. cacheName过期时间配置

配置文件为:config/ehredis.properties

# cacheName配置
tabCache.cache.name=tabCache
# cacheName下key的过期时间配置
tabCache.redis.expiration=60
# cacheName下key的随机过期时间配置
tabCache.redis.random-expiration=120

tabCacheDao.cache.name=tabCacheDao
tabCacheDao.redis.expiration=600
tabCacheDao.redis.random-expiration=120

5. 创建CacheManager的bean

package com.common.instance.demo.config.cache;

import com.common.cache.core.config.SimpleCacheManagerConfig;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cache.support.SimpleCacheManager;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.context.annotation.PropertySource;

/**
 * @description SimpleCacheManager的bean管理
 * @author tcm
 * @version 1.0.0
 * @date 2021/7/9 15:14
 **/
@Configuration
@PropertySource({"classpath:/config/ehredis.properties"})
public class SimpleCacheManagerBean extends SimpleCacheManagerConfig {

    @Bean("tabCacheManager")
    @Primary
    public SimpleCacheManager tabCacheManagerBean(@Value("${tabCache.cache.name}") String cacheName,
                                     @Value("${tabCache.redis.expiration}") Long expiration,
                                     @Value("${tabCache.redis.random-expiration}") Long randomExpiration) {
        return super.create(cacheName, expiration, randomExpiration);
    }

    @Bean("tabCacheDaoManager")
    public SimpleCacheManager tabCacheDaoManagerBean(@Value("${tabCacheDao.cache.name}") String cacheName,
                                     @Value("${tabCacheDao.redis.expiration}") Long expiration,
                                     @Value("${tabCacheDao.redis.random-expiration}") Long randomExpiration) {
        return super.create(cacheName, expiration, randomExpiration);
    }

}

6. @Cacheable使用

注意:a. value:对应ehredis.properties下的xxx.cache.name的配置,见缓存实战第三步

           b. cacheManager:对应CacheManager的bean名,见缓存实战第四步

           c. unless = "#result eq null":值为null时,根本没有走put方法

@Cacheable(value = {"tabCache"}, key = "'tabCache:' + #tabId", cacheManager = "tabCacheManager", unless = "#result eq null")
public List testRedisCacheAble(String tabId) {
    // 组装查询参数
    WcPendantTab tab = new WcPendantTab();
    tab.setTabId(tabId);

    // 查询
    List tabs = wcPendantTabDao.queryAll(tab);

    return tabs;
}

7. 测试

注意:a. 前缀"cache":是所有key的前缀,见缓存封装第5步

多级缓存框架<一>_第3张图片

你可能感兴趣的:(系统框架,redis,缓存,redis,ehcache)