Spring Data Redis是spring基于Redis做的一些模块化功能,属于spring全家桶其中之一,本篇文章主要是讲其中关于数据缓存方面的实现+源码解读+优化。都知道Redis的其中一个很好的应用场景就是做数据缓存,在没研究Spring的这一块功能之前,我自己也写过基于redis做的缓存实现spring aop结合redis实现数据缓存,Spring对redis缓存的设计思想则更加精妙,更加全面,不得不说大佬终究还是大佬,Spring能够一统Java不是没有原因的。
首先须要了解几个具体类的概念:
RedisConnection:
提供了Redis通信的核心构建块,因为它处理与Redis后端通信。它还自动将底层连接库异常转换为Spring一致的DAO异常层次结构,这样您就可以在不更改任何代码的情况下切换连接器,因为操作语义保持不变。(翻译自官网解释,其实简单理解下,就是用来连接Redis的)
RedisConnectionFactory:
是构建RedisConnection的工厂类
RedisStandaloneConfiguration:
Redis标准配置,用来构建RedisConnectionFactory须要的必要参数,比如host;port;password
RedisSentinelConfiguration:
Redis哨兵模式的相关配置
RedisClusterConfiguration:
Redis集群模式的相关配置
RedisCacheManager:
顾名思义是Redis缓存管理器,是Spring基于Redis实现缓存的核心类
RedisCacheConfiguration:
是RedisCacheManager的具体个性化配置项
Key过期时间 | 默认不会过期,可通过entryTtl(Duration ttl)设置过期时间 |
---|---|
缓存null值 | 默认允许缓存null,可通过disableCachingNullValues()禁用此功能 |
Key前缀 |
默认有前缀,可通过disableKeyPrefix()禁用前缀 |
默认的前缀值 |
默认是@Cacheable注解的cacheNames属性的值,可通过prefixKeysWith(String prefix)覆盖 |
Key的序列化器 |
|
Value的序列化器 |
|
RedisCacheManagerBuilder:
是RedisCacheManager的构建器,内部维护了RedisConnectionFactory
RedisTemplate:
同样底层是维护了RedisConnectionFactory,在Spring Data Redis中和RedisCacheManager属于不同的Redis应用模块,RedisCacheManager负责Redis数据缓存的核心实现,RedisTemplate是Spring为方便操作Redis各种命令而封装出来的工具类,其功能类似于jedis,这里对其不做讨论
@Configuration
@EnableCaching//启用Spring Data Redis基于缓存的支持
class RedisConfiguration {
/**
* 构建JedisConnectionFactory
*/
@Bean
public JedisConnectionFactory redisConnectionFactory() {
RedisStandaloneConfiguration config = new RedisStandaloneConfiguration("server", 6379);
return new JedisConnectionFactory(config);
}
/**
* 构建RedisCacheManager
*/
@Bean
public RedisCacheManager cacheManager(RedisConnectionFactory connectionFactory) {
RedisCacheManager cm = RedisCacheManager.builder(connectionFactory)
.cacheDefaults(defaultCacheConfig())//设置默认的缓存配置
.withInitialCacheConfigurations(singletonMap("predefined",
defaultCacheConfig().disableCachingNullValues()))
.transactionAware()
.build();
}
/**
* 默认的缓存配置
*/
private RedisCacheConfiguration defaultCacheConfig(){
RedisCacheConfiguration config = RedisCacheConfiguration.defaultCacheConfig()
.entryTtl(Duration.ofSeconds(1))//缓存过期时间设为1s后过期
.disableCachingNullValues();//禁用缓存null
return config;
}
}
如果是spring boot只需要增加start依赖,上面的配置类boot会自动帮你配置好,配置参数在application.properties中指定
如果须要更多个性化的配置建议覆盖boot的相关配置,这里讲一个对Redis连接池的优化。
boot中的自动配置中对连接池的相关配置及其简单:
/**
* 这是Spring Boot自动注入JedisConnectionFactory的配置类
*
* @author Mark Paluch
* @author Stephane Nicoll
*/
@Configuration
@ConditionalOnClass({ GenericObjectPool.class, JedisConnection.class, Jedis.class })
class JedisConnectionConfiguration extends RedisConnectionConfiguration {
private final RedisProperties properties;
private final List builderCustomizers;
JedisConnectionConfiguration(RedisProperties properties,
ObjectProvider sentinelConfiguration,
ObjectProvider clusterConfiguration,
ObjectProvider> builderCustomizers) {
super(properties, sentinelConfiguration, clusterConfiguration);
this.properties = properties;
this.builderCustomizers = builderCustomizers
.getIfAvailable(Collections::emptyList);
}
/**
* RedisConnectionFactory.class不存在时触发
* @return
* @throws UnknownHostException
*/
@Bean
@ConditionalOnMissingBean(RedisConnectionFactory.class)
public JedisConnectionFactory redisConnectionFactory() throws UnknownHostException {
return createJedisConnectionFactory();
}
private JedisConnectionFactory createJedisConnectionFactory() {
JedisClientConfiguration clientConfiguration = getJedisClientConfiguration();
if (getSentinelConfig() != null) {
return new JedisConnectionFactory(getSentinelConfig(), clientConfiguration);
}
if (getClusterConfiguration() != null) {
return new JedisConnectionFactory(getClusterConfiguration(),
clientConfiguration);
}
return new JedisConnectionFactory(getStandaloneConfig(), clientConfiguration);
}
private JedisClientConfiguration getJedisClientConfiguration() {
JedisClientConfigurationBuilder builder = applyProperties(
JedisClientConfiguration.builder());
RedisProperties.Pool pool = this.properties.getJedis().getPool();
if (pool != null) {
applyPooling(pool, builder);
}
if (StringUtils.hasText(this.properties.getUrl())) {
customizeConfigurationFromUrl(builder);
}
customize(builder);
return builder.build();
}
private JedisClientConfigurationBuilder applyProperties(
JedisClientConfigurationBuilder builder) {
if (this.properties.isSsl()) {
builder.useSsl();
}
if (this.properties.getTimeout() != null) {
Duration timeout = this.properties.getTimeout();
builder.readTimeout(timeout).connectTimeout(timeout);
}
return builder;
}
private void applyPooling(RedisProperties.Pool pool,
JedisClientConfiguration.JedisClientConfigurationBuilder builder) {
builder.usePooling().poolConfig(jedisPoolConfig(pool));
}
/**
* 这里只对连接池做了比较简单的配置
* @param pool
* @return
*/
private JedisPoolConfig jedisPoolConfig(RedisProperties.Pool pool) {
JedisPoolConfig config = new JedisPoolConfig();
config.setMaxTotal(pool.getMaxActive());
config.setMaxIdle(pool.getMaxIdle());
config.setMinIdle(pool.getMinIdle());
if (pool.getMaxWait() != null) {
config.setMaxWaitMillis(pool.getMaxWait().toMillis());
}
return config;
}
private void customizeConfigurationFromUrl(
JedisClientConfiguration.JedisClientConfigurationBuilder builder) {
ConnectionInfo connectionInfo = parseUrl(this.properties.getUrl());
if (connectionInfo.isUseSsl()) {
builder.useSsl();
}
}
private void customize(
JedisClientConfiguration.JedisClientConfigurationBuilder builder) {
for (JedisClientConfigurationBuilderCustomizer customizer : this.builderCustomizers) {
customizer.customize(builder);
}
}
}
我们可以通过覆盖JedisConnectionFactory的方式配置我们自定义的连接池:
/**
* 自定义一个JedisConnectionFactory代替spring boot注入的JedisConnectionFactory
* @see org.springframework.boot.autoconfigure.data.redis.JedisConnectionConfiguration#redisConnectionFactory()
* 连接池配置代替boot默认的连接池配置
* @return
*/
@Bean
public JedisConnectionFactory redisConnectionFactory() {
return createJedisConnectionFactory();
}
private JedisConnectionFactory createJedisConnectionFactory() {
JedisClientConfiguration clientConfiguration = getJedisClientConfiguration();
return new JedisConnectionFactory(getStandaloneConfig(), clientConfiguration);
}
private RedisStandaloneConfiguration getStandaloneConfig() {
RedisStandaloneConfiguration config = new RedisStandaloneConfiguration();
config.setHostName(this.properties.getHost());
config.setPort(this.properties.getPort());
config.setPassword(RedisPassword.of(this.properties.getPassword()));
config.setDatabase(REDIS_DATA_BASE);
return config;
}
private JedisClientConfiguration getJedisClientConfiguration() {
JedisClientConfiguration.JedisClientConfigurationBuilder builder = applyProperties(
JedisClientConfiguration.builder());
RedisProperties.Pool pool = this.properties.getJedis().getPool();
if (pool != null) {
applyPooling(pool, builder);
}
return builder.build();
}
private JedisClientConfiguration.JedisClientConfigurationBuilder applyProperties(
JedisClientConfiguration.JedisClientConfigurationBuilder builder) {
if (this.properties.isSsl()) {
builder.useSsl();
}
if (this.properties.getTimeout() != null) {
Duration timeout = this.properties.getTimeout();
builder.readTimeout(timeout).connectTimeout(timeout);
}
return builder;
}
private void applyPooling(RedisProperties.Pool pool,
JedisClientConfiguration.JedisClientConfigurationBuilder builder) {
builder.usePooling().poolConfig(jedisPoolConfig(pool));
}
/**
* 这里自定义连接池配置
* @param pool
* @return
*/
private JedisPoolConfig jedisPoolConfig(RedisProperties.Pool pool) {
JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
jedisPoolConfig.setMaxTotal(100);
jedisPoolConfig.setMaxIdle(100);
jedisPoolConfig.setTestOnBorrow(true);
jedisPoolConfig.setMinIdle(8);
jedisPoolConfig.setMaxWaitMillis(10000L);
jedisPoolConfig.setTestOnReturn(true);
jedisPoolConfig.setTestWhileIdle(true);
jedisPoolConfig.setTimeBetweenEvictionRunsMillis(30000L);
jedisPoolConfig.setNumTestsPerEvictionRun(10);
jedisPoolConfig.setMinEvictableIdleTimeMillis(60000L);
jedisPoolConfig.setMaxTotal(pool.getMaxActive());
jedisPoolConfig.setMaxIdle(pool.getMaxIdle());
jedisPoolConfig.setMinIdle(pool.getMinIdle());
if (pool.getMaxWait() != null) {
jedisPoolConfig.setMaxWaitMillis(pool.getMaxWait().toMillis());
}
return jedisPoolConfig;
}
https://docs.spring.io/spring/docs/5.1.7.RELEASE/spring-framework-reference/integration.html#cache-annotations
其实这里有个很不方便的地方,通过entryTtl(Duration ttl)设置的缓存过期时间是全局性的,所有的缓存的存活时间都根据这个配置,这显然是不合理的,我们肯定希望对特定的缓存使用特定的过期时间设置,然而Spring官网这一块并没有提到。所以只能从源码入手研究。
spring底层通过封装多个RedisCache来实现不同场景下的缓存配置,其成员变量如下:
public class RedisCache extends AbstractValueAdaptingCache {
private final String name;
private final RedisCacheWriter cacheWriter;
private final RedisCacheConfiguration cacheConfig;
private final ConversionService conversionService;
}
name:标识不同的缓存,类似分组的概念。
RedisCacheWriter:是对Redis底层操作的封装,我们是通过RedisCache的相关接口来执行redis命令的,而RedisCache内部又是通过RedisCacheWriter来间接操作redis。
RedisCacheConfiguration:前面说RedisCacheConfiguration是RedisCacheManager的具体配置信息,更准确的说其实是对应RedisCache中的配置信息,因为RedisCacheManager内部维护了一个由若干个RedisCacheConfiguration组成的Map,在其初始化时会通过这个Map组装生成对应的RedisCache。
public class RedisCacheManager extends AbstractTransactionSupportingCacheManager {
private final RedisCacheWriter cacheWriter;
private final RedisCacheConfiguration defaultCacheConfig;
private final Map initialCacheConfiguration;
private final boolean allowInFlightCacheCreation;
/**
* 根据initialCacheConfiguration这个Map生成所有的RedisCache
* initialCacheConfiguration是通过外部的配置类中调用以下方法加载的
* @see RedisCacheManagerBuilder#withInitialCacheConfigurations(Map)
* @see RedisCacheManagerBuilder#withCacheConfiguration(String, RedisCacheConfiguration)
* 由它的抽象父类的初始化方法触发
* @see AbstractCacheManager#initializeCaches()
* (non-Javadoc)
* @see org.springframework.cache.support.AbstractCacheManager#loadCaches()
*/
@Override
protected Collection loadCaches() {
List caches = new LinkedList<>();
for (Map.Entry entry :
initialCacheConfiguration.entrySet()) {
caches.add(createRedisCache(entry.getKey(), entry.getValue()));
}
return caches;
}
/**
* 这里创建的时候如果不存在配置则会使用默认的缓存配置
*/
protected RedisCache createRedisCache(String name, @Nullable RedisCacheConfiguration
cacheConfig) {
return new RedisCache(name, cacheWriter, cacheConfig != null ? cacheConfig :
defaultCacheConfig);
}
}
其父类AbstractCacheManager的初始化方法中会调用子类实现的loadCaches方法,所以所有的RedisCache实例都存在父类AbstractCacheManager的成员变量cacheMap中,通过实现的getCache方法获取指定的RedisCache实例:
public abstract class AbstractCacheManager implements CacheManager, InitializingBean {
private final ConcurrentMap cacheMap = new ConcurrentHashMap(16);
private volatile Set cacheNames = Collections.emptySet();
public void initializeCaches() {
Collection caches = loadCaches();
synchronized (this.cacheMap) {
this.cacheNames = Collections.emptySet();
this.cacheMap.clear();
Set cacheNames = new LinkedHashSet<>(caches.size());
for (Cache cache : caches) {
String name = cache.getName();
this.cacheMap.put(name, decorateCache(cache));
cacheNames.add(name);
}
this.cacheNames = Collections.unmodifiableSet(cacheNames);
}
}
@Override
@Nullable
public Cache getCache(String name) {
Cache cache = this.cacheMap.get(name);
if (cache != null) {
return cache;
}
else {
// Fully synchronize now for missing cache creation...
synchronized (this.cacheMap) {
cache = this.cacheMap.get(name);
if (cache == null) {
cache = getMissingCache(name);
if (cache != null) {
cache = decorateCache(cache);
this.cacheMap.put(name, cache);
updateCacheNames(name);
}
}
return cache;
}
}
}
}
总结:RedisCacheManager和RedisCache是核心,前者是管理器,负责生成和获取RedisCache实例,后者封装了具体的redis操作。可以自定义RedisCache和RedisCacheManager,通过实现createRedisCache接口和getCache接口管理自定义的RedisCache实例。
事实上类似@Cacheable等注解中的cacheNames参数对应的就是RedisCache中的name属性,可以发现cacheNames属性可以支持多个cacheName,底层对应的是多个RedisCache,即可以生成多份缓存,由于每个RedisCache都对应其自己的RedisCacheConfiguration,因此没份缓存都对应其不同的个性化配置。事实上存储多份缓存的应用场景并不是很多,不过我们可以通过这个特性解决上面过期时间的问题。
@Bean
@Override
public CacheManager cacheManager() {
RedisCacheManager cm = RedisCacheManager.builder(redisConnectionFactory())
.cacheDefaults(defaultCacheConfig())
.transactionAware()
.withInitialCacheConfigurations(getRedisCacheConfigurationMap())
.build();
return cm;
}
/**
* 配置cacheName与RedisCacheConfiguration映射map
* 基于不同的cacheName使用不同的缓存配置项,主要是过期时间的配置项有所不同,未匹配到的cacheName会走默认的defaultCacheConfig配置项
* @see RedisCacheManager#loadCaches()
* NOTE: 在disableKeyPrefix()禁用key的前提下,当使用cacheNames时,不支持申明多个,只有最后一个cacheName生效
* @return
*/
private Map getRedisCacheConfigurationMap(){
Map cacheConfigurations = new HashMap<>();
RedisCacheConfiguration config = defaultCacheConfig().entryTtl(Duration.ofMinutes(5));
cacheConfigurations.put("CorrectionRange",config);
cacheConfigurations.put("CustomizeQuery",config);
cacheConfigurations.put("UserExpress",config);
RedisCacheConfiguration config10 = defaultCacheConfig().entryTtl(Duration.ofMinutes(10));
cacheConfigurations.put("UserExpressTemplate",config10);
return cacheConfigurations;
}
public RedisCacheConfiguration defaultCacheConfig(){
RedisCacheConfiguration config = RedisCacheConfiguration.defaultCacheConfig()
.serializeKeysWith(RedisSerializationContext.SerializationPair.fromSerializer(new StringRedisSerializer()))
.serializeValuesWith(RedisSerializationContext.SerializationPair.fromSerializer(new GenericFastJsonRedisSerializer()));
return config;
}
原理是通过withInitialCacheConfigurations方法将须要的RedisCacheConfiguration放到initialCacheConfiguration这个map中,其中key是默认的CacheName,通过给RedisCacheConfiguration指定不同的缓存过期时间限制可以达到上面的目的。
事实上Spring Data Redis的缓存实现依赖的是将一个CacheInterceptor的拦截器添加到bean的代理类中。CacheInterceptor中有所有配置的keyGenerator,cacheResolver,cacheManager等,其作为一个方法拦截器,负责对缓存方法的拦截,因此所有启用了缓存的接口都会经过这个拦截器(关于这一块的源码不再另行展开,网上有很多解读的帖子),那么是不是说我们可以自定义自己的拦截器做些特殊的配置呢?答案是肯定的,在spring的源码中有一个CustomInterceptorTests的单元测试类,专门讲了如何去自定义一个拦截器,有兴趣的朋友可以去看一下,这里贴出部分代码:
@Before
public void setup() {
this.ctx = new AnnotationConfigApplicationContext(EnableCachingConfig.class);
this.cs = ctx.getBean("service", CacheableService.class);
}
@After
public void tearDown() {
this.ctx.close();
}
@Test
public void onlyOneInterceptorIsAvailable() {
Map interceptors = this.ctx.getBeansOfType(CacheInterceptor.class);
//测试容器中是否只有一个拦截器
assertEquals("Only one interceptor should be defined", 1, interceptors.size());
CacheInterceptor interceptor = interceptors.values().iterator().next();
//测试这个拦截器是不是我们自定义实现的
assertEquals("Custom interceptor not defined", TestCacheInterceptor.class, interceptor.getClass());
}
@Configuration
@EnableCaching
static class EnableCachingConfig {
@Bean
public CacheManager cacheManager() {
return CacheTestUtils.createSimpleCacheManager("testCache", "primary", "secondary");
}
@Bean
public CacheableService service() {
return new DefaultCacheableService();
}
@Bean
public CacheInterceptor cacheInterceptor(CacheOperationSource cacheOperationSource) {
CacheInterceptor cacheInterceptor = new TestCacheInterceptor();
cacheInterceptor.setCacheManager(cacheManager());
cacheInterceptor.setCacheOperationSources(cacheOperationSource);
return cacheInterceptor;
}
}
/**
* A test {@link CacheInterceptor} that handles special exception
* types.
*/
@SuppressWarnings("serial")
static class TestCacheInterceptor extends CacheInterceptor {
@Override
protected Object invokeOperation(CacheOperationInvoker invoker) {
try {
return super.invokeOperation(invoker);
}
catch (CacheOperationInvoker.ThrowableWrapper e) {
Throwable original = e.getOriginal();
if (original.getClass() == UnsupportedOperationException.class) {
return 55L;
}
else {
throw new CacheOperationInvoker.ThrowableWrapper(
new RuntimeException("wrapping original", original));
}
}
}
}
这个单元测试是可以跑通的,因此我们可以这样做:
/**
* 配置自定义缓存拦截器,代替spring默认的拦截器
* @param cacheOperationSource
* @return
*/
@Bean
public CacheInterceptor cacheInterceptor(CacheOperationSource cacheOperationSource) {
CacheInterceptor cacheInterceptor = new CustomCacheInterceptor();
cacheInterceptor.setCacheManager(cacheManager());
cacheInterceptor.setCacheOperationSources(cacheOperationSource);
return cacheInterceptor;
}
/**
* 自定义缓存拦截器
* 通过redis开关来决定是否走缓存
* @author sunnyLu
*/
public class CustomCacheInterceptor extends CacheInterceptor {
@Override
public Object invoke(MethodInvocation invocation) throws Throwable {
String cache = RedisExecutor.get(RedisKeyHermes.CACHE_SWITCH);
if (null == cache || "0".equals(cache)){//不走缓存
return invocation.proceed();
} else {
return super.invoke(invocation);
}
}
}
不过这里如果项目是boot的话会有个坑,因为这里我们自定义的bean的name还是cacheInterceptor,spring boot项目默认遇到重名的bean是不会覆盖的,这个机制是通过DefaultListableBeanFactory的allowBeanDefinitionOverriding属性控制的,默认是true,所以传统spring项目这么配置没问题,但是boot不行(SpringApplication中也有个allowBeanDefinitionOverriding属性是false,会覆盖DefaultListableBeanFactory中的同名属性),因此须要将其设为true。
public static void main(String[] args) {
SpringApplication app = new SpringApplication(new Class[] { XX.class });
app.setAllowBeanDefinitionOverriding(true);
app.run(args);
//下面两种启动方式都不行
//SpringApplication.run(XX.class, args);
//new SpringApplicationBuilder(XX.class).initializers((GenericApplicationContext c) ->
//c.setAllowBeanDefinitionOverriding(true)).run(args);
}