SpringBoot2+Redis动态切换db数据源(db)最佳实践

动态DB切换代码已开源,有用star一下
https://github.com/it235/knife4j-redis-lettuce

需求

在使用Redis的时候,默认是16个库,非常小的项目默认0库就够了,但是对于体量稍微大一些的项目,需要将其他各个库充分利用,比如:

  • db0存公用的热点缓存
  • db1存商品服务的缓存
  • db2存订单相关的缓存
  • db3存库存相关的缓存

这个时候我们就需要实现多库切换进行操作,接下来我们看看如何在SpringBoot中实现多DB的切换动作。

实现

lettuce默认实现(只能取1个db)

  • 添加Maven依赖(版本默认采用当前parent的)
    <dependencies>
            <dependency>
                <groupId>com.fasterxml.jackson.coregroupId>
                <artifactId>jackson-databindartifactId>
            dependency>
            <dependency>
                <groupId>org.apache.commonsgroupId>
                <artifactId>commons-pool2artifactId>
            dependency>
            <dependency>
                <groupId>org.springframework.bootgroupId>
                <artifactId>spring-boot-starter-data-redis-reactiveartifactId>
            dependency>
        dependencies>
    
  • yml配置
    spring:
      redis:
        host: 192.168.0.245
        password: 123456
        port: 6379
        database: 1 # 使用库 1
        timeout: 60s
        lettuce: # lettuce基于netty,线程安全,支持并发
          pool:
            max-active: 8
            max-wait: -1ms
            max-idle: 2
            min-idle: 0
    
  • Configuration配置
    
    import com.alibaba.fastjson.parser.ParserConfig;
    import org.springframework.cache.annotation.EnableCaching;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.Configuration;
    import org.springframework.data.redis.connection.RedisConnectionFactory;
    import org.springframework.data.redis.core.RedisTemplate;
    import org.springframework.data.redis.serializer.StringRedisSerializer;
    
    @EnableCaching
    @Configuration
    public class FastJsonRedisConfig {
    
        @Bean
        public RedisTemplate<String,Object> redisTemplate(RedisConnectionFactory factory){
            RedisTemplate<String,Object> template = new RedisTemplate <>();
            template.setConnectionFactory(factory);
            Jackson2JsonRedisSerializer<Object> jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer<>();
            ObjectMapper om = new ObjectMapper();
            om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
            om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
            jackson2JsonRedisSerializer.setObjectMapper(om);
            StringRedisSerializer stringRedisSerializer = new StringRedisSerializer();
            // key采用String的序列化方式
            template.setKeySerializer(stringRedisSerializer);
            // hash的key也采用String的序列化方式
            template.setHashKeySerializer(stringRedisSerializer);
            // value序列化方式采用jackson
            template.setValueSerializer(jackson2JsonRedisSerializer);
            // hash的value序列化方式采用jackson
            template.setHashValueSerializer(jackson2JsonRedisSerializer);
            template.afterPropertiesSet();
            return template;
        }
    }
    

以上我们就可以注入restTemplate进行redis的相关操作了,这样做只能满足最基本的取db0的操作,如果要使用到其他库,接下来我们看第二种方案

动态切换DB

源码:https://github.com/it235/knife4j-redis-lettuce

思路

如果要动态切换DB,我们需要考虑以下几个问题:

  • DB个数该如何指定?比如项目只采用0-5库
  • DB切换时直接用一个redisTemplate是否可以?
  • DB1切换到DB2后,其他线程如果要继续使用DB1该如何选取?
  • 工具类该如何封装?

构思:接下来我构思来解决上面几个问题

  1. DB个数指定,我们能否通过yml配置文件指定个数,比如:[0,1,2,3,4,5,6]这样
  2. DB切换时redisTemplate一个显然可以,代码如下,但不是特别合适,因为database的指定是在connectionFactory.setDatabase(num);这里完成,所以我们需要操作连接工厂,通过最上面实现的代码可以看到,factory是由redisTemplate去指定的template.setConnectionFactory(factory);,如果我们采用一个redisTemplate的方式,那么我们要不停的进行resetConnection的方式。
    	//不推荐使用该方式
    	@Autowired
        private StringRedisTemplate redisTemplate;
     
        public void setDataBase(int num) {
            LettuceConnectionFactory connectionFactory = (LettuceConnectionFactory) redisTemplate.getConnectionFactory();
            if (connectionFactory != null && num != connectionFactory.getDatabase()) {
            	//切换DB
                connectionFactory.setDatabase(num);
                //是否允许多个线程操作共用同一个缓存连接,默认 true,false 时每个操作都将开辟新的连接
                connectionFactory.setShareNativeConnection(false);
                this.redisTemplate.setConnectionFactory(connectionFactory);
                connectionFactory.resetConnection();
            }
        }
    

这里我们最合适的方式,就是为每个库建立一个单独的redisTemplate,然后通过一个Map维护,需要时告诉我具体的dbNum,我来根据你想要的从池子中取出进行操作即可,这样也不会涉及到db的频繁切换

redisTemplate难点

  1. 如何通过factory创建多个?
  2. 如何维护Bean的生命周期?yml如何规划
  3. redis工具类该如何定义?

动手实现

  • 第一步,定义理想中的yml
    # 此处Key由spring改为自己定义的
    knife4j:
      redis:
        # 是否采用json序列化方式,若不采用jackson序列化
        jsonSerialType: 'Fastjson'
        host: localhost
        password: knife
        port: 6379
        databases: [0,1,2,3,4,5,6] # 要使用的库,会根据此处填写的库生成redisTemplate
        timeout: 60s
        lettuce: # lettuce基于netty,线程安全,支持并发
          pool:
            max-active: 50
            max-wait: -1ms
            max-idle: 8
            min-idle: 0
    
  • 第二步,读取yml,并创建redisTemplate注入到Spring容器中
    package com.github.it235.register;
    
    /**
     * @description: 根据yml创建redisTemplate并注入到Spring容器
     * @author: www.it235.com
     * @date: Created in 2020/9/27 15:27
     */
    public class Knife4jRedisRegister implements EnvironmentAware, ImportBeanDefinitionRegistrar {
    
        private static final Logger logger = LoggerFactory.getLogger(Knife4jRedisRegister.class);
    
        private static Map<String, Object> registerBean = new ConcurrentHashMap<>();
    
        private Environment environment;
        private Binder binder;
    
        @Override
        public void setEnvironment(Environment environment) {
            this.environment = environment;
            this.binder = Binder.get(this.environment);
        }
    
        @Override
        public void registerBeanDefinitions(AnnotationMetadata importingClassMetadata, BeanDefinitionRegistry beanDefinitionRegistry) {
            RedisEntity redisEntity;
            try {
                redisEntity = binder.bind("knife4j.redis", RedisEntity.class).get();
            } catch (NoSuchElementException e) {
                logger.error("Failed to configure knife4j redis: 'knife4j.redis' attribute is not specified and no embedded redis could be configured.");
                return;
            }
            boolean onPrimary = true;
    
            //根据多个库实例化出多个连接池和Template
            List<Integer> databases = redisEntity.getDatabases();
            for (Integer database : databases) {
                //单机模式
                RedisStandaloneConfiguration configuration = new RedisStandaloneConfiguration();
                configuration.setHostName(String.valueOf(redisEntity.getHost()));
                configuration.setPort(Integer.parseInt(String.valueOf(redisEntity.getPort())));
                configuration.setDatabase(database);
                String password = redisEntity.getPassword();
                if (password != null && !"".equals(password)) {
                    RedisPassword redisPassword = RedisPassword.of(password);
                    configuration.setPassword(redisPassword);
                }
    
                //池配置
                GenericObjectPoolConfig genericObjectPoolConfig = new GenericObjectPoolConfig();
    
                RedisProperties.Pool pool = redisEntity.getLettuce().getPool();
                genericObjectPoolConfig.setMaxIdle(pool.getMaxIdle());
                genericObjectPoolConfig.setMaxTotal(pool.getMaxActive());
                genericObjectPoolConfig.setMinIdle(pool.getMinIdle());
                if (pool.getMaxWait() != null) {
                    genericObjectPoolConfig.setMaxWaitMillis(pool.getMaxWait().toMillis());
                }
                Supplier<LettuceConnectionFactory> lettuceConnectionFactorySupplier = () -> {
                    LettuceConnectionFactory factory = (LettuceConnectionFactory) registerBean.get("LettuceConnectionFactory" + database);
                    if (factory != null) {
                        return factory;
                    }
                    LettucePoolingClientConfiguration.LettucePoolingClientConfigurationBuilder builder = LettucePoolingClientConfiguration.builder();
                    Duration shutdownTimeout = redisEntity.getLettuce().getShutdownTimeout();
                    if(shutdownTimeout == null){
                        shutdownTimeout = binder.bind("knife4j.redis.shutdown-timeout", Duration.class).get();
                    }
                    if (shutdownTimeout != null) {
                        builder.shutdownTimeout(shutdownTimeout);
                    }
                    LettuceClientConfiguration clientConfiguration = builder.poolConfig(genericObjectPoolConfig).build();
                    factory = new LettuceConnectionFactory(configuration, clientConfiguration);
                    registerBean.put("LettuceConnectionFactory" + database, factory);
                    return factory;
                };
    
                LettuceConnectionFactory lettuceConnectionFactory = lettuceConnectionFactorySupplier.get();
                BeanDefinitionBuilder builder = BeanDefinitionBuilder.genericBeanDefinition(LettuceConnectionFactory.class, lettuceConnectionFactorySupplier);
                AbstractBeanDefinition factoryBean = builder.getRawBeanDefinition();
                factoryBean.setPrimary(onPrimary);
                beanDefinitionRegistry.registerBeanDefinition("lettuceConnectionFactory" + database, factoryBean);
                // StringRedisTemplate
                GenericBeanDefinition stringRedisTemplate = new GenericBeanDefinition();
                stringRedisTemplate.setBeanClass(StringRedisTemplate.class);
                ConstructorArgumentValues constructorArgumentValues = new ConstructorArgumentValues();
                constructorArgumentValues.addIndexedArgumentValue(0, lettuceConnectionFactory);
                stringRedisTemplate.setConstructorArgumentValues(constructorArgumentValues);
                stringRedisTemplate.setAutowireMode(AutowireCapableBeanFactory.AUTOWIRE_BY_NAME);
                beanDefinitionRegistry.registerBeanDefinition("stringRedisTemplate" + database, stringRedisTemplate);
                // 定义RedisTemplate对象
                GenericBeanDefinition redisTemplate = new GenericBeanDefinition();
                redisTemplate.setBeanClass(RedisTemplate.class);
                redisTemplate.getPropertyValues().add("connectionFactory", lettuceConnectionFactory);
                redisTemplate.setAutowireMode(AutowireCapableBeanFactory.AUTOWIRE_BY_NAME);
                JsonSerialType jsonSerialType = redisEntity.getJsonSerialType();
                RedisSerializer stringRedisSerializer = null;
                Jackson2JsonRedisSerializer<Object> jackson2JsonRedisSerializer = null;
                if(jsonSerialType != null && jsonSerialType == JsonSerialType.Fastjson){
                    //若配置,则采用该方式
                    jackson2JsonRedisSerializer = new CustomFastJsonRedisSerializer<>(Object.class);
                    ParserConfig.getGlobalInstance().setAutoTypeSupport(false);
                    stringRedisSerializer = new CustomStringRedisSerializer();
                } else {
                    // 内置默认序列化(此处若不设置则采用默认的JDK设置,也可以在使用使自定义序列化方式)
                    jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer<>(Object.class);
                    ObjectMapper om = new ObjectMapper();
                    om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
                    om.activateDefaultTyping(om.getPolymorphicTypeValidator(), ObjectMapper.DefaultTyping.NON_FINAL);
                    jackson2JsonRedisSerializer.setObjectMapper(om);
                    stringRedisSerializer = new StringRedisSerializer();
                }
                // key采用String的序列化方式,value采用json序列化方式
                redisTemplate.getPropertyValues().add("keySerializer",stringRedisSerializer);
                redisTemplate.getPropertyValues().add("hashKeySerializer",stringRedisSerializer);
                redisTemplate.getPropertyValues().add("valueSerializer",jackson2JsonRedisSerializer);
                redisTemplate.getPropertyValues().add("hashValueSerializer",jackson2JsonRedisSerializer);
    
                //注册Bean
                beanDefinitionRegistry.registerBeanDefinition("redisTemplate" + database, redisTemplate);
                logger.info("Registration redis ({}) !", database);
                if (onPrimary) {
                    onPrimary = false;
                }
            }
        }
    }
    
    
  • 第三步,创建redisManager进行管理redisTemplate
/**
 * @description: 给工具类提供manager,由先的configuration进行初始化和赋值
 * @author: www.it235.com
 * @date: Created in 2020/9/27 15:27
 */
public class Knife4jRedisManager {

    private Map<String, RedisTemplate> redisTemplateMap;

    private Map<String, StringRedisTemplate> stringRedisTemplateMap;

    public Knife4jRedisManager(Map<String, RedisTemplate> redisTemplateMap ,
                               Map<String, StringRedisTemplate> stringRedisTemplateMap) {
        this.redisTemplateMap = redisTemplateMap;
        this.stringRedisTemplateMap = stringRedisTemplateMap;
    }

    public RedisTemplate redisTemplate(int dbIndex) {
        RedisTemplate redisTemplate = redisTemplateMap.get("redisTemplate" + dbIndex);
        return redisTemplate;
    }

    public StringRedisTemplate stringRedisTemplate(int dbIndex) {
        StringRedisTemplate stringRedisTemplate = stringRedisTemplateMap.get("stringRedisTemplate" + dbIndex);
        stringRedisTemplate.setEnableTransactionSupport(true);
        return stringRedisTemplate;
    }

    public Map<String, RedisTemplate> getRedisTemplateMap() {
        return redisTemplateMap;
    }

    public Map<String, StringRedisTemplate> getStringRedisTemplateMap() {
        return stringRedisTemplateMap;
    }
}
  • 第四部,创建Configuration,管理redisTemplate,加载redisManager
/**
 * @description: 核心配置,取出spring中的redisTemplate暂存到map中,并初始化redisManager
 * @author: www.it235.com
 * @date: Created in 2020/9/27 15:27
 */
@AutoConfigureBefore({RedisAutoConfiguration.class})
@Import(Knife4jRedisRegister.class)
@EnableCaching
@Configuration
public class Knife4jRedisConfiguration implements EnvironmentAware , ApplicationContextAware {

    private static final Logger logger = LoggerFactory.getLogger(Knife4jRedisConfiguration.class);

    private static String key1 = "redisTemplate";
    private static String key2 = "stringRedisTemplate";

    Map<String, RedisTemplate> redisTemplateMap = new HashMap<>();
    Map<String, StringRedisTemplate> stringRedisTemplateMap = new HashMap<>();
    private Binder binder;
    private Environment environment;

    @Override
    public void setEnvironment(Environment environment) {
        this.environment = environment;
        this.binder = Binder.get(this.environment);
    }
    @PostConstruct
    public Map<String,RedisTemplate> initRedisTemplate(){
        RedisEntity redisEntity;
        try {
            redisEntity = binder.bind("knife4j.redis", RedisEntity.class).get();
        } catch (NoSuchElementException e) {
            throw new RuntimeException("Failed to configure knife4j redis: 'knife4j.redis' attribute is not specified and no embedded redis could be configured.");
        }

        //根据多个库实例化出多个连接池和Template
        List<Integer> databases = redisEntity.getDatabases();
        if(databases == null || databases.size() == 0){
            logger.warn("no config property knife4j.redis.databases , default use db0!!!");
            databases.add(0);
        }

        //根据指定的数据库个数来加载对应的RedisTemplate
        for (Integer database : databases) {
            String key = key1 + database;
            RedisTemplate redisTemplate = applicationContext.getBean(key , RedisTemplate.class);
            if(redisTemplate != null){
                redisTemplateMap.put(key , redisTemplate);
            }

            key = key2 + database;
            if(stringRedisTemplateMap != null){
                StringRedisTemplate stringRedisTemplate = applicationContext.getBean(key , StringRedisTemplate.class);
                stringRedisTemplateMap.put(key , stringRedisTemplate);
            }
        }
        if(redisTemplateMap.size() == 0 && stringRedisTemplateMap.size() == 0){
            throw new RuntimeException("load redisTemplate failure , please check knife4j.redis property config!!!");
        }
        return redisTemplateMap;
    }

    private ApplicationContext applicationContext;

    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        this.applicationContext = applicationContext;
    }


    @Bean
    public Knife4jRedisManager knife4jRedisManager(){
        return new Knife4jRedisManager(redisTemplateMap , stringRedisTemplateMap);
    @Bean
    public RedisBaseUtil redisBaseUtil(){
        return new RedisBaseUtil();
    }
}

spring.factories配置

org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
  com.github.it235.config.Knife4jRedisConfiguration\
  • 第五部,编写工具类

/**
 * @description: redis工具类基类,集成通用的方法,由`configuration`进行初始化
 * @author: jianjun.ren
 * @date: Created in 2020/9/27 11:20
 */
public class RedisBaseUtil {

    @Autowired
    protected Knife4jRedisManager knife4jRedisManager;

    protected int defaultDB = 0;

    public void delete(String key) {
        delete(0,key);
    }
    public void delete(int dbIndex ,String key) {
        knife4jRedisManager.redisTemplate(dbIndex).delete(key);
    }

    public boolean set(String key, Object value) {
        return set(0,key,value);
    }
    public boolean set(int dbIndex ,String key, Object value) {
        try {
            knife4jRedisManager.redisTemplate(dbIndex).opsForValue().set(key, value);
            return true;
        } catch (Exception e) {
            e.printStackTrace();
            return false;
        }
    }

    public void delete(Collection keys){
        delete(defaultDB , keys);
    }
    public void delete(int dbIndex ,Collection keys){
        knife4jRedisManager.redisTemplate(dbIndex).delete(keys);
    }

    public Set getKeys(String redisKey) {
        return getKeys(0,redisKey);
    }
    public Set getKeys(int dbIndex ,String redisKey) {
        Set keys = knife4jRedisManager.redisTemplate(dbIndex).opsForHash().keys(redisKey);
        Set retKeys = new HashSet<>();
        for (Object key : keys) {
            retKeys.add(String.valueOf(key));
        }
        return retKeys;
    }

    /**
     * 每个redis
     * 指定缓存失效时间
     * @param key 键
     * @param time 时间(秒)
     * @return
     */
    public boolean expire(String key, long time) {
        return expire(defaultDB , key,time);
    }
    public boolean expire(int dbIndex ,String key, long time) {
        try {
            if (time > 0) {
                knife4jRedisManager.redisTemplate(dbIndex).expire(key, time, TimeUnit.SECONDS);
            }
            return true;
        } catch (Exception e) {
            e.printStackTrace();
            return false;
        }
    }
    /**
     * 根据key 获取过期时间
     * @param key 键 不能为null
     * @return 时间(秒) 返回0代表为永久有效
     */
    public long getExpire(int dbIndex , String key) {
        return knife4jRedisManager.redisTemplate(dbIndex).getExpire(key, TimeUnit.SECONDS);
    }
    public long getExpire(String key) {
        return getExpire(defaultDB , key);
    }
    /**
     * 判断key是否存在
     * @param key 键
     * @return true 存在 false不存在
     */
    public boolean hasKey(String key) {
        return hasKey(defaultDB , key);
    }
    public boolean hasKey(int dbIndex ,String key) {
        try {
            return knife4jRedisManager.redisTemplate(dbIndex).hasKey(key);
        } catch (Exception e) {
            e.printStackTrace();
            return false;
        }
    }
}
 
  
  • 编写测试类
@RestController
@RequestMapping("/test")
public class RedisTestController {

    @Autowired
    private RedisBaseUtil redisBaseUtil;

    /**
     * 单值操作测试
     * @param key
     * @return
     */
    @GetMapping("/val/{key}")
    public String test(@PathVariable("key") String key){
        redisBaseUtil.set(key , "默认库设置");
        redisBaseUtil.set(1 , key , "指定1库设置值");
        //查看key是否存在
        boolean flag = redisValUtil.hasKey(1, key);
        System.out.println("指定1库获取值:" + flag);
        return "ok";
    }
}

到此测试并查看数据库,就会发现,基础功能已经完成了。
如果有帮助,别忘了star三连:https://github.com/it235/knife4j-redis-lettuce

你可能感兴趣的:(#,SpringBoot2.0,redis,spring,boot,动态切换Redis数据源DB)