spring-session之4 redis集群配置
前文,我们 spring-session之3 redis配置参数配置
, 我们可以配置简单的ip和port,但是生产环境,我们的redis是做了集群
,肯定不是单点,此时就不能单单hostName 和 port了, 怎么办?
好,今天的目标是:
- 配置
spring-session
redis 集群
1. 我们原来的 redis data 配置
1.1. spring-redis.xml
xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context" xmlns:loxia="http://loxia.benjamin.cn/schema/core"
xmlns:p="http://www.springframework.org/schema/p" xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
http://loxia.benjamin.cn/schema/core http://loxia.benjamin.cn/schema/core/loxia-spring-ext.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
">
<bean id="redisconfig" class="redis.clients.jedis.JedisPoolConfig">
<property name="maxActive" >
<value type="long">#{redis['redis.MaxActive']}value>
property>
<property name="maxIdle" >
<value type="long">#{redis['redis.MaxIdle']}value>
property>
<property name="maxWait">
<value type="long">#{redis['redis.MaxWait']}value>
property>
<property name="testOnBorrow" >
<value type="boolean">truevalue>
property>
<property name="testOnReturn" >
<value type="boolean">truevalue>
property>
bean>
<bean id="jedisSentinelPool" class="redis.clients.jedis.JedisSentinelPool">
<constructor-arg ref="redisconfig">constructor-arg>
<constructor-arg type="java.util.Set">
<set>
<value>#{redis['redis.sentinel1']}value>
<value>#{redis['redis.sentinel2']}value>
<value>#{redis['redis.sentinel3']}value>
set>
constructor-arg>
<constructor-arg >
<value>#{redis['redis.mastername']}value>
constructor-arg>
bean>
beans>
1.2 redis.properties
#A string containing whitespace or comma separated host or IP addresses and port numbers of the form "host:port host2:port" or "host:port, host2:port".
redis.sentinel1=redis01.public.test.baozun.cn:26379
redis.sentinel2=redis02.public.test.baozun.cn:26379
redis.sentinel3=redis03.public.test.baozun.cn:26379
redis.mastername=testenv
redis.timeout=1000
redis.MaxActive=50
redis.MaxIdle=60000
redis.MaxWait=5000
##这个是我们自定义的 key前缀, 以便用同一套redis集群,适应多个不同的商城
redis.keystart=feilongdev
2.redis 集群配置
那么我们怎么使用上面的环境来配置我们的spring-session呢?
使用通过上述的参数表格, JedisPoolConfig 可以使用,但是没有 JedisSentinelPool参数
而 JedisShardInfo
属性并不是list,只是 包含了jedis服务器的一些信息 ,咋整?
这时候,我们来看看 JedisConnectionFactory 构造函数:
可以看出 ,构造函数支持 RedisSentinelConfiguration
或者 RedisClusterConfiguration
此处的 JedisConnectionFactory
是 spring-data-redis 内容, 有兴趣的可以看看, 这里也有入门版的教程,参见使用Spring Data Redis操作Redis(一)
2.1 RedisSentinelConfiguration
我们先使用 redis 哨兵机制的集群
那么我们尝试修改以下配置配置:
xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
xmlns:p="http://www.springframework.org/schema/p" xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
">
<context:annotation-config />
<util:properties id="redis" location="classpath:redis.properties">util:properties>
<bean class="org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration" />
<bean class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
<constructor-arg index="0">
<bean class="org.springframework.data.redis.connection.RedisSentinelConfiguration">
<constructor-arg index="0" value="#{redis['redis.mastername']}" />
<constructor-arg index="1">
<set>
<value>#{redis['redis.sentinel1']}value>
<value>#{redis['redis.sentinel2']}value>
<value>#{redis['redis.sentinel3']}value>
set>
constructor-arg>
bean>
constructor-arg>
<constructor-arg index="1">
<bean class="redis.clients.jedis.JedisPoolConfig">
<property name="maxActive" value="#{redis['redis.MaxActive']}" />
<property name="maxIdle" value="#{redis['redis.MaxIdle']}" />
<property name="maxWait" value="#{redis['redis.MaxWait']}" />
<property name="testOnBorrow" value="true" />
<property name="testOnReturn" value="true" />
bean>
constructor-arg>
bean>
beans>
启动应用
出现以下的错误
Caused by: org.springframework.beans.NotWritablePropertyException: Invalid property 'maxActive' of bean class [redis.clients.jedis.JedisPoolConfig]: Bean property 'maxActive' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
at org.springframework.beans.BeanWrapperImpl.setPropertyValue(BeanWrapperImpl.java:1044)
at org.springframework.beans.BeanWrapperImpl.setPropertyValue(BeanWrapperImpl.java:904)
at org.springframework.beans.AbstractPropertyAccessor.setPropertyValues(AbstractPropertyAccessor.java:75)
at org.springframework.beans.AbstractPropertyAccessor.setPropertyValues(AbstractPropertyAccessor.java:57)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1452)
... 30 more
JedisPoolConfig 不支持 maxActive 属性了 see xetorthio/jedis#1031
好吧, 修改下配置 spring-session.xml
xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
xmlns:p="http://www.springframework.org/schema/p" xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
">
<context:annotation-config />
<util:properties id="redis" location="classpath:redis.properties">util:properties>
<bean class="org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration" />
<bean class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
<constructor-arg index="0">
<bean class="org.springframework.data.redis.connection.RedisSentinelConfiguration">
<constructor-arg index="0" value="#{redis['redis.mastername']}" />
<constructor-arg index="1">
<set>
<value>#{redis['redis.sentinel1']}value>
<value>#{redis['redis.sentinel2']}value>
<value>#{redis['redis.sentinel3']}value>
set>
constructor-arg>
bean>
constructor-arg>
<constructor-arg index="1">
<bean class="redis.clients.jedis.JedisPoolConfig">
<property name="maxIdle" value="#{redis['redis.MaxIdle']}" />
<property name="testOnBorrow" value="true" />
<property name="testOnReturn" value="true" />
bean>
constructor-arg>
bean>
beans>
redis.properties
#A string containing whitespace or comma separated host or IP addresses and port numbers of the form "host:port host2:port" or "host:port, host2:port".
redis.sentinel1=redis01.public.test.baozun.cn:26379
redis.sentinel2=redis02.public.test.baozun.cn:26379
redis.sentinel3=redis03.public.test.baozun.cn:26379
redis.mastername=testenv
redis.timeout=1000
redis.MaxIdle=60000
#新版jedis 不支持这个参数了
#redis.MaxActive=50
#redis.MaxWait=5000
redis.keystart=gowilddev
启动
报了新错
Caused by: redis.clients.jedis.exceptions.JedisDataException: ERR Unsupported CONFIG parameter: notify-keyspace-events
at redis.clients.jedis.Protocol.processError(Protocol.java:117)
at redis.clients.jedis.Protocol.process(Protocol.java:151)
at redis.clients.jedis.Protocol.read(Protocol.java:205)
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:297)
at redis.clients.jedis.Connection.getStatusCodeReply(Connection.java:196)
at redis.clients.jedis.Jedis.configSet(Jedis.java:2612)
at org.springframework.data.redis.connection.jedis.JedisConnection.setConfig(JedisConnection.java:633)
... 25 more
什么鬼,参见 ERR Unsupported CONFIG parameter: notify-keyspace-events
看来我是使用新的jedis 连接了老版本的redis集群
服务器上面的 redis 版本
找运维组重新搭建 redis新版本的集群环境
[vmuser@test01 ~]$ redis-cli --version
redis-cli 3.2.4
由于采用了新版的集群技术,所以我们只能使用 RedisClusterConfiguration
2.2 RedisClusterConfiguration
2.2.1 3主3从 Cluster服务端配置
我们先看看运维组兄弟配置的集群情况
[vmuser@test01 ~]$ /home/vmuser/redis-3.2.4/src/redis-trib.rb check 10.88.21.31:10000
可以看到是官网推荐的标准的 3主 3从
方案,参见 官方文档 http://redisdoc.com/topic/cluster-tutorial.html
10.88.21.31:10000 | 10.88.22.25:10001 |
10.88.22.25:10000 | 10.88.21.31:10002 |
10.88.21.31:10001 | 10.88.22.25:10002 |
2.2.2 spring配置
这时我们可以使用 org.springframework.data.redis.connection.RedisClusterConfiguration
,注意 since spring-data-redis
1.7
xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
xmlns:p="http://www.springframework.org/schema/p" xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
">
<context:annotation-config />
<util:properties id="redis" location="classpath:redis-cluster.properties">util:properties>
<bean class="org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration" />
<bean class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
<constructor-arg index="0">
<bean class="org.springframework.data.redis.connection.RedisClusterConfiguration">
<constructor-arg index="0">
<set>
<value>10.88.21.31:10000value>
<value>10.88.22.25:10000value>
<value>10.88.21.31:10001value>
set>
constructor-arg>
bean>
constructor-arg>
<constructor-arg index="1">
<bean class="redis.clients.jedis.JedisPoolConfig">
<property name="maxIdle" value="#{redis['redis.jedisPoolConfig.MaxIdle']}" />
<property name="testOnBorrow" value="#{redis['redis.jedisPoolConfig.testOnBorrow']}" />
<property name="testOnReturn" value="#{redis['redis.jedisPoolConfig.testOnReturn']}" />
bean>
constructor-arg>
bean>
beans>
此时,我们仅需要配置 master 到 RedisClusterConfiguration
中,系统会自动分配
上面配置可以提取精炼到配置文件中
xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
xmlns:p="http://www.springframework.org/schema/p" xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
">
<context:annotation-config />
<util:properties id="redis" location="classpath:redis-cluster.properties">util:properties>
<bean class="org.springframework.session.data.redis.config.annotation.web.http.RedisHttpSessionConfiguration" />
<bean class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
<constructor-arg index="0">
<bean class="org.springframework.data.redis.connection.RedisClusterConfiguration">
<constructor-arg index="0">
<set>
<value>#{redis['redis.redisClusterConfiguration.clusters']}value>
set>
constructor-arg>
<property name="maxRedirects" value="#{redis['redis.redisClusterConfiguration.maxRedirects']}" />
bean>
constructor-arg>
<constructor-arg index="1">
<bean class="redis.clients.jedis.JedisPoolConfig">
<property name="maxIdle" value="#{redis['redis.jedisPoolConfig.MaxIdle']}" />
<property name="testOnBorrow" value="#{redis['redis.jedisPoolConfig.testOnBorrow']}" />
<property name="testOnReturn" value="#{redis['redis.jedisPoolConfig.testOnReturn']}" />
bean>
constructor-arg>
bean>
beans>
redis-cluster.properties
#############for org.springframework.data.redis.connection.RedisClusterConfiguration###################
#只需要配置 master
#理论上只需要配置一个节点即可,配置多个是为了防止单个节点挂掉,
redis.redisClusterConfiguration.clusters=10.88.21.31:10000,10.88.22.25:10000,10.88.21.31:10001
#用于 redis.clients.jedis.JedisCluster.JedisCluster(Set, int, int, GenericObjectPoolConfig) 第三个参数 maxRedirections
#默认值是5
#一般当此值设置过大时,容易报:Too many Cluster redirections
redis.redisClusterConfiguration.maxRedirects=3
###########for redis.clients.jedis.JedisPoolConfig##############################
redis.jedisPoolConfig.MaxIdle=60000
redis.jedisPoolConfig.testOnBorrow=true
redis.jedisPoolConfig.testOnReturn=true
#新版jedis 不支持这个参数了
#redis.jedisPoolConfig.MaxActive=50
#redis.jedisPoolConfig.MaxWait=5000
#redis.keystart=gowilddev
启动
成功没有报错
2.2.3 为什么标准的 3主 3从
方案,我们 clusterNodes
仅配置了 3个 master?
- 首先配置多个是为了防止单个节点挂掉,理论上只需要配置一个节点即可
- 至于为什么不配置slave节点,参见 https://github.com/spring-projects/spring-data-examples spring data 示例中也只配置了master
2.2.4 关于 RedisClusterConfiguration
中的 maxRedirects
属性
重试次数,在执行失败后,进行的重试次数,默认是5, 参见 https://www.zybuluo.com/SailorXiao/note/159072#解读
主要用于 org.springframework.data.redis.connection.jedis.JedisConnectionFactory.createCluster(RedisClusterConfiguration, GenericObjectPoolConfig)
构造 redis.clients.jedis.JedisCluster.JedisCluster(Set, int, int, GenericObjectPoolConfig) 使用的
源码:
/**
* Creates {@link JedisCluster} for given {@link RedisClusterConfiguration} and {@link GenericObjectPoolConfig}.
*
* @param clusterConfig must not be {@literal null}.
* @param poolConfig can be {@literal null}.
* @return
* @since 1.7
*/
protected JedisCluster createCluster(RedisClusterConfiguration clusterConfig, GenericObjectPoolConfig poolConfig) {
Assert.notNull("Cluster configuration must not be null!");
Set<HostAndPort> hostAndPort = new HashSet<HostAndPort>();
for (RedisNode node : clusterConfig.getClusterNodes()) {
hostAndPort.add(new HostAndPort(node.getHost(), node.getPort()));
}
int redirects = clusterConfig.getMaxRedirects() != null ? clusterConfig.getMaxRedirects().intValue() : 5;
if (StringUtils.hasText(getPassword())) {
throw new IllegalArgumentException("Jedis does not support password protected Redis Cluster configurations!");
}
if (poolConfig != null) {
return new JedisCluster(hostAndPort, timeout, redirects, poolConfig);
}
return new JedisCluster(hostAndPort, timeout, redirects, poolConfig);
}
一般当此值设置过大时,容易报:Too many Cluster redirections
3.问题
3.1 spring-session>1.2.2.RELEASE 不兼容 spring-data-redis > 1.7.4.RELEASE
目前 spring-session
最新版本是 1.2.2.RELEASE
,内置依赖 spring-data-redis
版本是 1.7.1.RELEASE
而目前 spring-data-redis
最新版本是 1.7.4.RELEASE
尝试升级 spring-data-redis
>1.7.4.RELEASE
启动报异常
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionRedisTemplate' defined in class path resource [org/springframework/session/data/redis/config/annotation/web/http/RedisHttpSessionConfiguration.class]: Invocation of init method failed; nested exception is java.lang.NoSuchMethodError: org.springframework.core.serializer.support.DeserializingConverter.(Ljava/lang/ClassLoader;)V
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1514)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:191)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:921)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:864)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:779)
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:817)
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:745)
... 38 more
Caused by: java.lang.NoSuchMethodError: org.springframework.core.serializer.support.DeserializingConverter.(Ljava/lang/ClassLoader;)V
at org.springframework.data.redis.serializer.JdkSerializationRedisSerializer.(JdkSerializationRedisSerializer.java:53)
at org.springframework.data.redis.core.RedisTemplate.afterPropertiesSet(RedisTemplate.java:119)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1573)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1511)
... 49 more
4.参考
- 使用Spring Data Redis操作Redis(一)
- Why 2.7 JedisPoolConfig not Compatible with 2.1 JedisPoolConfig
- ERR Unsupported CONFIG parameter: notify-keyspace-events
- http://redisdoc.com/topic/cluster-tutorial.html
- https://github.com/spring-projects/spring-data-examples/tree/master/redis/cluster
--to be continued