Redis高可用-Cluster-测试

I.高可用测试

II.关闭所有节点

ps -ef|grep redis 
kill -9 888
 

II.删除

cd /usr/local/redis3/cluster/7111/
-rw-r--r--. 1 root root 352415 3月  29 11:42 appendonly.aof
-rw-r--r--. 1 root root   7008 3月  29 11:46 dump.rdb
-rw-r--r--. 1 root root    750 3月  29 10:53 nodes.conf
cd /usr/local/redis3/cluster/7112/
cd /usr/local/redis3/cluster/7113/
cd /usr/local/redis3/cluster/7114/
cd /usr/local/redis3/cluster/7115/
cd /usr/local/redis3/cluster/7116/


保留redis-7111.conf
#删除redis*之外的所有文件
rm -f !(redis*) 

II.启动所有节点

/usr/local/redis3/bin/redis-server /usr/local/redis3/cluster/7111/redis-7111.conf 
/usr/local/redis3/bin/redis-server /usr/local/redis3/cluster/7112/redis-7112.conf 
/usr/local/redis3/bin/redis-server /usr/local/redis3/cluster/7113/redis-7113.conf 
/usr/local/redis3/bin/redis-server /usr/local/redis3/cluster/7114/redis-7114.conf 
/usr/local/redis3/bin/redis-server /usr/local/redis3/cluster/7115/redis-7115.conf 
/usr/local/redis3/bin/redis-server /usr/local/redis3/cluster/7116/redis-7116.conf 
 检查启动
ps -ef|grep redis 

II.重启启动集群

# cd /usr/local/src/redis-3.0.3/src/ 
# cp redis-trib.rb /usr/local/bin/redis-trib 
/usr/local/bin/redis-trib  create  --replicas  1 10.1.125.152:7111 10.1.125.89:7112 10.1.125.212:7113 10.1.125.203:7114 10.1.125.129:7115 10.1.125.180:7116

II.登录slave5 

-登录集群必须加-c 
[root@localhost 7116]# /usr/local/redis3/bin/redis-cli -c -p 7116
127.0.0.1:7116> set name dingsai
-> Redirected to slot [5798] located at 10.1.125.89:7112
OK
10.1.125.89:7112> get name
"dingsai"

II.登录master

/usr/local/redis3/bin/redis-cli -c -p 7111
127.0.0.1:7112> get name
"dingsai"

II.检查节点

在任意节点上执行
[root@localhost local]# /usr/local/bin/redis-trib check 10.1.125.152:7111
Connecting to node 10.1.125.152:7111: OK
Connecting to node 10.1.125.212:7113: OK
Connecting to node 10.1.125.180:7116: OK
Connecting to node 10.1.125.203:7114: OK
Connecting to node 10.1.125.129:7115: OK
Connecting to node 10.1.125.89:7112: OK
>>> Performing Cluster Check (using node 10.1.125.152:7111)
M: adaebf848247181fba9d50886a8712bf2a38a9d1 10.1.125.152:7111
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: e3c9199d23ce76c2df3d6ddb0fa5145364f0bb18 10.1.125.212:7113
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: a6e06d51f4f99ce1adc306dc62da84aa3dc95f7a 10.1.125.180:7116
   slots: (0 slots) slave
   replicates e3c9199d23ce76c2df3d6ddb0fa5145364f0bb18
S: 510a0b83d2b39b43ecf893a27c2c3bbf1a431399 10.1.125.203:7114
   slots: (0 slots) slave
   replicates adaebf848247181fba9d50886a8712bf2a38a9d1
S: e6d20715e56350fb690c609d5fc68a7a166b4fea 10.1.125.129:7115
   slots: (0 slots) slave
   replicates 4ac21be2d805505cbedee871161cf856607a29f0
M: 4ac21be2d805505cbedee871161cf856607a29f0 10.1.125.89:7112
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

II.模拟用户使用redis

 
package com.ding.study.redis;

import java.util.HashSet;
import java.util.Set;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import redis.clients.jedis.HostAndPort;
import redis.clients.jedis.JedisCluster;
import redis.clients.jedis.JedisPoolConfig;

/**
 * 
 * @author daniel
 * @email [email protected]
 * @time 2016-3-29 下午3:41:31
 */
public class RedisClusterFailoverTest {
	private static final Log log = LogFactory.getLog(RedisClusterFailoverTest.class);

	public static void main(String[] args) {
		
		// 数据库链接池配置
		JedisPoolConfig config = new JedisPoolConfig();  
        config.setMaxTotal(100);  
        config.setMaxIdle(50);  
        config.setMinIdle(20);  
        config.setMaxWaitMillis(6 * 1000);  
        config.setTestOnBorrow(true);  
		
		// Redis集群的节点集合
		Set<HostAndPort> jedisClusterNodes = new HashSet<HostAndPort>();
		jedisClusterNodes.add(new HostAndPort("10.1.125.152", 7111));
		jedisClusterNodes.add(new HostAndPort("10.1.125.89", 7112));
		jedisClusterNodes.add(new HostAndPort("10.1.125.212", 7113));
		jedisClusterNodes.add(new HostAndPort("10.1.125.203", 7114));
		jedisClusterNodes.add(new HostAndPort("10.1.125.129", 7115));
		jedisClusterNodes.add(new HostAndPort("10.1.125.180", 7116));
		
		try {
			
			// 根据节点集创集群链接对象
			//JedisCluster jedisCluster = new JedisCluster(jedisClusterNodes);
			// 集群各节点集合，超时时间，最多重定向次数，链接池
			JedisCluster jedisCluster = new JedisCluster(jedisClusterNodes, 5000, 100, config);
			int num = 1000;
			String key = "dingsai";
			String value = "";
			int count = 1;

			while(true){
				for (int i=1; i <= num; i++){
					try {
						// 存数据
				      jedisCluster.set(key+i, "Value"+i); 
						
						// 取数据
						 value = jedisCluster.get(key+i); 
						log.info(key+i + "=" + value);
						if (value == null || "".equals(value)){
							log.error("===>break" + key+i + " value is null");
							break;
						}
					} catch (Exception e) {
						log.error("====>", e);
						Thread.sleep(3000);
						continue;
					}
					// 删除数据
					//jedisCluster.del(key+i); 
					//value = jedisCluster.get(key+i); 
					//log.info(key+i + "=" + value);
				}
				log.info("===================================>count:" + count);
				if (value == null || "".equals(value)){
					break;
				}
				count++;
				Thread.sleep(1000);
			}
		} catch (Exception e) {
			log.error("====>", e);
		}
		
	}
}

 

II.关闭master节点10.1.125.212

[root@localhost local]# ps -ef|grep redis
root     37230     1  3 14:45 ?        00:00:43 /usr/local/redis3/bin/redis-server *:7113 [cluster]
root     38794 22391  0 15:07 pts/2    00:00:00 grep --color=auto redis
[root@localhost local]# kill -9 37230
客户端很大的延迟，等了好几秒才进行操作

II.再次查看集群状态

127.0.0.1:7116> cluster nodes
a6e06d51f4f99ce1adc306dc62da84aa3dc95f7a 10.1.125.180:7116 myself,master - 0 0 7 connected 10923-16383
4ac21be2d805505cbedee871161cf856607a29f0 10.1.125.89:7112 master - 0 1459235330419 2 connected 5461-10922
adaebf848247181fba9d50886a8712bf2a38a9d1 10.1.125.152:7111 master - 0 1459235328412 1 connected 0-5460
e6d20715e56350fb690c609d5fc68a7a166b4fea 10.1.125.129:7115 slave 4ac21be2d805505cbedee871161cf856607a29f0 0 1459235329416 5 connected
e3c9199d23ce76c2df3d6ddb0fa5145364f0bb18 10.1.125.212:7113 master, fail - 1459235250356 1459235244712 3 disconnected
510a0b83d2b39b43ecf893a27c2c3bbf1a431399 10.1.125.203:7114 slave adaebf848247181fba9d50886a8712bf2a38a9d1 0 1459235327407 4 connected
127.0.0.1:7116> 


[root@localhost local]# /usr/local/bin/redis-trib check 10.1.125.152:7111
Connecting to node 10.1.125.152:7111: OK
Connecting to node 10.1.125.180:7116: OK
Connecting to node 10.1.125.203:7114: OK
Connecting to node 10.1.125.129:7115: OK
Connecting to node 10.1.125.89:7112: OK
>>> Performing Cluster Check (using node 10.1.125.152:7111)
M: adaebf848247181fba9d50886a8712bf2a38a9d1 10.1.125.152:7111
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: a6e06d51f4f99ce1adc306dc62da84aa3dc95f7a 10.1.125.180:7116
   slots:10923-16383 (5461 slots) master
   0 additional replica(s)
S: 510a0b83d2b39b43ecf893a27c2c3bbf1a431399 10.1.125.203:7114
   slots: (0 slots) slave
   replicates adaebf848247181fba9d50886a8712bf2a38a9d1
S: e6d20715e56350fb690c609d5fc68a7a166b4fea 10.1.125.129:7115
   slots: (0 slots) slave
   replicates 4ac21be2d805505cbedee871161cf856607a29f0
M: 4ac21be2d805505cbedee871161cf856607a29f0 10.1.125.89:7112
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

II.启动master3(slave2)节点

 /usr/local/redis3/bin/redis-server /usr/local/redis3/cluster/7113/redis-7113.conf 


 

II.再次查看集群状态

[root@localhost local]# /usr/local/bin/redis-trib check 10.1.125.152:7111
Connecting to node 10.1.125.152:7111: OK
Connecting to node 10.1.125.212:7113: OK
Connecting to node 10.1.125.180:7116: OK
Connecting to node 10.1.125.203:7114: OK
Connecting to node 10.1.125.129:7115: OK
Connecting to node 10.1.125.89:7112: OK
>>> Performing Cluster Check (using node 10.1.125.152:7111)
M: adaebf848247181fba9d50886a8712bf2a38a9d1 10.1.125.152:7111
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: e3c9199d23ce76c2df3d6ddb0fa5145364f0bb18 10.1.125.212:7113
   slots: (0 slots) slave
   replicates a6e06d51f4f99ce1adc306dc62da84aa3dc95f7a
M: a6e06d51f4f99ce1adc306dc62da84aa3dc95f7a 10.1.125.180:7116
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 510a0b83d2b39b43ecf893a27c2c3bbf1a431399 10.1.125.203:7114
   slots: (0 slots) slave
   replicates adaebf848247181fba9d50886a8712bf2a38a9d1
S: e6d20715e56350fb690c609d5fc68a7a166b4fea 10.1.125.129:7115
   slots: (0 slots) slave
   replicates 4ac21be2d805505cbedee871161cf856607a29f0
M: 4ac21be2d805505cbedee871161cf856607a29f0 10.1.125.89:7112
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.


127.0.0.1:7116> cluster nodes
a6e06d51f4f99ce1adc306dc62da84aa3dc95f7a 10.1.125.180:7116 myself,master - 0 0 7 connected 10923-16383
4ac21be2d805505cbedee871161cf856607a29f0 10.1.125.89:7112 master - 0 1459235640927 2 connected 5461-10922
adaebf848247181fba9d50886a8712bf2a38a9d1 10.1.125.152:7111 master - 0 1459235643946 1 connected 0-5460
e6d20715e56350fb690c609d5fc68a7a166b4fea 10.1.125.129:7115 slave 4ac21be2d805505cbedee871161cf856607a29f0 0 1459235641934 5 connected
e3c9199d23ce76c2df3d6ddb0fa5145364f0bb18 10.1.125.212:7113 slave a6e06d51f4f99ce1adc306dc62da84aa3dc95f7a 0 1459235642943 7 connected
510a0b83d2b39b43ecf893a27c2c3bbf1a431399 10.1.125.203:7114 slave adaebf848247181fba9d50886a8712bf2a38a9d1 0 1459235636891 4 connected




 

I.JedisCluster 常见错误

II.重定向次数过多默认5次
redis.clients.jedis.exceptions.JedisClusterMaxRedirectionsException: Too many Cluster redirections  
默认超时2s，重定向5次 初始化JedisCluster 配置maxRedirections 
 // 集群各节点集合，超时时间，最多重定向次数，链接池
 JedisCluster jedisCluster = new JedisCluster(jedisClusterNodes, 5000, 100, config);


II.集群不可用，高可用切换的时候可能报错.
redis.clients.jedis.exceptions.JedisClusterException: CLUSTERDOWN The cluster is down 
 重试即可
 
II.超时 
redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out 

修改超时时间