07：redis cluster（分布式集群）

redis cluster（分布式集群）

高性能：
1、在多分片节点中，将16384个槽位，均匀分布到多个分片节点中
2、存数据时，将key做crc16(key),然后和16384进行取模，得出槽位值（0-16383之间）
3、根据计算得出的槽位值，找到相对应的分片节点的主节点，存储到相应槽位上
4、如果客户端当时连接的节点不是将来要存储的分片节点，分片集群会将客户端连接切换至真正存储节点进行数据存储

高可用：
在搭建集群时，会为每一个分片的主节点，对应一个从节点，实现slaveof的功能，同时当主节点down，实现类似于sentinel的自动failover的功能。

规划、搭建过程：

6个redis实例，一般会放到3台硬件服务器
注：在企业规划中，一个分片的两个分到不同的物理机，防止硬件主机宕机造成的整个分片数据丢失。

端口号：7000-7005

1、安装集群插件
EPEL源安装ruby支持

yum install ruby rubygems -y

使用国内源
gem sources -l
gem sources -a http://mirrors.aliyun.com/rubygems/
gem sources --remove http://rubygems.org/
gem install redis -v 3.3.3
gem sources -l

或者：
gem sources -a http://mirrors.aliyun.com/rubygems/ --remove http://rubygems.org/

2、集群节点准备

mkdir /nosql/700{0..5}

vim /nosql/7000/redis.conf
port 7000
daemonize yes
pidfile /nosql/7000/redis.pid
loglevel notice
logfile "/nosql/7000/redis.log"
dbfilename dump.rdb
dir /nosql/7000
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

vim /nosql/7001/redis.conf
port 7001
daemonize yes
pidfile /nosql/7001/redis.pid
loglevel notice
logfile "/nosql/7001/redis.log"
dbfilename dump.rdb
dir /nosql/7001
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

vim /nosql/7002/redis.conf
port 7002
daemonize yes
pidfile /nosql/7002/redis.pid
loglevel notice
logfile "/nosql/7002/redis.log"
dbfilename dump.rdb
dir /nosql/7002
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

vim /nosql/7003/redis.conf
port 7003
daemonize yes
pidfile /nosql/7003/redis.pid
loglevel notice
logfile "/nosql/7003/redis.log"
dbfilename dump.rdb
dir /nosql/7003
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

vim /nosql/7004/redis.conf
port 7004
daemonize yes
pidfile /nosql/7004/redis.pid
loglevel notice
logfile "/nosql/7004/redis.log"
dbfilename dump.rdb
dir /nosql/7004
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

vim /nosql/7005/redis.conf
port 7005
daemonize yes
pidfile /nosql/7005/redis.pid
loglevel notice
logfile "/nosql/7005/redis.log"
dbfilename dump.rdb
dir /nosql/7005
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

启动节点：
redis-server /nosql/7000/redis.conf
redis-server /nosql/7001/redis.conf
redis-server /nosql/7002/redis.conf
redis-server /nosql/7003/redis.conf
redis-server /nosql/7004/redis.conf
redis-server /nosql/7005/redis.conf

[root@db01 ~]# ps -ef |grep redis
root 8854 1 0 03:56 ? 00:00:00 redis-server *:7000 [cluster]
root 8858 1 0 03:56 ? 00:00:00 redis-server *:7001 [cluster]
root 8860 1 0 03:56 ? 00:00:00 redis-server *:7002 [cluster]
root 8864 1 0 03:56 ? 00:00:00 redis-server *:7003 [cluster]
root 8866 1 0 03:56 ? 00:00:00 redis-server *:7004 [cluster]
root 8874 1 0 03:56 ? 00:00:00 redis-server *:7005 [cluster]

3、将节点加入集群管理
redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 \
127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

4、集群状态查看
集群主节点状态
redis-cli -p 7000 cluster nodes | grep master
集群从节点状态
redis-cli -p 7000 cluster nodes | grep slave

5、集群节点管理

5.1 增加新的节点

mkdir /nosql/7006
mkdir /nosql/7007

vim /nosql/7006/redis.conf
port 7006
daemonize yes
pidfile /nosql/7006/redis.pid
loglevel notice
logfile "/nosql/7006/redis.log"
dbfilename dump.rdb
dir /nosql/7006
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

vim /nosql/7007/redis.conf
port 7007
daemonize yes
pidfile /nosql/7007/redis.pid
loglevel notice
logfile "/nosql/7007/redis.log"
dbfilename dump.rdb
dir /nosql/7007
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

redis-server /nosql/7006/redis.conf
redis-server /nosql/7007/redis.conf

5.2 添加主节点：
redis-trib.rb add-node 127.0.0.1:7006 127.0.0.1:7000

5.3 转移slot（重新分片）
redis-trib.rb reshard 127.0.0.1:7000
计算以下应该分配多少个slot到新节点：；bc 16384/4 = 4096

移动多少个槽位
(1)How many slots do you want to move (from 1 to 16384)? 4096

谁来接受这些槽位
(2)What is the receiving node ID? dd76db8928575125e0046c995df5054f7acf69fa
从哪些节点去获取这些槽位：
(3)Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1:all

最后确认分片计划
(4)Do you want to proceed with the proposed reshard plan (yes/no)? yes

5.4 添加一个从节点
redis-trib.rb add-node --slave --master-id dd76db8928575125e0046c995df5054f7acf69fa 127.0.0.1:7007 127.0.0.1:7000

6.删除节点

将需要删除节点（7006）slot移动走

redis-trib.rb reshard 127.0.0.1:7000

（1）How many slots do you want to move (from 1 to 16384)? 4096
（2）What is the receiving node ID? 2d8667a8005786cc966dd18a07a442df471e9fe3
（3）Source node #1:dd76db8928575125e0046c995df5054f7acf69fa
（4）done
（5）Do you want to proceed with the proposed reshard plan (yes/no)? yes

删除一个节点
redis-trib.rb del-node ip:port ''
删除master节点之前首先要使用reshard移除master的全部slot,然后再删除当前节点

redis-trib.rb del-node 127.0.0.1:7006 dd76db8928575125e0046c995df5054f7acf69fa
redis-trib.rb del-node 127.0.0.1:7007 5c0b10501e4f6775eac3fd2e7c330f35fc81b1e2

---------------------
redis-cli -p 7000 cluster nodes | grep master
redis-cli -p 7000 cluster nodes | grep slave

------------------------------------------------------------------------------------------------------------------

扩展知识：

redis的多API支持
python为例

tar xf Python-3.5.2.tar.xz
cd Python-3.5.2
./configure
make && make install

https://redis.io/clients

下载redis-py-master.zip

unzip redis-py-master.zip
cd redis-py-master

python3 setup.py install

安装redis-cluser的客户端程序

cd redis-py-cluster-unstable
python3 setup.py install

1、对redis的单实例进行连接操作
python3
>>>import redis
>>>r = redis.StrictRedis(host='localhost', port=6379, db=0,password='root')
>>>r.set('foo', 'bar')
True
>>>r.get('foo')
'bar'
--------------------

2、sentinel集群连接并操作
redis-server /nosql/6380/redis.conf
redis-server /nosql/6381/redis.conf
redis-server /nosql/6382/redis.conf
redis-sentinel /nosql/26380/sentinel.conf &
--------------------------------
## 导入redis sentinel包
>>> from redis.sentinel import Sentinel
##指定sentinel的地址和端口号
>>> sentinel = Sentinel([('localhost', 26380)], socket_timeout=0.1)
##测试，获取以下主库和从库的信息
>>> sentinel.discover_master('mymaster')
>>> sentinel.discover_slaves('mymaster')
##配置读写分离
#写节点
>>> master = sentinel.master_for('mymaster', socket_timeout=0.1)
#读节点
>>> slave = sentinel.slave_for('mymaster', socket_timeout=0.1)
###读写分离测试 key
>>> master.set('oldboy', '123')
>>> slave.get('oldboy')
'123'

 

----------------------
redis cluster的连接并操作（python2.7.2以上版本才支持redis cluster，我们选择的是3.5）

https://github.com/Grokzen/redis-py-cluster

3、python连接rediscluster集群测试
使用

python3
>>> from rediscluster import StrictRedisCluster
>>> startup_nodes = [{"host": "127.0.0.1", "port": "7000"}]
### Note: decode_responses must be set to True when used with python3
>>> rc = StrictRedisCluster(startup_nodes=startup_nodes, decode_responses=True)
>>> rc.set("foo", "bar")
True
>>> print(rc.get("foo"))
'bar'
----------------------