经过半个月的学习,自己总结了下Redis集群创建,以及相关的节点操作
因为这里主要使用redis-cluster的命令,所以选择redis5.0以上版本(5.0以下版本有些redis-cluster命令不支持,需要redis-ruby工具的支持)。
下载redis5.0以上版本
解压
tar –zxvf redis-5.0.5.tar.gz
编译redis
cd redis-5.0.5
make
创建必要目录
mkdir /usr/local/reids
cp redis-server /usr/local/bin
cp redis-benchmark /usr/local/bin
cp redis-check-rdb /usr/local/bin
cp redis-sentinel /usr/local/bin
cp redis-cli /usr/local/bin
cp redis.conf /usr/local/bin
mkdir /usr/local/cluster/7000
mkdir /usr/local/cluster/7000/data
cp redis-5.0.5/redis.conf /usr/local/cluster/7000/
编辑redis.conf
port:7000
cluster-enabled yes
cluster-config-file /usr/local/cluster/7000/data/node.conf
pidfile /var/run/redis_7000.pid
dir /usr/local/bin
同理创建7001、7002、8000、8001、8002
cp /usr/local/cluster/7000/ redis.conf /usr/local/cluster/7001
port:7001
cluster-enabled yes
cluster-config-file /usr/local/cluster/7001/data/node.conf
pidfile /var/run/redis_7001.pid
dir /usr/local/bin
…
创建6个节点
redis-server ./cluster/7000/redis.conf &
redis-server ./cluster/7001/redis.conf &
redis-server ./cluster/7002/redis.conf &
edis-server ./cluster/8000/redis.conf &
redis-server ./cluster/8001/redis.conf &
redis-server ./cluster/8002/redis.conf &
使用客户端redis-cli查看集群状态
[root@centos7 cluster]# redis-cli -c -h 127.0.0.1 -p 7000
127.0.0.1:7000> cluster info
cluster_state:fail
cluster_slots_assigned:0
cluster_slots_ok:0
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:1
cluster_size:0
cluster_current_epoch:0
cluster_my_epoch:0
cluster_stats_messages_sent:0
cluster_stats_messages_received:0
使用客户端redis-cli查看集群节点
127.0.0.1:7000> cluster nodes
fa96dbb2ea63f16f1bc8f8315386b183d8f77abf :7000@17000 myself,master - 0 0 0 connected
关联各个节点
[root@centos7 cluster]# redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:8000 127.0.0.1:8001 127.0.0.1:8002 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:8001 to 127.0.0.1:7000
Adding replica 127.0.0.1:8002 to 127.0.0.1:7001
Adding replica 127.0.0.1:8000 to 127.0.0.1:7002
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 127.0.0.1:7000
slots:[0-5460] (5461 slots) master
M: c6f8c708f7be74280c15eb6e744c18c3fa697776 127.0.0.1:7001
slots:[5461-10922] (5462 slots) master
M: 2205895bf212f0e6f02ea5d119c9ff5ddb4344e3 127.0.0.1:7002
slots:[10923-16383] (5461 slots) master
S: f111aa110aa32a6ead47858131f9836747a8e5a4 127.0.0.1:8000
replicates 2205895bf212f0e6f02ea5d119c9ff5ddb4344e3
S: 41359b81cd266c9faadb243bad979bbae0307a17 127.0.0.1:8001
replicates fa96dbb2ea63f16f1bc8f8315386b183d8f77abf
S: 10c3ab97d14e90f688adaa3eb36165176d2996c4 127.0.0.1:8002
replicates c6f8c708f7be74280c15eb6e744c18c3fa697776
Can I set the above configuration? (type 'yes' to accept):
键入:yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
51117:M 20 Jul 2019 23:34:32.773 # configEpoch set to 1 via CLUSTER SET-CONFIG-EPOCH
51125:M 20 Jul 2019 23:34:32.773 # configEpoch set to 2 via CLUSTER SET-CONFIG-EPOCH
51133:M 20 Jul 2019 23:34:32.773 # configEpoch set to 3 via CLUSTER SET-CONFIG-EPOCH
51154:M 20 Jul 2019 23:34:32.774 # configEpoch set to 4 via CLUSTER SET-CONFIG-EPOCH
51162:M 20 Jul 2019 23:34:32.774 # configEpoch set to 5 via CLUSTER SET-CONFIG-EPOCH
51171:M 20 Jul 2019 23:34:32.774 # configEpoch set to 6 via CLUSTER SET-CONFIG-EPOCH
>>> Sending CLUSTER MEET messages to join the cluster
51117:M 20 Jul 2019 23:34:32.843 # IP address for this node updated to 127.0.0.1
51154:M 20 Jul 2019 23:34:32.868 # IP address for this node updated to 127.0.0.1
51162:M 20 Jul 2019 23:34:32.870 # IP address for this node updated to 127.0.0.1
51171:M 20 Jul 2019 23:34:32.871 # IP address for this node updated to 127.0.0.1
51125:M 20 Jul 2019 23:34:32.873 # IP address for this node updated to 127.0.0.1
51133:M 20 Jul 2019 23:34:32.877 # IP address for this node updated to 127.0.0.1
Waiting for the cluster to join
....51133:M 20 Jul 2019 23:34:37.793 # Cluster state changed: ok
51154:S 20 Jul 2019 23:34:37.810 * Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
51154:S 20 Jul 2019 23:34:37.810 # Cluster state changed: ok
51162:S 20 Jul 2019 23:34:37.812 * Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
51162:S 20 Jul 2019 23:34:37.812 # Cluster state changed: ok
51117:M 20 Jul 2019 23:34:37.815 # Cluster state changed: ok
51171:S 20 Jul 2019 23:34:37.816 * Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
51171:S 20 Jul 2019 23:34:37.816 # Cluster state changed: ok
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 127.0.0.1:7000
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: c6f8c708f7be74280c15eb6e744c18c3fa697776 127.0.0.1:7001
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
M: 2205895bf212f0e6f02ea5d119c9ff5ddb4344e3 127.0.0.1:7002
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 10c3ab97d14e90f688adaa3eb36165176d2996c4 127.0.0.1:8002
slots: (0 slots) slave
replicates c6f8c708f7be74280c15eb6e744c18c3fa697776
S: f111aa110aa32a6ead47858131f9836747a8e5a4 127.0.0.1:8000
slots: (0 slots) slave
replicates 2205895bf212f0e6f02ea5d119c9ff5ddb4344e3
S: 41359b81cd266c9faadb243bad979bbae0307a17 127.0.0.1:8001
slots: (0 slots) slave
replicates fa96dbb2ea63f16f1bc8f8315386b183d8f77abf
51125:M 20 Jul 2019 23:34:37.836 # Cluster state changed: ok
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@centos7 cluster]# 51162:S 20 Jul 2019 23:34:38.138 * Connecting to MASTER 127.0.0.1:7000
51162:S 20 Jul 2019 23:34:38.138 * MASTER <-> REPLICA sync started
51162:S 20 Jul 2019 23:34:38.139 * Non blocking connect for SYNC fired the event.
51162:S 20 Jul 2019 23:34:38.139 * Master replied to PING, replication can continue...
51162:S 20 Jul 2019 23:34:38.139 * Trying a partial resynchronization (request 6e5530f70a818ca64770e0fbbebba6b551975566:1).
51117:M 20 Jul 2019 23:34:38.139 * Replica 127.0.0.1:8001 asks for synchronization
51117:M 20 Jul 2019 23:34:38.139 * Partial resynchronization not accepted: Replication ID mismatch (Replica asked for '6e5530f70a818ca64770e0fbbebba6b551975566', my replication IDs are '512a0e3fc18ac6c82e17facdb2a87b53eff3428e' and '0000000000000000000000000000000000000000')
51117:M 20 Jul 2019 23:34:38.139 * Starting BGSAVE for SYNC with target: disk
51117:M 20 Jul 2019 23:34:38.140 * Background saving started by pid 51674
51162:S 20 Jul 2019 23:34:38.140 * Full resync from master: e0e7c98ac2f10cc90e941eeb072c9f6c214dc3db:0
51162:S 20 Jul 2019 23:34:38.140 * Discarding previously cached master state.
51674:C 20 Jul 2019 23:34:38.142 * DB saved on disk
51674:C 20 Jul 2019 23:34:38.142 * RDB: 2 MB of memory used by copy-on-write
51154:S 20 Jul 2019 23:34:38.194 * Connecting to MASTER 127.0.0.1:7002
51154:S 20 Jul 2019 23:34:38.194 * MASTER <-> REPLICA sync started
51154:S 20 Jul 2019 23:34:38.194 * Non blocking connect for SYNC fired the event.
51154:S 20 Jul 2019 23:34:38.194 * Master replied to PING, replication can continue...
51154:S 20 Jul 2019 23:34:38.194 * Trying a partial resynchronization (request 7b8d7e52220026011c44e8764a69da4356d5d9ee:1).
51133:M 20 Jul 2019 23:34:38.194 * Replica 127.0.0.1:8000 asks for synchronization
51133:M 20 Jul 2019 23:34:38.194 * Partial resynchronization not accepted: Replication ID mismatch (Replica asked for '7b8d7e52220026011c44e8764a69da4356d5d9ee', my replication IDs are 'caf5671ce79d2942ed5e58389fc4eb2277d0e075' and '0000000000000000000000000000000000000000')
51133:M 20 Jul 2019 23:34:38.194 * Starting BGSAVE for SYNC with target: disk
51133:M 20 Jul 2019 23:34:38.195 * Background saving started by pid 51675
51154:S 20 Jul 2019 23:34:38.196 * Full resync from master: be79475f78131e9c067b4d0c765925ad30f89819:0
51154:S 20 Jul 2019 23:34:38.196 * Discarding previously cached master state.
51675:C 20 Jul 2019 23:34:38.198 * DB saved on disk
51675:C 20 Jul 2019 23:34:38.199 * RDB: 4 MB of memory used by copy-on-write
51117:M 20 Jul 2019 23:34:38.219 * Background saving terminated with success
51117:M 20 Jul 2019 23:34:38.219 * Synchronization with replica 127.0.0.1:8001 succeeded
51162:S 20 Jul 2019 23:34:38.219 * MASTER <-> REPLICA sync: receiving 175 bytes from master
51162:S 20 Jul 2019 23:34:38.220 * MASTER <-> REPLICA sync: Flushing old data
51162:S 20 Jul 2019 23:34:38.220 * MASTER <-> REPLICA sync: Loading DB in memory
51162:S 20 Jul 2019 23:34:38.220 * MASTER <-> REPLICA sync: Finished with success
51162:S 20 Jul 2019 23:34:38.221 * Background append only file rewriting started by pid 51676
51162:S 20 Jul 2019 23:34:38.262 * AOF rewrite child asks to stop sending diffs.
51676:C 20 Jul 2019 23:34:38.263 * Parent agreed to stop sending diffs. Finalizing AOF...
51676:C 20 Jul 2019 23:34:38.263 * Concatenating 0.00 MB of AOF diff received from parent.
51676:C 20 Jul 2019 23:34:38.263 * SYNC append only file rewrite performed
51676:C 20 Jul 2019 23:34:38.265 * AOF rewrite: 4 MB of memory used by copy-on-write
51133:M 20 Jul 2019 23:34:38.296 * Background saving terminated with success
51133:M 20 Jul 2019 23:34:38.296 * Synchronization with replica 127.0.0.1:8000 succeeded
51154:S 20 Jul 2019 23:34:38.297 * MASTER <-> REPLICA sync: receiving 175 bytes from master
51154:S 20 Jul 2019 23:34:38.297 * MASTER <-> REPLICA sync: Flushing old data
51154:S 20 Jul 2019 23:34:38.297 * MASTER <-> REPLICA sync: Loading DB in memory
51154:S 20 Jul 2019 23:34:38.297 * MASTER <-> REPLICA sync: Finished with success
51154:S 20 Jul 2019 23:34:38.299 * Background append only file rewriting started by pid 51677
51162:S 20 Jul 2019 23:34:38.340 * Background AOF rewrite terminated with success
51162:S 20 Jul 2019 23:34:38.340 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
51162:S 20 Jul 2019 23:34:38.340 * Background AOF rewrite finished successfully
51154:S 20 Jul 2019 23:34:38.348 * AOF rewrite child asks to stop sending diffs.
51677:C 20 Jul 2019 23:34:38.348 * Parent agreed to stop sending diffs. Finalizing AOF...
51677:C 20 Jul 2019 23:34:38.348 * Concatenating 0.00 MB of AOF diff received from parent.
51677:C 20 Jul 2019 23:34:38.349 * SYNC append only file rewrite performed
51677:C 20 Jul 2019 23:34:38.350 * AOF rewrite: 4 MB of memory used by copy-on-write
51154:S 20 Jul 2019 23:34:38.397 * Background AOF rewrite terminated with success
51154:S 20 Jul 2019 23:34:38.397 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
51154:S 20 Jul 2019 23:34:38.397 * Background AOF rewrite finished successfully
51171:S 20 Jul 2019 23:34:38.499 * Connecting to MASTER 127.0.0.1:7001
51171:S 20 Jul 2019 23:34:38.501 * MASTER <-> REPLICA sync started
51171:S 20 Jul 2019 23:34:38.502 * Non blocking connect for SYNC fired the event.
51171:S 20 Jul 2019 23:34:38.505 * Master replied to PING, replication can continue...
51171:S 20 Jul 2019 23:34:38.506 * Trying a partial resynchronization (request 3338a75734915ae4ac66f8d92c397586269c580b:1).
51125:M 20 Jul 2019 23:34:38.507 * Replica 127.0.0.1:8002 asks for synchronization
51125:M 20 Jul 2019 23:34:38.507 * Partial resynchronization not accepted: Replication ID mismatch (Replica asked for '3338a75734915ae4ac66f8d92c397586269c580b', my replication IDs are '93d42b1d560f630f03fe372c1b6e7871b3ffb3a9' and '0000000000000000000000000000000000000000')
51125:M 20 Jul 2019 23:34:38.507 * Starting BGSAVE for SYNC with target: disk
51125:M 20 Jul 2019 23:34:38.509 * Background saving started by pid 51679
51171:S 20 Jul 2019 23:34:38.510 * Full resync from master: 0937b53a5c14f0a112afb9256cadd43540433963:0
51171:S 20 Jul 2019 23:34:38.510 * Discarding previously cached master state.
51679:C 20 Jul 2019 23:34:38.516 * DB saved on disk
51679:C 20 Jul 2019 23:34:38.519 * RDB: 2 MB of memory used by copy-on-write
51125:M 20 Jul 2019 23:34:38.544 * Background saving terminated with success
51125:M 20 Jul 2019 23:34:38.544 * Synchronization with replica 127.0.0.1:8002 succeeded
51171:S 20 Jul 2019 23:34:38.544 * MASTER <-> REPLICA sync: receiving 175 bytes from master
51171:S 20 Jul 2019 23:34:38.545 * MASTER <-> REPLICA sync: Flushing old data
51171:S 20 Jul 2019 23:34:38.545 * MASTER <-> REPLICA sync: Loading DB in memory
51171:S 20 Jul 2019 23:34:38.545 * MASTER <-> REPLICA sync: Finished with success
51171:S 20 Jul 2019 23:34:38.547 * Background append only file rewriting started by pid 51680
51171:S 20 Jul 2019 23:34:38.590 * AOF rewrite child asks to stop sending diffs.
51680:C 20 Jul 2019 23:34:38.590 * Parent agreed to stop sending diffs. Finalizing AOF...
51680:C 20 Jul 2019 23:34:38.591 * Concatenating 0.00 MB of AOF diff received from parent.
51680:C 20 Jul 2019 23:34:38.592 * SYNC append only file rewrite performed
51680:C 20 Jul 2019 23:34:38.594 * AOF rewrite: 4 MB of memory used by copy-on-write
51171:S 20 Jul 2019 23:34:38.602 * Background AOF rewrite terminated with success
51171:S 20 Jul 2019 23:34:38.602 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
51171:S 20 Jul 2019 23:34:38.603 * Background AOF rewrite finished successfully
集群创建成功,使用客户端redis-cli查看集群状态
[root@centos7 cluster]# redis-cli -c -h 127.0.0.1 -p 7000
127.0.0.1:7000> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:115
cluster_stats_messages_pong_sent:118
cluster_stats_messages_sent:233
cluster_stats_messages_ping_received:113
cluster_stats_messages_pong_received:115
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:233
查看节点信息
127.0.0.1:7000> cluster nodes
c6f8c708f7be74280c15eb6e744c18c3fa697776 127.0.0.1:7001@17001 master - 0 1563680220000 2 connected 5461-10922
2205895bf212f0e6f02ea5d119c9ff5ddb4344e3 127.0.0.1:7002@17002 master - 0 1563680223000 3 connected 10923-16383
10c3ab97d14e90f688adaa3eb36165176d2996c4 127.0.0.1:8002@18002 slave c6f8c708f7be74280c15eb6e744c18c3fa697776 0 1563680221983 6 connected
f111aa110aa32a6ead47858131f9836747a8e5a4 127.0.0.1:8000@18000 slave 2205895bf212f0e6f02ea5d119c9ff5ddb4344e3 0 1563680220973 4 connected
41359b81cd266c9faadb243bad979bbae0307a17 127.0.0.1:8001@18001 slave fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 0 1563680221000 5 connected
fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 127.0.0.1:7000@17000 myself,master - 0 1563680222000 1 connected 0-5460
将节点7003加入到集群中
[root@centos7 cluster]# redis-cli --cluster add-node 127.0.0.1:7003 127.0.0.1:7000
>>> Adding node 127.0.0.1:7003 to cluster 127.0.0.1:7000
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 127.0.0.1:7000
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: c6f8c708f7be74280c15eb6e744c18c3fa697776 127.0.0.1:7001
slots: (0 slots) slave
replicates 10c3ab97d14e90f688adaa3eb36165176d2996c4
S: 2205895bf212f0e6f02ea5d119c9ff5ddb4344e3 127.0.0.1:7002
slots: (0 slots) slave
replicates f111aa110aa32a6ead47858131f9836747a8e5a4
M: 10c3ab97d14e90f688adaa3eb36165176d2996c4 127.0.0.1:8002
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
M: f111aa110aa32a6ead47858131f9836747a8e5a4 127.0.0.1:8000
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 41359b81cd266c9faadb243bad979bbae0307a17 127.0.0.1:8001
slots: (0 slots) slave
replicates fa96dbb2ea63f16f1bc8f8315386b183d8f77abf
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 127.0.0.1:7003 to make it join the cluster.
[OK] New node added correctly.
查看集群状态
[root@centos7 cluster]# redis-cli -p 7000
127.0.0.1:7000> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:7
cluster_size:3
cluster_current_epoch:8
cluster_my_epoch:1
cluster_stats_messages_ping_sent:8101
cluster_stats_messages_pong_sent:8420
cluster_stats_messages_auth-ack_sent:2
cluster_stats_messages_update_sent:1
cluster_stats_messages_sent:16524
cluster_stats_messages_ping_received:8414
cluster_stats_messages_pong_received:8098
cluster_stats_messages_meet_received:6
cluster_stats_messages_fail_received:2
cluster_stats_messages_auth-req_received:2
cluster_stats_messages_received:16522
查看节点信息,
新加入的7003为主节点,没有分配槽
127.0.0.1:7000> cluster nodes
c6f8c708f7be74280c15eb6e744c18c3fa697776 127.0.0.1:7001@17001 slave 10c3ab97d14e90f688adaa3eb36165176d2996c4 0 1563690713000 8 connected
2205895bf212f0e6f02ea5d119c9ff5ddb4344e3 127.0.0.1:7002@17002 slave f111aa110aa32a6ead47858131f9836747a8e5a4 0 1563690713617 7 connected
10c3ab97d14e90f688adaa3eb36165176d2996c4 127.0.0.1:8002@18002 master - 0 1563690715660 8 connected 5461-10922
f111aa110aa32a6ead47858131f9836747a8e5a4 127.0.0.1:8000@18000 master - 0 1563690714000 7 connected 10923-16383
41359b81cd266c9faadb243bad979bbae0307a17 127.0.0.1:8001@18001 slave fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 0 1563690710000 5 connected
fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 127.0.0.1:7000@17000 myself,master - 0 1563690715000 1 connected 0-5460
3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57 127.0.0.1:7003@17003 master - 0 1563690714644 0 connected
为节点7003分配槽,这里我们分配1000个槽
[root@centos7 cluster]# redis-cli --cluster reshard 127.0.0.1:7003
>>> Performing Cluster Check (using node 127.0.0.1:7003)
M: 3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57 127.0.0.1:7003
slots: (0 slots) master
S: 2205895bf212f0e6f02ea5d119c9ff5ddb4344e3 127.0.0.1:7002
slots: (0 slots) slave
replicates f111aa110aa32a6ead47858131f9836747a8e5a4
M: 10c3ab97d14e90f688adaa3eb36165176d2996c4 127.0.0.1:8002
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: c6f8c708f7be74280c15eb6e744c18c3fa697776 127.0.0.1:7001
slots: (0 slots) slave
replicates 10c3ab97d14e90f688adaa3eb36165176d2996c4
M: fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 127.0.0.1:7000
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: f111aa110aa32a6ead47858131f9836747a8e5a4 127.0.0.1:8000
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 41359b81cd266c9faadb243bad979bbae0307a17 127.0.0.1:8001
slots: (0 slots) slave
replicates fa96dbb2ea63f16f1bc8f8315386b183d8f77abf
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)?
要分配多少个槽,这里我们选择1000个槽
How many slots do you want to move (from 1 to 16384)? 1000
What is the receiving node ID?
接受槽的节点ID,也就是新增节点7003的ID:3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57
What is the receiving node ID? 3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1:
第一个源节点id,也就是我们要引入槽所在的节点id,这里我们先填入节点7000的id:fa96dbb2ea63f16f1bc8f8315386b183d8f77abf
Source node #1: fa96dbb2ea63f16f1bc8f8315386b183d8f77abf
Source node #2:
第二个源节点id,我们填入节点8000的id:f111aa110aa32a6ead47858131f9836747a8e5a4
Source node #2: f111aa110aa32a6ead47858131f9836747a8e5a4
Source node #3:
第三个源节点id,我们填入节点8002的id:10c3ab97d14e90f688adaa3eb36165176d2996c4
Source node #3: 10c3ab97d14e90f688adaa3eb36165176d2996c4
Source node #4:
这里提示第四个节点id,其实我们只有三个主节点,所以槽的来源最多也只能是3个节点了,所以我填入done,让服务器开始分配槽
Source node #4:done
Ready to move 1000 slots.
Source nodes:
M: fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 127.0.0.1:7000
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: f111aa110aa32a6ead47858131f9836747a8e5a4 127.0.0.1:8000
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
M: 10c3ab97d14e90f688adaa3eb36165176d2996c4 127.0.0.1:8002
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
Destination node:
M: 3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57 127.0.0.1:7003
slots: (0 slots) master
Resharding plan:
Moving slot 5461 from 10c3ab97d14e90f688adaa3eb36165176d2996c4
Moving slot 5462 from 10c3ab97d14e90f688adaa3eb36165176d2996c4
Moving slot 5463 from 10c3ab97d14e90f688adaa3eb36165176d2996c4
Moving slot 5464 from 10c3ab97d14e90f688adaa3eb36165176d2996c4
Moving slot 5465 from 10c3ab97d14e90f688adaa3eb36165176d2996c4
Moving slot 5466 from 10c3ab97d14e90f688adaa3eb36165176d2996c4
Moving slot 5467 from 10c3ab97d14e90f688adaa3eb36165176d2996c4
Moving slot 5468 from 10c3ab97d14e90f688adaa3eb36165176d2996c4
….
Moving slot 11255 from f111aa110aa32a6ead47858131f9836747a8e5a4
Do you want to proceed with the proposed reshard plan (yes/no)?
最后确认执行分配计划,yes
Moving slot 5461 from 127.0.0.1:8002 to 127.0.0.1:7003:
Moving slot 5462 from 127.0.0.1:8002 to 127.0.0.1:7003:
Moving slot 5463 from 127.0.0.1:8002 to 127.0.0.1:7003:
Moving slot 5464 from 127.0.0.1:8002 to 127.0.0.1:7003:
…
我们发现节点7000 ,节点8000 节点8002 不断的迁移槽到节点7003上
槽已经重新分配完了,我们执行cluster info查看节点状态
[root@centos7 cluster]# redis-cli -p 7000
127.0.0.1:7000> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:7
cluster_size:4
cluster_current_epoch:9
cluster_my_epoch:1
cluster_stats_messages_ping_sent:9511
cluster_stats_messages_pong_sent:9842
cluster_stats_messages_auth-ack_sent:2
cluster_stats_messages_update_sent:8
cluster_stats_messages_sent:19363
cluster_stats_messages_ping_received:9836
cluster_stats_messages_pong_received:9508
cluster_stats_messages_meet_received:6
cluster_stats_messages_fail_received:2
cluster_stats_messages_auth-req_received:2
cluster_stats_messages_received:19354
查看节点信息 cluster nodes
127.0.0.1:7000> cluster nodes
c6f8c708f7be74280c15eb6e744c18c3fa697776 127.0.0.1:7001@17001 slave 10c3ab97d14e90f688adaa3eb36165176d2996c4 0 1563692116650 8 connected
2205895bf212f0e6f02ea5d119c9ff5ddb4344e3 127.0.0.1:7002@17002 slave f111aa110aa32a6ead47858131f9836747a8e5a4 0 1563692113000 7 connected
10c3ab97d14e90f688adaa3eb36165176d2996c4 127.0.0.1:8002@18002 master - 0 1563692115000 8 connected 5795-10922
f111aa110aa32a6ead47858131f9836747a8e5a4 127.0.0.1:8000@18000 master - 0 1563692114000 7 connected 11256-16383
41359b81cd266c9faadb243bad979bbae0307a17 127.0.0.1:8001@18001 slave fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 0 1563692112000 5 connected
fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 127.0.0.1:7000@17000 myself,master - 0 1563692112000 1 connected 333-5460
3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57 127.0.0.1:7003@17003 master - 0 1563692115643 9 connected 0-332 5461-5794 10923-11255
至此,我们发现槽0-332 5461-5794 10923-11255分配给了新节点7003
添加节点8003,作为7003的从节点
[root@centos7 cluster]# redis-cli --cluster add-node 127.0.0.1:8003 127.0.0.1:7003 --cluster-slave
>>> Adding node 127.0.0.1:8003 to cluster 127.0.0.1:7003
>>> Performing Cluster Check (using node 127.0.0.1:7003)
M: 3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57 127.0.0.1:7003
slots:[0-332],[5461-5794],[10923-11255] (1000 slots) master
S: 2205895bf212f0e6f02ea5d119c9ff5ddb4344e3 127.0.0.1:7002
slots: (0 slots) slave
replicates f111aa110aa32a6ead47858131f9836747a8e5a4
M: 10c3ab97d14e90f688adaa3eb36165176d2996c4 127.0.0.1:8002
slots:[5795-10922] (5128 slots) master
1 additional replica(s)
S: c6f8c708f7be74280c15eb6e744c18c3fa697776 127.0.0.1:7001
slots: (0 slots) slave
replicates 10c3ab97d14e90f688adaa3eb36165176d2996c4
M: fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 127.0.0.1:7000
slots:[333-5460] (5128 slots) master
1 additional replica(s)
M: f111aa110aa32a6ead47858131f9836747a8e5a4 127.0.0.1:8000
slots:[11256-16383] (5128 slots) master
1 additional replica(s)
S: 41359b81cd266c9faadb243bad979bbae0307a17 127.0.0.1:8001
slots: (0 slots) slave
replicates fa96dbb2ea63f16f1bc8f8315386b183d8f77abf
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
Automatically selected master 127.0.0.1:7003
>>> Send CLUSTER MEET to node 127.0.0.1:8003 to make it join the cluster.
Waiting for the cluster to join
>>> Configure node as replica of 127.0.0.1:7003.
[OK] New node added correctly.
查看节点状态信息
[root@centos7 cluster]# redis-cli -p 7000
127.0.0.1:7000> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:8
cluster_size:4
cluster_current_epoch:9
cluster_my_epoch:1
cluster_stats_messages_ping_sent:9914
cluster_stats_messages_pong_sent:10246
cluster_stats_messages_meet_sent:1
cluster_stats_messages_auth-ack_sent:2
cluster_stats_messages_update_sent:8
cluster_stats_messages_sent:20171
cluster_stats_messages_ping_received:10239
cluster_stats_messages_pong_received:9912
cluster_stats_messages_meet_received:7
cluster_stats_messages_fail_received:2
cluster_stats_messages_auth-req_received:2
cluster_stats_messages_received:20162
查看节点信息
127.0.0.1:7000> cluster nodes
624565cbe3905c214b8607f6b64d469e100c8683 127.0.0.1:8003@18003 slave 3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57 0 1563692540000 9 connected
2205895bf212f0e6f02ea5d119c9ff5ddb4344e3 127.0.0.1:7002@17002 slave f111aa110aa32a6ead47858131f9836747a8e5a4 0 1563692541000 7 connected
10c3ab97d14e90f688adaa3eb36165176d2996c4 127.0.0.1:8002@18002 master - 0 1563692537815 8 connected 5795-10922
fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 127.0.0.1:7000@17000 myself,master - 0 1563692537000 1 connected 333-5460
3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57 127.0.0.1:7003@17003 master - 0 1563692541868 9 connected 0-332 5461-5794 10923-11255
c6f8c708f7be74280c15eb6e744c18c3fa697776 127.0.0.1:7001@17001 slave 10c3ab97d14e90f688adaa3eb36165176d2996c4 0 1563692539851 8 connected
41359b81cd266c9faadb243bad979bbae0307a17 127.0.0.1:8001@18001 slave fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 0 1563692538000 5 connected
f111aa110aa32a6ead47858131f9836747a8e5a4 127.0.0.1:8000@18000 master - 0 1563692540857 7 connected 11256-16383
至此,我们发现节点8003已经为7003的从节点
分析:
redis-cli –cluster add-node ip:port clusertip:clusterport –cluster-slave
这个命令会将节点ip:port随机添加到集群中的任何一个节点,而不是确定的节点,所以要确定成为某个节点的主节点,必须使用以下命令
redis-cli –cluster add-node addIp:addPort clustertIp:clusterPort –cluster-slave –cluster-master-id
例如上文将8003节点作为7003的从节点,可以使用命令
redis-cli –cluster add-node 127.0.0.1:8003 127.0.0.1:7000 –cluster-slave –cluster-master-id 3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57
方案三
将新节点作为主节点加入到集群中
Redis-cli –cluster add-node addip:addport clusterip:clusertport
然后连接到节点
Redis-cli –c –h
指定新节点为某个节点的从节点
cluster replicate master-id
例如:我们添加节点8004,作为7000的从节点
redis-cli --cluster add-node 127.0.0.1:8004 127.0.0.1:7000
>>> Adding node 127.0.0.1:8004 to cluster 127.0.0.1:7000
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 127.0.0.1:7000
slots:[333-5460] (5128 slots) master
1 additional replica(s)
S: 624565cbe3905c214b8607f6b64d469e100c8683 127.0.0.1:8003
slots: (0 slots) slave
replicates 3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57
S: 2205895bf212f0e6f02ea5d119c9ff5ddb4344e3 127.0.0.1:7002
slots: (0 slots) slave
replicates f111aa110aa32a6ead47858131f9836747a8e5a4
M: 10c3ab97d14e90f688adaa3eb36165176d2996c4 127.0.0.1:8002
slots:[5795-10922] (5128 slots) master
1 additional replica(s)
M: 3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57 127.0.0.1:7003
slots:[0-332],[5461-5794],[10923-11255] (1000 slots) master
1 additional replica(s)
S: c6f8c708f7be74280c15eb6e744c18c3fa697776 127.0.0.1:7001
slots: (0 slots) slave
replicates 10c3ab97d14e90f688adaa3eb36165176d2996c4
S: 41359b81cd266c9faadb243bad979bbae0307a17 127.0.0.1:8001
slots: (0 slots) slave
replicates fa96dbb2ea63f16f1bc8f8315386b183d8f77abf
M: f111aa110aa32a6ead47858131f9836747a8e5a4 127.0.0.1:8000
slots:[11256-16383] (5128 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 127.0.0.1:8004 to make it join the cluster.
[OK] New node added correctly.
[root@centos7 cluster]# 61776:M 21 Jul 2019 03:25:24.548 # IP address for this node updated to 127.0.0.1
61776:M 21 Jul 2019 03:25:29.629 # Cluster state changed: ok
查看节点8004,此时节点8004为未分配槽的主节点
redis-cli -p 7000
127.0.0.1:7000> cluster-nodes
(error) ERR unknown command `cluster-nodes`, with args beginning with:
127.0.0.1:7000> cluster nodes
624565cbe3905c214b8607f6b64d469e100c8683 127.0.0.1:8003@18003 slave 3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57 0 1563694000370 9 connected
2205895bf212f0e6f02ea5d119c9ff5ddb4344e3 127.0.0.1:7002@17002 slave f111aa110aa32a6ead47858131f9836747a8e5a4 0 1563693998339 7 connected
707d49c11fb1d7ccc7f8b8ea183a8e0261b4166a 127.0.0.1:8004@18004 master - 0 1563693997320 0 connected
10c3ab97d14e90f688adaa3eb36165176d2996c4 127.0.0.1:8002@18002 master - 0 1563693996000 8 connected 5795-10922
fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 127.0.0.1:7000@17000 myself,master - 0 1563693993000 1 connected 333-5460
3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57 127.0.0.1:7003@17003 master - 0 1563693999355 9 connected 0-332 5461-5794 10923-11255
c6f8c708f7be74280c15eb6e744c18c3fa697776 127.0.0.1:7001@17001 slave 10c3ab97d14e90f688adaa3eb36165176d2996c4 0 1563693996000 8 connected
41359b81cd266c9faadb243bad979bbae0307a17 127.0.0.1:8001@18001 slave fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 0 1563693997000 5 connected
f111aa110aa32a6ead47858131f9836747a8e5a4 127.0.0.1:8000@18000 master - 0 1563693997000 7 connected 11256-16383
将主节点8004作为7000的从节点
[root@centos7 cluster]# redis-cli -p 8004
127.0.0.1:8004> cluster replicate fa96dbb2ea63f16f1bc8f8315386b183d8f77abf
61776:S 21 Jul 2019 03:28:51.738 * Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
OK
127.0.0.1:8004> 61776:S 21 Jul 2019 03:28:52.199 * Connecting to MASTER 127.0.0.1:7000
61776:S 21 Jul 2019 03:28:52.199 * MASTER <-> REPLICA sync started
61776:S 21 Jul 2019 03:28:52.199 * Non blocking connect for SYNC fired the event.
61776:S 21 Jul 2019 03:28:52.201 * Master replied to PING, replication can continue...
61776:S 21 Jul 2019 03:28:52.202 * Trying a partial resynchronization (request 78bffb5248811bafcd4e4308a6e42a9cc8447587:1).
61776:S 21 Jul 2019 03:28:52.333 * Full resync from master: e0e7c98ac2f10cc90e941eeb072c9f6c214dc3db:16044
61776:S 21 Jul 2019 03:28:52.333 * Discarding previously cached master state.
61776:S 21 Jul 2019 03:28:52.333 * MASTER <-> REPLICA sync: receiving 176 bytes from master
61776:S 21 Jul 2019 03:28:52.333 * MASTER <-> REPLICA sync: Flushing old data
61776:S 21 Jul 2019 03:28:52.333 * MASTER <-> REPLICA sync: Loading DB in memory
61776:S 21 Jul 2019 03:28:52.333 * MASTER <-> REPLICA sync: Finished with success
61776:S 21 Jul 2019 03:28:52.334 * Background append only file rewriting started by pid 62404
61776:S 21 Jul 2019 03:28:52.388 * AOF rewrite child asks to stop sending diffs.
62404:C 21 Jul 2019 03:28:52.389 * Parent agreed to stop sending diffs. Finalizing AOF...
62404:C 21 Jul 2019 03:28:52.389 * Concatenating 0.00 MB of AOF diff received from parent.
62404:C 21 Jul 2019 03:28:52.389 * SYNC append only file rewrite performed
62404:C 21 Jul 2019 03:28:52.389 * AOF rewrite: 4 MB of memory used by copy-on-write
61776:S 21 Jul 2019 03:28:52.415 * Background AOF rewrite terminated with success
61776:S 21 Jul 2019 03:28:52.415 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
61776:S 21 Jul 2019 03:28:52.415 * Background AOF rewrite finished successfully
查看节点信息
[root@centos7 cluster]# redis-cli -p 7000
127.0.0.1:7000> cluster nodes
624565cbe3905c214b8607f6b64d469e100c8683 127.0.0.1:8003@18003 slave 3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57 0 1563694180753 9 connected
2205895bf212f0e6f02ea5d119c9ff5ddb4344e3 127.0.0.1:7002@17002 slave f111aa110aa32a6ead47858131f9836747a8e5a4 0 1563694180000 7 connected
707d49c11fb1d7ccc7f8b8ea183a8e0261b4166a 127.0.0.1:8004@18004 slave fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 0 1563694177713 1 connected
10c3ab97d14e90f688adaa3eb36165176d2996c4 127.0.0.1:8002@18002 master - 0 1563694178000 8 connected 5795-10922
fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 127.0.0.1:7000@17000 myself,master - 0 1563694178000 1 connected 333-5460
3dfb41469d6da4b3d215e4f85a7b0e7fba67cd57 127.0.0.1:7003@17003 master - 0 1563694179742 9 connected 0-332 5461-5794 10923-11255
c6f8c708f7be74280c15eb6e744c18c3fa697776 127.0.0.1:7001@17001 slave 10c3ab97d14e90f688adaa3eb36165176d2996c4 0 1563694181762 8 connected
41359b81cd266c9faadb243bad979bbae0307a17 127.0.0.1:8001@18001 slave fa96dbb2ea63f16f1bc8f8315386b183d8f77abf 0 1563694177000 5 connected
f111aa110aa32a6ead47858131f9836747a8e5a4 127.0.0.1:8000@18000 master - 0 1563694179000 7 connected 11256-16383
至此,我们发现8004已经为7000的从节点
Redis-cli –cluster del-node clusterip:clusterport del-id
因为从节点上没有分配槽,所以可以直接删除
例如删除从节点8004
[root@centos7 cluster]# redis-cli --cluster del-node 127.0.0.1:8004 fa96dbb2ea63f16f1bc8f8315386b183d8f77abf
>>> Removing node fa96dbb2ea63f16f1bc8f8315386b183d8f77abf from cluster 127.0.0.1:8004
[ERR] Node 127.0.0.1:7000 is not empty! Reshard data away and try again.
[root@centos7 cluster]# redis-cli --cluster del-node 127.0.0.1:7000 707d49c11fb1d7ccc7f8b8ea183a8e0261b4166a
>>> Removing node 707d49c11fb1d7ccc7f8b8ea183a8e0261b4166a from cluster 127.0.0.1:7000
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
61776:S 21 Jul 2019 03:52:12.508 # User requested shutdown...
61776:S 21 Jul 2019 03:52:12.508 * Calling fsync() on the AOF file.
61776:S 21 Jul 2019 03:52:12.508 * Saving the final RDB snapshot before exiting.
61776:S 21 Jul 2019 03:52:12.509 * DB saved on disk
61776:S 21 Jul 2019 03:52:12.509 * Removing the pid file.
61776:S 21 Jul 2019 03:52:12.510 # Redis is now ready to exit, bye bye...
[1]+ Done redis-server ./8004/redis.conf
由于删除主节点上分配的槽不为空,所以要删除主节点,首先要通过reshard命令将待删除的节点分配到其他主节点之后才能执行节点的删除动作。