Redis 全实践（超长文预警）
一：Redis 安装
1.1：yum 安装
1.1.1：安装前准备和查看
安装 epel 源
yum 安装 Redis 需要准备 epel 源：
yum install epel-release
查看 epel 仓库中的 Redis 版本信息
[root@node106 ~]# yum info redis
Available Packages
Name : redis
Arch : x86_64
Version : 3.2.12
Release : 2.el7
Size : 544 k
Repo : epel/x86_64
Summary : A persistent key-value database
URL : http://redis.io
License : BSD
Description : Redis is an advanced key-value store. It is often referred to as a data
: structure server since keys can contain strings, hashes, lists, sets and
: sorted sets.
:
: You can run atomic operations on these types, like appending to a string;
: incrementing the value in a hash; pushing to a list; computing set
: intersection, union and difference; or getting the member with highest
: ranking in a sorted set.
:
: In order to achieve its outstanding performance, Redis works with an
: in-memory dataset. Depending on your use case, you can persist it either
: by dumping the dataset to disk every once in a while, or by appending
: each command to a log.
:
: Redis also supports trivial-to-setup master-slave replication, with very
: fast non-blocking first synchronization, auto-reconnection on net split
: and so forth.
:
: Other features include Transactions, Pub/Sub, Lua scripting, Keys with a
: limited time-to-live, and configuration settings to make Redis behave like
: a cache.
:
: You can use Redis from most programming languages also.
1.1.2：安装 Redis
yum install redis -y
1.1.3：启动 Redis
systemctl start redis && systemctl enable redis
1.1.4：验证 Redis
查看版本信息
127.0.0.1:6379> INFO Server
# Server
redis_version:3.2.12
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:7897e7d0e13773f
redis_mode:standalone
os:Linux 3.10.0-327.el7.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
gcc_version:4.8.5
process_id:2658
run_id:90205fe7a6bef1e9d05d01996650e1f3026f97f3
tcp_port:6379
uptime_in_seconds:19
uptime_in_days:0
hz:10
lru_clock:15487927
executable:/usr/bin/redis-server
config_file:/etc/redis.conf
查看生成文件
[root@node106 ~]# rpm -ql redis
/etc/logrotate.d/redis
/etc/redis-sentinel.conf
/etc/redis.conf
/etc/systemd/system/redis-sentinel.service.d
/etc/systemd/system/redis-sentinel.service.d/limit.conf
/etc/systemd/system/redis.service.d
/etc/systemd/system/redis.service.d/limit.conf
/usr/bin/redis-benchmark
/usr/bin/redis-check-aof
/usr/bin/redis-check-rdb
/usr/bin/redis-cli
/usr/bin/redis-sentinel
/usr/bin/redis-server
/usr/lib/systemd/system/redis-sentinel.service
/usr/lib/systemd/system/redis.service
/usr/libexec/redis-shutdown
/usr/share/doc/redis-3.2.12
/usr/share/doc/redis-3.2.12/00-RELEASENOTES
/usr/share/doc/redis-3.2.12/BUGS
/usr/share/doc/redis-3.2.12/CONTRIBUTING
/usr/share/doc/redis-3.2.12/MANIFESTO
/usr/share/doc/redis-3.2.12/README.md
/usr/share/licenses/redis-3.2.12
/usr/share/licenses/redis-3.2.12/COPYING
/usr/share/man/man1/redis-benchmark.1.gz
/usr/share/man/man1/redis-check-aof.1.gz
/usr/share/man/man1/redis-check-rdb.1.gz
/usr/share/man/man1/redis-cli.1.gz
/usr/share/man/man1/redis-sentinel.1.gz
/usr/share/man/man1/redis-server.1.gz
/usr/share/man/man5/redis-sentinel.conf.5.gz
/usr/share/man/man5/redis.conf.5.gz
/var/lib/redis
/var/log/redis
/var/run/redis
查看初始配置
[root@node106 ~]# grep '^[a-Z]' /etc/redis.conf
bind 127.0.0.1
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize no
supervised no
pidfile /var/run/redis_6379.pid
loglevel notice
logfile /var/log/redis/redis.log
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /var/lib/redis
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes
查看监听端口
[root@node106 ~]# ss -tnlp | grep 6379
LISTEN 0 511 127.0.0.1:6379 *:* users:(("redis-server",pid=2658,fd=4))
1.2：编译安装
官方编译安装文档：https://redis.io/download
Download, extract and compile Redis with:
$ wget https://download.redis.io/releases/redis-6.0.9.tar.gz $ tar xzf redis-6.0.9.tar.gz $ cd redis-6.0.9 $ makeThe binaries that are now compiled are available in the
srcdirectory. Run Redis with:$ src/redis-serverYou can interact with Redis using the built-in client:
$ src/redis-cli redis> set foo bar OK redis> get foo "bar"
1.2.1：下载并解压 Redis 源码包
下载地址：http://download.redis.io/releases/
[root@node106 ~]# cd /usr/local/src
[root@node106 src]# wget http://download.redis.io/releases/redis-4.0.14.tar.gz
[root@node106 src]# tar xf redis-4.0.14.tar.gz
1.2.2：安装 Redis
[root@node106 src]# cd redis-4.0.14
[root@node106 redis-4.0.14]# make PREFIX=/apps/redis install
1.2.3：启动准备
创建 Redis 用户
[root@node106 ~]# groupadd redis -g 1000 && useradd redis -g 1000 -u 1000 -s /sbin/nologin
准备相关目录及文件
创建配置\日志\数据\运行目录：
[root@node106 ~]# mkdir -pv /apps/redis/{etc,logs,data,run}
拷贝配置文件：
[root@node106 ~]# cp /usr/local/src/redis-4.0.14/redis.conf /apps/redis/etc/
修改目录属主：
[root@node106 ~]# chown redis.redis /apps/redis/ -R
准备启动脚本
[root@node106 ~]# vim /usr/lib/systemd/system/redis.service
[Unit]
Description=Redis persistent key-value database
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
ExecStart=/apps/redis/bin/redis-server /apps/redis/etc/redis.conf --supervised systemd
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
Type=notify
User=redis
Group=redis
RuntimeDirectory=redis
RuntimeDirectoryMode=0755
[Install]
WantedBy=multi-user.target
创建命令软链接
[root@node106 ~]# ln -sv /apps/redis/bin/redis-* /usr/bin/
1.2.4：启动优化
未优化时的告警信息
前台启动 Redis，查看告警：
[root@node106 ~]# /apps/redis/bin/redis-server /apps/redis/etc/redis.conf
3260:C 30 Dec 18:26:51.554 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
3260:C 30 Dec 18:26:51.554 # Redis version=4.0.14, bits=64, commit=00000000, modified=0, pid=3260, just started
3260:C 30 Dec 18:26:51.554 # Configuration loaded
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 4.0.14 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in standalone mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 6379
| `-._ `._ / _.-' | PID: 3260
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
3260:M 30 Dec 18:26:51.559 # Server initialized
3260:M 30 Dec 18:26:51.559 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
3260:M 30 Dec 18:26:51.559 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
3260:M 30 Dec 18:26:51.560 * Ready to accept connections
net.core.somaxconn
TCP backlog 的定义：
backlog 控制的是 TCP 三次握手时 Server 端收到 Client 端的 ACK 确认号之后的队列值。
The backlog argument defines the maximum length to which the queue of pending connections for sockfd may grow. If a connection request arrives when the queue is full, the client may receive an error with an indication of ECONNREFUSED or, if the underlying protocol supports retransmission, the request may be ignored so that a later reattempt at connection succeeds
net.core.somaxconn = 512
在 CentOS 系统初始化中，就设置了
net.core.somaxconn = 20480
vm.overcommit_memory
相关告警信息：
WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
vm.overcommit_memory 取值含义：
- 0：表示内核将检查是否有足够的可用内存供应用进程使用；
- 如果有足够的可用内存，内存申请允许；
- 否则，内存申请失败，并把错误返回给应用进程。
- 1：表示内核允许分配所有的物理内存，而不管当前的内存状态如何。
- 2：表示内核允许分配超过所有物理内存和交换空间总和的内存。
vm.overcommit_memory = 1
hugepage 大页内存
相关告警信息：
WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
关闭大页内存动态分配，让redis 负责内存管理：
echo never > /sys/kernel/mm/transparent_hugepage/enabled
优化内核参数
优化内核参数：
[root@node106 ~]# vim /etc/sysctl.conf
# Redis
#net.core.somaxconn = 512
vm.overcommit_memory = 1
[root@node106 ~]# sysctl -p
系统启动时关闭大页内存动态分配：
[root@node106 ~]# echo never > /sys/kernel/mm/transparent_hugepage/enabled
[root@node106 ~]# vim /etc/rc.local
/usr/bin/echo never > /sys/kernel/mm/transparent_hugepage/enabled
[root@node106 ~]# chmod a+x /etc/rc.local
重新前台启动 Redis，已无告警信息：
[root@node106 ~]# /apps/redis/bin/redis-server /apps/redis/etc/redis.conf
3820:C 30 Dec 18:36:40.494 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
3820:C 30 Dec 18:36:40.494 # Redis version=4.0.14, bits=64, commit=00000000, modified=0, pid=3820, just started
3820:C 30 Dec 18:36:40.494 # Configuration loaded
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 4.0.14 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in standalone mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 6379
| `-._ `._ / _.-' | PID: 3820
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
3820:M 30 Dec 18:36:40.499 # Server initialized
3820:M 30 Dec 18:36:40.499 * Ready to accept connections
1.2.5：启动 Redis
[root@node106 ~]# systemctl daemon-reload
[root@node106 ~]# systemctl enable redis && systemctl start redis
Created symlink from /etc/systemd/system/multi-user.target.wants/redis.service to /usr/lib/systemd/system/redis.service.
1.2.6：验证 Redis
查看版本信息
[root@node106 ~]# redis-cli
127.0.0.1:6379> INFO SERVER
# Server
redis_version:4.0.14
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:431928bb1a7cecbc
redis_mode:standalone
os:Linux 3.10.0-327.el7.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:4.8.5
process_id:3903
run_id:bde1fbfec55af1621fb54fe19a2d8b378a0cd44e
tcp_port:6379
uptime_in_seconds:41
uptime_in_days:0
hz:10
lru_clock:15489162
executable:/apps/redis/bin/redis-server
config_file:/apps/redis/etc/redis.conf
查看初始配置
[root@node106 ~]# grep '^[a-Z]' /usr/local/src/redis-4.0.14/redis.conf
bind 127.0.0.1
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize no
supervised no
pidfile /var/run/redis_6379.pid
loglevel notice
logfile ""
databases 16
always-show-logo yes
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir ./
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
slave-lazy-flush no
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble no
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes
查看监听端口
[root@node106 ~]# ss -tnlp | grep 6379
LISTEN 0 511 *:6379 *:* users:(("redis-server",pid=3903,fd=6))
二：Redis 运维实践
2.1：Redis 数据持久化实验
2.1.1：RDB 模式
配置 RDB save 策略
[root@node106 ~]# vim /usr/local/src/redis-4.0.14/redis.conf
################################ SNAPSHOTTING ################################
save 900 1
save 300 10
save 60 1000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump_6379.rdb
dir /apps/redis/data
重启 Redis
[root@node106 ~]# systemctl restart redis
写入测试数据
[root@node106 ~]# cat redis.sh
#!/bin/bash
NUM=`seq 1 1000`
for i in ${NUM};do
redis-cli -h 127.0.0.1 set key-${i} value-${i} 1>/dev/null
done
echo "数据写入完成"
[root@node106 ~]# bash redis.sh
数据写入完成
127.0.0.1:6379> keys *
……
996) "key-402"
997) "key-807"
998) "key-455"
999) "key-124"
1000) "key-626"
验证 RDB
刚刚的写入操作，符合 save 60 1000 策略，60s 时执行了 bgsave：
[root@node106 ~]# tail -f /apps/redis/logs/redis_6379.log
f14827:M 30 Dec 21:43:36.012 * 1000 changes in 60 seconds. Saving...
14827:M 30 Dec 21:43:36.012 * Background saving started by pid 15886
15886:C 30 Dec 21:43:36.018 * DB saved on disk
15886:C 30 Dec 21:43:36.019 * RDB: 0 MB of memory used by copy-on-write
14827:M 30 Dec 21:43:36.114 * Background saving terminated with success
查看保存的 RDB 文件：
[root@node106 ~]# ll /apps/redis/data/
total 20
-rw-r--r-- 1 redis redis 18885 Dec 30 21:43 dump_6379.rdb
重启 Redis 后，会载入 RDB 文件中的数据：
[root@node106 ~]# systemctl restart redis
[root@node106 ~]# tail -f /apps/redis/logs/redis_6379.log
18248:M 30 Dec 23:20:09.438 * DB loaded from disk: 0.001 seconds
[root@node106 ~]# redis-cli
127.0.0.1:6379> keys *
……
996) "key-402"
997) "key-807"
998) "key-455"
999) "key-124"
1000) "key-626"
2.1.2：AOF 模式
配置 AOF 策略
将存储策略设置为每秒写入（everysec）：
[root@node106 ~]# vim /apps/redis/etc/redis.conf
############################## APPEND ONLY MODE ###############################
appendonly yes
appendfilename "appendonly_6379.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble no
重启 Redis
[root@node106 ~]# systemctl restart redis
虽然 RDB 文件仍然存在，但 Redis 启动后并未将其数据载入内存，说明 AOF 模式开启时，会忽略 RDB 文件，而是载入 AOF 文件中的数据；
[root@node106 ~]# ll /apps/redis/data/
total 20
-rw-r--r-- 1 redis redis 0 Dec 30 21:55 appendonly_6379.aof
-rw-r--r-- 1 redis redis 18885 Dec 30 21:43 dump_6379.rdb
[root@node106 ~]# redis-cli
127.0.0.1:6379> keys *
(empty list or set)
写入测试数据
127.0.0.1:6379> set foo bar
OK
验证 AOF
查看 AOF 文件中的记录：
[root@node106 ~]# cat /apps/redis/data/appendonly_6379.aof
*2
$6
SELECT
$1
0
*3
$3
set
$3
foo
$3
bar
重启 Redis，会载入 AOF 文件中的数据：
[root@node106 ~]# systemctl restart redis
[root@node106 ~]# tail -f /apps/redis/logs/redis_6379.log
16824:M 30 Dec 23:13:35.059 * DB loaded from append only file: 0.000 seconds
[root@node106 ~]# redis-cli
127.0.0.1:6379> keys *
1) "foo"
2.2：Redis 主从复制配置
2.2.1：拓扑图
Master：192.168.1.201
Slave：192.168.1.202
2.2.2：临时实现主从复制
通过命令行实现的主从复制关系，在重启 Redis 后会失效；
Master 写入测试数据
shell 脚本：
[root@redis1 ~]# vim redis.sh
#!/bin/bash
NUM=`seq 1 1000`
PASS='123456'
for i in ${NUM};do
redis-cli -h 127.0.0.1 -a ${PASS} set key-${i} value-${i} &>/dev/null
done
echo "数据写入完成"
执行脚本，写入数据：
[root@redis1 ~]# bash redis.sh
数据写入完成
查看 Master 数据：
127.0.0.1:6379> GET key-111
"value-111"
127.0.0.1:6379> GET key-999
"value-999"
Slave 命令行配置
指定 Master 为 192.168.1.201：
127.0.0.1:6379> SLAVEOF 192.168.1.201 6379
设置 Master 连接的认证密码：
127.0.0.1:6379> CONFIG SET masterauth 123456
验证主从复制
查看 Master 日志：
[root@redis1 ~]# tail -f /apps/redis/logs/redis_6379.log
……
6538:M 04 Jan 11:04:25.156 * Slave 192.168.1.202:6379 asks for synchronization
6538:M 04 Jan 11:04:25.156 * Full resync requested by slave 192.168.1.202:6379
6538:M 04 Jan 11:04:26.684 * Starting BGSAVE for SYNC with target: slaves sockets
6538:M 04 Jan 11:04:26.686 * Background RDB transfer started by pid 7574
7574:C 04 Jan 11:04:26.694 * RDB: 6 MB of memory used by copy-on-write
6538:M 04 Jan 11:04:26.789 * Background RDB transfer terminated with success
6538:M 04 Jan 11:04:26.789 # Slave 192.168.1.202:6379 correctly received the streamed RDB file.
6538:M 04 Jan 11:04:26.789 * Streamed RDB transfer with slave 192.168.1.202:6379 succeeded (socket). Waiting for REPLCONF ACK from slave to enable streaming
6538:M 04 Jan 11:04:27.194 * Synchronization with slave 192.168.1.202:6379 succeeded
查看 Slave 日志：
[root@redis2 ~]# tail -f /apps/redis/logs/redis_6379.log
……
2511:S 04 Jan 11:04:25.146 * Connecting to MASTER 192.168.1.201:6379
2511:S 04 Jan 11:04:25.146 * MASTER <-> SLAVE sync started
2511:S 04 Jan 11:04:25.147 * Non blocking connect for SYNC fired the event.
2511:S 04 Jan 11:04:25.150 * Master replied to PING, replication can continue...
2511:S 04 Jan 11:04:25.156 * Partial resynchronization not possible (no cached master)
2511:S 04 Jan 11:04:26.685 * Full resync from master: d8b79f11217860aeb81a74da06408af954e306f4:0
2511:S 04 Jan 11:04:26.692 * MASTER <-> SLAVE sync: receiving streamed RDB from master
2511:S 04 Jan 11:04:26.696 * MASTER <-> SLAVE sync: Flushing old data
2511:S 04 Jan 11:04:26.696 * MASTER <-> SLAVE sync: Loading DB in memory
2511:S 04 Jan 11:04:26.699 * MASTER <-> SLAVE sync: Finished with success
2511:S 04 Jan 11:04:26.701 * Background append only file rewriting started by pid 2519
2511:S 04 Jan 11:04:26.771 * AOF rewrite child asks to stop sending diffs.
2519:C 04 Jan 11:04:26.771 * Parent agreed to stop sending diffs. Finalizing AOF...
2519:C 04 Jan 11:04:26.771 * Concatenating 0.00 MB of AOF diff received from parent.
2519:C 04 Jan 11:04:26.771 * SYNC append only file rewrite performed
2519:C 04 Jan 11:04:26.772 * AOF rewrite: 6 MB of memory used by copy-on-write
2511:S 04 Jan 11:04:26.784 * Background AOF rewrite terminated with success
2511:S 04 Jan 11:04:26.784 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
2511:S 04 Jan 11:04:26.784 * Background AOF rewrite finished successfully
查看 Slave 中同步过来的数据：
127.0.0.1:6379> GET key-1000
"value-1000"
Master 的 Replication 信息：
127.0.0.1:6379> INFO Replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.1.202,port=6379,state=online,offset=420,lag=1
master_replid:d8b79f11217860aeb81a74da06408af954e306f4
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:420
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:104857600
repl_backlog_first_byte_offset:1
repl_backlog_histlen:420
Slave 的 Replication 信息：
127.0.0.1:6379> INFO Replication
# Replication
role:slave
master_host:192.168.1.201
master_port:6379
master_link_status:up
master_last_io_seconds_ago:4
master_sync_in_progress:0
slave_repl_offset:448
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:d8b79f11217860aeb81a74da06408af954e306f4
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:448
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:104857600
repl_backlog_first_byte_offset:1
repl_backlog_histlen:448
2.2.3：永久实现主从复制
一般采用编辑配置文件的方式来保存主从复制关系，再次启动 Redis 时，主从复制关系仍然存在；
编辑 Slave 配置文件
设置 Master 为 192.168.1.201，并配置 Master 连接密码：
[root@redis2 ~]# vim /apps/redis/etc/redis.conf
slaveof 192.168.1.201 6379
masterauth 123456
重启 Slave 的 Redis 进程
[root@redis2 ~]# systemctl restart redis
验证主从复制
查看主从复制关系：
master_link_status:up，在 Slave 重启后主从复制关系仍然存在；
127.0.0.1:6379> INFO Replication
# Replication
role:slave
master_host:192.168.1.201
master_port:6379
master_link_status:up
master_last_io_seconds_ago:5
master_sync_in_progress:0
slave_repl_offset:728
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:d8b79f11217860aeb81a74da06408af954e306f4
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:728
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:104857600
repl_backlog_first_byte_offset:631
repl_backlog_histlen:98
Master 添加数据：
127.0.0.1:6379> SET key-1001 value-1001
OK
127.0.0.1:6379> SET key-test value-test
OK
Slave 验证同步数据：
127.0.0.1:6379> GET key-1001
"value-1001"
127.0.0.1:6379> GET key-test
"value-test"
2.2.4：模拟 Master 不可用
验证 Slave 的只读状态
配置文件中设置了 slave-read-only yes，所以 Slave 节点为只读：
127.0.0.1:6379> SET foo bar
(error) READONLY You can't write against a read only slave.
停止 Master 的 Redis 进程
停止 Redis 进程：
[root@redis1 ~]# systemctl stop redis
Slave 日志中显示已失去与 Master 的连接：
2590:S 04 Jan 11:20:40.223 # Connection with master lost.
2590:S 04 Jan 11:20:40.223 * Caching the disconnected master state.
2590:S 04 Jan 11:20:40.314 * Connecting to MASTER 192.168.1.201:6379
2590:S 04 Jan 11:20:40.314 * MASTER <-> SLAVE sync started
2590:S 04 Jan 11:20:40.315 # Error condition on socket for SYNC: Connection refused
Slave 的 Replication 状态：
master_link_status:down，主从复制关系已 down；
127.0.0.1:6379> INFO Replication
# Replication
role:slave
master_host:192.168.1.201
master_port:6379
master_link_status:down
master_last_io_seconds_ago:-1
master_sync_in_progress:0
slave_repl_offset:1413
master_link_down_since_seconds:103
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:d8b79f11217860aeb81a74da06408af954e306f4
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:1413
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:104857600
repl_backlog_first_byte_offset:631
repl_backlog_histlen:783
手动将 Slave 提升为 Master
手动取消之前的主从复制关系：
127.0.0.1:6379> SLAVEOF no one
OK
Slave 日志：
2590:M 04 Jan 11:23:12.376 # Setting secondary replication ID to d8b79f11217860aeb81a74da06408af954e306f4, valid up to offset: 1414. New replication ID is 37c7c687adda7c9d53578ff8aa16e403413c8a8f
2590:M 04 Jan 11:23:12.376 * Discarding previously cached master state.
2590:M 04 Jan 11:23:12.376 * MASTER MODE enabled (user request from 'id=4 addr=127.0.0.1:52112 fd=7 name= age=596 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=slaveof')
验证 Slave 已提升为 Master
查看当前的 Replicaiton 状态：
角色已成为 master，且之前的 master_replid 已成为 master_replid2；
127.0.0.1:6379> INFO Replication
# Replication
role:master
connected_slaves:0
master_replid:37c7c687adda7c9d53578ff8aa16e403413c8a8f
master_replid2:d8b79f11217860aeb81a74da06408af954e306f4
master_repl_offset:1413
second_repl_offset:1414
repl_backlog_active:1
repl_backlog_size:104857600
repl_backlog_first_byte_offset:631
repl_backlog_histlen:783
验证写入操作：
127.0.0.1:6379> SET foo bar
OK
127.0.0.1:6379> GET foo
"bar"
此时之前的 Slave 已经可以单独对外提供服务了；
2.2.5：将之前的 Master 作为 Slave 添加进来
启动 Redis 并指定 Master
启动 192.168.1.201（原 Master）的 Redis 进程，并设为 192.168.1.202 的 Slave：
[root@redis1 ~]# systemctl start redis
[root@redis1 ~]# redis-cli
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> SLAVEOF 192.168.1.202 6379
OK
127.0.0.1:6379> CONFIG SET masterauth 123456
OK
验证新的主从复制关系
Master 的 Replication 状态：
127.0.0.1:6379> INFO Replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.1.201,port=6379,state=online,offset=1607,lag=0
master_replid:37c7c687adda7c9d53578ff8aa16e403413c8a8f
master_replid2:d8b79f11217860aeb81a74da06408af954e306f4
master_repl_offset:1607
second_repl_offset:1414
repl_backlog_active:1
repl_backlog_size:104857600
repl_backlog_first_byte_offset:631
repl_backlog_histlen:977
Slave 的 Replication 状态：
127.0.0.1:6379> INFO Replication
# Replication
role:slave
master_host:192.168.1.202
master_port:6379
master_link_status:up
master_last_io_seconds_ago:10
master_sync_in_progress:0
slave_repl_offset:1635
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:37c7c687adda7c9d53578ff8aa16e403413c8a8f
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:1635
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:104857600
repl_backlog_first_byte_offset:1468
repl_backlog_histlen:168
Master 日志：
2590:M 04 Jan 11:32:15.306 * Slave 192.168.1.201:6379 asks for synchronization
2590:M 04 Jan 11:32:15.306 * Full resync requested by slave 192.168.1.201:6379
2590:M 04 Jan 11:32:16.768 * Starting BGSAVE for SYNC with target: slaves sockets
2590:M 04 Jan 11:32:16.769 * Background RDB transfer started by pid 2615
2615:C 04 Jan 11:32:16.776 * RDB: 6 MB of memory used by copy-on-write
2590:M 04 Jan 11:32:16.877 * Background RDB transfer terminated with success
2590:M 04 Jan 11:32:16.877 # Slave 192.168.1.201:6379 correctly received the streamed RDB file.
2590:M 04 Jan 11:32:16.877 * Streamed RDB transfer with slave 192.168.1.201:6379 succeeded (socket). Waiting for REPLCONF ACK from slave to enable streaming
2590:M 04 Jan 11:32:17.486 * Synchronization with slave 192.168.1.201:6379 succeeded
Slave 日志：
7621:S 04 Jan 11:32:15.302 * Connecting to MASTER 192.168.1.202:6379
7621:S 04 Jan 11:32:15.302 * MASTER <-> SLAVE sync started
7621:S 04 Jan 11:32:15.303 * Non blocking connect for SYNC fired the event.
7621:S 04 Jan 11:32:15.304 * Master replied to PING, replication can continue...
7621:S 04 Jan 11:32:15.306 * Partial resynchronization not possible (no cached master)
7621:S 04 Jan 11:32:16.769 * Full resync from master: 37c7c687adda7c9d53578ff8aa16e403413c8a8f:1467
7621:S 04 Jan 11:32:16.778 * MASTER <-> SLAVE sync: receiving streamed RDB from master
7621:S 04 Jan 11:32:16.782 * MASTER <-> SLAVE sync: Flushing old data
7621:S 04 Jan 11:32:16.783 * MASTER <-> SLAVE sync: Loading DB in memory
7621:S 04 Jan 11:32:16.786 * MASTER <-> SLAVE sync: Finished with success
7621:S 04 Jan 11:32:16.787 * Background append only file rewriting started by pid 7626
7621:S 04 Jan 11:32:17.080 * AOF rewrite child asks to stop sending diffs.
7626:C 04 Jan 11:32:17.081 * Parent agreed to stop sending diffs. Finalizing AOF...
7626:C 04 Jan 11:32:17.081 * Concatenating 0.00 MB of AOF diff received from parent.
7626:C 04 Jan 11:32:17.081 * SYNC append only file rewrite performed
7626:C 04 Jan 11:32:17.082 * AOF rewrite: 6 MB of memory used by copy-on-write
7621:S 04 Jan 11:32:17.158 * Background AOF rewrite terminated with success
7621:S 04 Jan 11:32:17.159 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
7621:S 04 Jan 11:32:17.159 * Background AOF rewrite finished successfully
验证 Slave 数据：
127.0.0.1:6379> GET foo
"bar"
127.0.0.1:6379> GET key-222
"value-222"
Master 的切换会导致 master_replid 发生变化，Slave 之前的 master_replid 就和当前 Master 不一致，从而会引发所有 Slave 的全量同步。
2.3：Redis Sentinel 部署
实现一主两从的 Redis Sentinel 高可用架构；
2.3.1：拓扑图
Master：Redis1，192.168.1.201
Slave1：Redis2，192.168.1.202
Slave2：Redis3，192.168.1.203
2.3.2：准备基本的主从结构
Redis2 配置为 Redis1 的 Slave
编辑配置文件：
[root@redis2 ~]# vim /apps/redis/etc/redis.conf
slaveof 192.168.1.201 6379
masterauth 123456
重启 Redis：
[root@redis2 ~]# systemctl restart redis
Redis3 配置为 Redis1 的 Slave
编辑配置文件：
[root@redis3 ~]# vim /apps/redis/etc/redis.conf
slaveof 192.168.1.201 6379
masterauth 123456
重启 Redis：
[root@redis3 ~]# systemctl restart redis
验证主从结构
Master 的 Replication 状态：
127.0.0.1:6379> INFO Replication
# Replication
role:master
connected_slaves:2
slave0:ip=192.168.1.202,port=6379,state=online,offset=1020,lag=0
slave1:ip=192.168.1.203,port=6379,state=online,offset=1020,lag=0
master_replid:ed864f7788f4ca787fefc7aba4c98399b48ac151
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:1020
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:104857600
repl_backlog_first_byte_offset:1
repl_backlog_histlen:1020
Slave1 的 Replication 状态：
127.0.0.1:6379> INFO Replication
# Replication
role:slave
master_host:192.168.1.201
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:1090
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:ed864f7788f4ca787fefc7aba4c98399b48ac151
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:1090
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:104857600
repl_backlog_first_byte_offset:1
repl_backlog_histlen:1090
Slave2 的 Replication 状态：
127.0.0.1:6379> INFO Replication
# Replication
role:slave
master_host:192.168.1.201
master_port:6379
master_link_status:up
master_last_io_seconds_ago:7
master_sync_in_progress:0
slave_repl_offset:1118
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:ed864f7788f4ca787fefc7aba4c98399b48ac151
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:1118
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:104857600
repl_backlog_first_byte_offset:979
repl_backlog_histlen:140
2.3.3：编辑 sentinel.conf
Master
[root@redis1 ~]# grep '^[a-Z]' /apps/redis/etc/sentinel.conf
bind 0.0.0.0
port 26379
dir /apps/redis/logs
daemonize yes
pidfile "redis-sentinel.pid"
logfile "sentinel_26379.log"
sentinel monitor mymaster 192.168.1.201 6379 2
sentinel auth-pass mymaster 123456
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
sentinel deny-scripts-reconfig yes
Slave1
[root@redis2 ~]# grep '^[a-Z]' /apps/redis/etc/sentinel.conf
bind 0.0.0.0
port 26379
dir /apps/redis/logs
daemonize yes
pidfile "redis-sentinel.pid"
logfile "sentinel_26379.log"
sentinel monitor mymaster 192.168.1.201 6379 2
sentinel auth-pass mymaster 123456
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
sentinel deny-scripts-reconfig yes
Slave2
[root@redis3 ~]# grep '^[a-Z]' /apps/redis/etc/sentinel.conf
bind 0.0.0.0
port 26379
dir /apps/redis/logs
daemonize yes
pidfile "redis-sentinel.pid"
logfile "sentinel_26379.log"
sentinel monitor mymaster 192.168.1.201 6379 2
sentinel auth-pass mymaster 123456
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
sentinel deny-scripts-reconfig yes
2.3.4：启动 Sentinel
三台 Redis 服务器启动哨兵服务：
[root@redis1 ~]# /apps/redis/bin/redis-sentinel /apps/redis/etc/sentinel.conf
[root@redis2 ~]# /apps/redis/bin/redis-sentinel /apps/redis/etc/sentinel.conf
[root@redis3 ~]# /apps/redis/bin/redis-sentinel /apps/redis/etc/sentinel.conf
设置开机自启动：
~]# vim /etc/rc.local
/apps/redis/bin/redis-sentinel /apps/redis/etc/sentinel.conf
~]# chmod a+x /etc/rc.local
2.3.5：验证 Sentinel 状态
查看端口
[root@redis1 ~]# ss -tnlp | grep 26379
LISTEN 0 511 *:26379 *:* users:(("redis-sentinel",pid=7878,fd=6))
查看日志
[root@redis1 ~]# tail -f /apps/redis/logs/sentinel_26379.log
7877:X 04 Jan 14:29:10.906 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
7877:X 04 Jan 14:29:10.907 # Redis version=4.0.14, bits=64, commit=00000000, modified=0, pid=7877, just started
7877:X 04 Jan 14:29:10.907 # Configuration loaded
7878:X 04 Jan 14:29:10.914 * Running mode=sentinel, port=26379.
7878:X 04 Jan 14:29:10.917 # Sentinel ID is 1d6913d96168a8b64bdbaafbaf799f51ed846212
7878:X 04 Jan 14:29:10.917 # +monitor master mymaster 192.168.1.201 6379 quorum 2
7878:X 04 Jan 14:29:10.919 * +slave slave 192.168.1.203:6379 192.168.1.203 6379 @ mymaster 192.168.1.201 6379
7878:X 04 Jan 14:29:10.924 * +slave slave 192.168.1.202:6379 192.168.1.202 6379 @ mymaster 192.168.1.201 6379
7878:X 04 Jan 14:29:20.452 * +sentinel sentinel 2ead37c274302593d6089935240e771cde053393 192.168.1.202 26379 @ mymaster 192.168.1.201 6379
7878:X 04 Jan 14:29:22.695 * +sentinel sentinel 50b97ee30a9fd1d122941c7be62cb532e3cc0c68 192.168.1.203 26379 @ mymaster 192.168.1.201 6379
查看 Sentinel 状态
验证 Sentinel 架构中的 Master I、Slave 数量，Sentinel 数量，是否符合：
[root@redis1 ~]# redis-cli -p 26379
127.0.0.1:26379> INFO Sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=192.168.1.201:6379,slaves=2,sentinels=3
2.3.6：测试 Master 切换
模拟当前 Master 不可用，测试是否能成功进行 Failover，从 Slaves 中选取一个新的 Master；
通过脚本向当前 Master 批量写入测试数据
shell 脚本：
[root@redis1 ~]# vim redis.sh
#!/bin/bash
NUM=`seq 1 1000`
PASS='123456'
for i in ${NUM};do
redis-cli -h 127.0.0.1 -a ${PASS} set key-${i} value-${i} &>/dev/null
done
echo "数据写入完成"
执行脚本，写入测试数据：
[root@redis1 ~]# bash redis.sh
数据写入完成
[root@redis1 ~]# redis-cli
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> get key-333
"value-333"
模拟 Master 不可用
停止 Master 的 Redis 服务：
[root@redis1 ~]# systemctl stop redis
查看 Master 切换过程中 Sentinel 的日志：
2813:X 04 Jan 14:40:10.224 # +sdown master mymaster 192.168.1.201 6379
2813:X 04 Jan 14:40:10.294 # +new-epoch 1
2813:X 04 Jan 14:40:10.295 # +vote-for-leader 1d6913d96168a8b64bdbaafbaf799f51ed846212 1
2813:X 04 Jan 14:40:11.324 # +odown master mymaster 192.168.1.201 6379 #quorum 3/2
2813:X 04 Jan 14:40:11.324 # Next failover delay: I will not start a failover before Mon Jan 4 14:46:11 2021
2813:X 04 Jan 14:40:11.423 # +config-update-from sentinel 1d6913d96168a8b64bdbaafbaf799f51ed846212 192.168.1.201 26379 @ mymaster 192.168.1.201 6379
2813:X 04 Jan 14:40:11.423 # +switch-master mymaster 192.168.1.201 6379 192.168.1.203 6379
2813:X 04 Jan 14:40:11.423 * +slave slave 192.168.1.202:6379 192.168.1.202 6379 @ mymaster 192.168.1.203 6379
2813:X 04 Jan 14:40:11.424 * +slave slave 192.168.1.201:6379 192.168.1.201 6379 @ mymaster 192.168.1.203 6379
2813:X 04 Jan 14:40:41.454 # +sdown slave 192.168.1.201:6379 192.168.1.201 6379 @ mymaster 192.168.1.203 6379
验证 Master 切换
连接随意一台 Sentinel，查看当前的 Sentinel 状态：
192.168.1.203 已切换为新的 Master，其余两台 Redis 成为它的 Slave（192.168.1.201 为 sdown 状态）；
[root@redis3 ~]# redis-cli -p 26379
127.0.0.1:26379> INFO Sentinel
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=192.168.1.203:6379,slaves=2,sentinels=3
查看新的 Master 的 Replication 状态：
显示有效的 Slave 仅有 1个；
127.0.0.1:6379> INFO Replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.1.202,port=6379,state=online,offset=249687,lag=0
master_replid:266fb844ca16fc4dc4cefdcd6a61bfa7d36008bd
master_replid2:ed864f7788f4ca787fefc7aba4c98399b48ac151
master_repl_offset:250110
second_repl_offset:175714
repl_backlog_active:1
repl_backlog_size:104857600
repl_backlog_first_byte_offset:979
repl_backlog_histlen:249132
查看 Slave 的 Replication 状态：
之前的 master_replid 转换为 master_replid2；
127.0.0.1:6379> INFO Replication
# Replication
role:slave
master_host:192.168.1.203
master_port:6379
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:406810
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:266fb844ca16fc4dc4cefdcd6a61bfa7d36008bd
master_replid2:ed864f7788f4ca787fefc7aba4c98399b48ac151
master_repl_offset:406810
second_repl_offset:175714
repl_backlog_active:1
repl_backlog_size:104857600
repl_backlog_first_byte_offset:1
repl_backlog_histlen:406810
查看经 Sentinel 自动修改的配置文件
新的 Master 的 redis.conf：
之前的 slaveof 配置已删除；
[root@redis3 ~]# vim /apps/redis/etc/redis.conf
# Generated by CONFIG REWRITE
maxclients 4064
Slave 的 redis.conf：
slaveof 指向新的 Master；
[root@redis2 ~]# vim /apps/redis/etc/redis.conf
slaveof 192.168.1.203 6379
# Generated by CONFIG REWRITE
maxclients 4064
sentinel.conf：
monitor 已指向新的 Master；
添加了已识别的 Slaves 和 Masters；
[root@redis2 ~]# grep '^[a-Z]' /apps/redis/etc/sentinel.conf
bind 0.0.0.0
port 26379
dir "/apps/redis/logs"
daemonize yes
pidfile "redis-sentinel.pid"
logfile "sentinel_26379.log"
sentinel myid 2ead37c274302593d6089935240e771cde053393
sentinel deny-scripts-reconfig yes
sentinel monitor mymaster 192.168.1.203 6379 2
sentinel auth-pass mymaster 123456
sentinel config-epoch mymaster 1
sentinel leader-epoch mymaster 1
sentinel known-slave mymaster 192.168.1.202 6379
sentinel known-slave mymaster 192.168.1.201 6379
sentinel known-sentinel mymaster 192.168.1.203 26379 50b97ee30a9fd1d122941c7be62cb532e3cc0c68
sentinel known-sentinel mymaster 192.168.1.201 26379 1d6913d96168a8b64bdbaafbaf799f51ed846212
sentinel current-epoch 1
验证新的主从复制结构
新的 Master 验证原有数据：
127.0.0.1:6379> GET key-999
"value-999"
新的 Master 写入数据：
127.0.0.1:6379> SET foo3 bar3
OK
Slave 查看新数据：
127.0.0.1:6379> GET foo3
"bar3"
2.4：Redis Cluster 部署
2.4.1：部署 Redis Cluster 的前提
- 时间同步；
- Redis Node 均采用相同的硬件配置、相同的密码、相同的 Redis 版本；
- Redis Node 必须开启的参数：
- cluster-enabled yes：启用集群，开启后 Redis 进程会有 Cluster 显示；
- cluster-config-file nodes-6380.conf：指定集群配置文件，此文件由 Redis Cluster 自动创建和维护，开启即可，不需要任何手动配置；
- 所有 Redis Node 必须没有任何数据；
2.4.2：拓扑图
共 6 台 Redis 服务器组成 Redis Cluster，三主三从，即集群中的 Redis Node 为 3 个；
奇数为 Master，偶数为 Slave，方便后期维护管理时辨别；
-
Masters：Redis1、Redis3、Redis5；
-
Slaves：Redis2、Redis4、Redis6；
各 Redis 服务器编译安装 redis-4.0.14 和 redis-5.0.10（为了测试 redis 5 版本的集群创建和维护）；
2.4.3：各节点配置并启动 Redis
编辑各节点的 Redis 配置文件：
配置 masterauth（如果有 requirepass）；
开启集群；
指定集群配置文件；
masterauth 123456
cluster-enabled yes
cluster-config-file nodes-6379.conf
各节点启动为单机 Redis：
systemctl start redis && systemctl enable redis
2.4.4：创建集群（Redis-3/4）
准备 redis-trib.rb 使用环境
Redis 3 和 4 版本创建集群时，需要使用集群管理工具 redis-trib.rb，这个工具是 Redis 官方推出的管理 Redis 集群的工具，集成在 Redis 源码的 src 目录下，是基于 Redis 提供的集群命令而封装的简单、便捷、实用的操作工具；
redis-trib.rb 是 Redis 作者用 Ruby 开发完成的，CentOS yum 安装的 Ruby 存在版本较低问题，需要编译安装 Ruby：
只需在一台 Redis 服务器上准备环境即可；
[root@redis1 ~]# yum remove ruby rubygems -y
[root@redis1 ~]# cd /usr/local/src
[root@s1 src]# wget https://cache.ruby-lang.org/pub/ruby/2.5/ruby-2.5.5.tar.gz
[root@redis1 src]# tar xf ruby-2.5.5.tar.gz
[root@redis1 src]# cd ruby-2.5.5/
[root@redis1 ruby-2.5.5]# ./configure
[root@redis1 ruby-2.5.5]# make -j 2
[root@redis1 ruby-2.5.5]# make install
安装 Ruby 的 Redis 模块：
[root@redis1 ruby-2.5.5]# gem install redis
如果无法在线安装 Ruby 的 Redis 模块，可以先下载 Redis 模块（https://rubygems.org/gems/redis），再离线安装：
gem install -l redis-3.3.0.gem
创建 redis-trib.rb 软链接：
[root@redis1 ~]# ln -sv /usr/local/src/redis-4.0.14/src/redis-trib.rb /usr/bin/redis-trib
‘/usr/bin/redis-trib’ -> ‘/usr/local/src/redis-4.0.14/src/redis-trib.rb’
查看 redis-trib.rb 使用帮助：
[root@redis1 ~]# redis-trib
Usage: redis-trib
create host1:port1 ... hostN:portN # 创建集群，指定集群中的节点；
--replicas # 指定每个Master的副本数量（即一个Master有几个Slave）；
check host:port # 检查集群状态
info host:port # 查看集群信息
fix host:port # 修复集群
--timeout
reshard host:port # 重新分片（热迁移slots）；
--from
--to
--slots
--yes
--timeout
--pipeline
rebalance host:port # 平衡集群中各节点的slot数量；
--weight
--auto-weights
--use-empty-masters
--timeout
--simulate
--pipeline
--threshold
add-node new_host:new_port existing_host:existing_port
# 向集群中添加节点；
--slave
--master-id
del-node host:port node_id # 从集群中删除节点；
set-timeout host:port milliseconds # 设置节点的超时时间（单位毫秒）；
call host:port command arg arg .. arg
# 在集群的所有节点上执行命令；
import host:port # 导入外部redis服务器的数据到当前集群；
--from
--copy
--replace
help (show this help)
For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.
为 Ruby Redis 模块配置 Redis 连接密码
[root@redis1 ~]# vim /usr/local/lib/ruby/gems/2.5.0/gems/redis-4.2.5/lib/redis/client.rb
class Redis
class Client
# Defaults are also used for converting string keys to symbols.
DEFAULTS = {
……
password: 123456,
创建 Redis Cluster
创建时，配置在前的节点会成为 Master：
[root@redis1 ~]# redis-trib create --replicas 1 192.168.1.201:6379 192.168.1.203:6379 192.168.1.205:6379 192.168.1.202:6379 192.168.1.204:6379 192.168.1.206:6379
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.1.201:6379
192.168.1.203:6379
192.168.1.205:6379
Adding replica 192.168.1.204:6379 to 192.168.1.201:6379
Adding replica 192.168.1.206:6379 to 192.168.1.203:6379
Adding replica 192.168.1.202:6379 to 192.168.1.205:6379
M: 469e0c16495598b5fadba28f856203e206464b99 192.168.1.201:6379
slots:0-5460 (5461 slots) master
M: 4dd0b78edebcfe93c93518b5817c00cc21bef07e 192.168.1.203:6379
slots:5461-10922 (5462 slots) master
M: 1b897eda6acfb806eb27326f151e4067f0d47f7b 192.168.1.205:6379
slots:10923-16383 (5461 slots) master
S: 674a2a0e6a4c9a88db493c649bbd584bc6ba1edc 192.168.1.202:6379
replicates 1b897eda6acfb806eb27326f151e4067f0d47f7b
S: 6ec84046944b4b82af12d8fbc2977c54fa5f670c 192.168.1.204:6379
replicates 469e0c16495598b5fadba28f856203e206464b99
S: 9c4b8d3a34266ba89cacb80ccf170c34169a0dd9 192.168.1.206:6379
replicates 4dd0b78edebcfe93c93518b5817c00cc21bef07e
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join........
>>> Performing Cluster Check (using node 192.168.1.201:6379)
M: 469e0c16495598b5fadba28f856203e206464b99 192.168.1.201:6379
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 674a2a0e6a4c9a88db493c649bbd584bc6ba1edc 192.168.1.202:6379
slots: (0 slots) slave
replicates 1b897eda6acfb806eb27326f151e4067f0d47f7b
M: 1b897eda6acfb806eb27326f151e4067f0d47f7b 192.168.1.205:6379
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: 6ec84046944b4b82af12d8fbc2977c54fa5f670c 192.168.1.204:6379
slots: (0 slots) slave
replicates 469e0c16495598b5fadba28f856203e206464b99
M: 4dd0b78edebcfe93c93518b5817c00cc21bef07e 192.168.1.203:6379
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 9c4b8d3a34266ba89cacb80ccf170c34169a0dd9 192.168.1.206:6379
slots: (0 slots) slave
replicates 4dd0b78edebcfe93c93518b5817c00cc21bef07e
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
2.4.5：创建集群（Redis-5）
Redis 5 版本可以直接使用 redis-cli 创建 Redis Cluster：
[root@redis1 ~]# redis-cli -a 123456 --cluster create 192.168.1.201:6379 192.168.1.203:6379 192.168.1.205:6379 192.168.1.202:6379 192.168.1.204:6379 192.168.1.206:6379 --cluster-replicas 1
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.1.204:6379 to 192.168.1.201:6379
Adding replica 192.168.1.206:6379 to 192.168.1.203:6379
Adding replica 192.168.1.202:6379 to 192.168.1.205:6379
M: feb6b43233fdcc01e0d5425fd03e0116b74f0833 192.168.1.201:6379
slots:[0-5460] (5461 slots) master
M: c2b48542344ece56e69d8ca9404ee29a48ae7b8e 192.168.1.203:6379
slots:[5461-10922] (5462 slots) master
M: df921d013a6cbf1588c1ad51809435fe39f6c25c 192.168.1.205:6379
slots:[10923-16383] (5461 slots) master
S: d7797d3410c1d43b918ee2e7c8cec8810a3fd519 192.168.1.202:6379
replicates df921d013a6cbf1588c1ad51809435fe39f6c25c
S: 90d1475f064839143bbaea911ac1e449bee174da 192.168.1.204:6379
replicates feb6b43233fdcc01e0d5425fd03e0116b74f0833
S: b11dfd69bd23ae0d987a136061d7fd15b500800a 192.168.1.206:6379
replicates c2b48542344ece56e69d8ca9404ee29a48ae7b8e
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 192.168.1.201:6379)
M: feb6b43233fdcc01e0d5425fd03e0116b74f0833 192.168.1.201:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: df921d013a6cbf1588c1ad51809435fe39f6c25c 192.168.1.205:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
M: c2b48542344ece56e69d8ca9404ee29a48ae7b8e 192.168.1.203:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: b11dfd69bd23ae0d987a136061d7fd15b500800a 192.168.1.206:6379
slots: (0 slots) slave
replicates c2b48542344ece56e69d8ca9404ee29a48ae7b8e
S: 90d1475f064839143bbaea911ac1e449bee174da 192.168.1.204:6379
slots: (0 slots) slave
replicates feb6b43233fdcc01e0d5425fd03e0116b74f0833
S: d7797d3410c1d43b918ee2e7c8cec8810a3fd519 192.168.1.202:6379
slots: (0 slots) slave
replicates df921d013a6cbf1588c1ad51809435fe39f6c25c
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
redis-cli -a 123456 --cluster create 192.168.7.101:6379
192.168.7.101:6380 192.168.7.102:6379 192.168.7.102:6380 1
92.168.7.103:6379 192.168.7.103:6380 --cluster-replicas 1
2.4.6：验证集群创建
集群中任一节点均可查看集群状态和信息；
CLUSTER 指令
查看集群状态：
127.0.0.1:6379> CLUSTER INFO
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:3
cluster_stats_messages_ping_sent:127
cluster_stats_messages_pong_sent:125
cluster_stats_messages_meet_sent:3
cluster_stats_messages_sent:255
cluster_stats_messages_ping_received:121
cluster_stats_messages_pong_received:130
cluster_stats_messages_meet_received:4
cluster_stats_messages_received:255
查看 Node 对应关系：
127.0.0.1:6379> CLUSTER NODES
4dd0b78edebcfe93c93518b5817c00cc21bef07e 192.168.1.203:6379@16379 master - 0 1609749076000 2 connected 5461-10922
9c4b8d3a34266ba89cacb80ccf170c34169a0dd9 192.168.1.206:6379@16379 myself,slave 4dd0b78edebcfe93c93518b5817c00cc21bef07e 0 1609749076000 6 connected
469e0c16495598b5fadba28f856203e206464b99 192.168.1.201:6379@16379 master - 0 1609749077996 1 connected 0-5460
674a2a0e6a4c9a88db493c649bbd584bc6ba1edc 192.168.1.202:6379@16379 slave 1b897eda6acfb806eb27326f151e4067f0d47f7b 0 1609749075000 4 connected
6ec84046944b4b82af12d8fbc2977c54fa5f670c 192.168.1.204:6379@16379 slave 469e0c16495598b5fadba28f856203e206464b99 0 1609749076973 5 connected
1b897eda6acfb806eb27326f151e4067f0d47f7b 192.168.1.205:6379@16379 master - 0 1609749075960 3 connected 10923-16383
redis-trib.rb（Redis-3/4）
指定集群中的任一节点均可查看集群的状态信息；
查看集群信息：
[root@redis1 ~]# redis-trib info 192.168.1.206:6379
192.168.1.203:6379 (4dd0b78e...) -> 0 keys | 5462 slots | 1 slaves.
192.168.1.201:6379 (469e0c16...) -> 0 keys | 5461 slots | 1 slaves.
192.168.1.205:6379 (1b897eda...) -> 0 keys | 5461 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
检查集群状态：
[root@redis1 ~]# redis-trib check 192.168.1.203:6379
>>> Performing Cluster Check (using node 192.168.1.203:6379)
M: 4dd0b78edebcfe93c93518b5817c00cc21bef07e 192.168.1.203:6379
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 6ec84046944b4b82af12d8fbc2977c54fa5f670c 192.168.1.204:6379
slots: (0 slots) slave
replicates 469e0c16495598b5fadba28f856203e206464b99
M: 469e0c16495598b5fadba28f856203e206464b99 192.168.1.201:6379
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 9c4b8d3a34266ba89cacb80ccf170c34169a0dd9 192.168.1.206:6379
slots: (0 slots) slave
replicates 4dd0b78edebcfe93c93518b5817c00cc21bef07e
M: 1b897eda6acfb806eb27326f151e4067f0d47f7b 192.168.1.205:6379
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: 674a2a0e6a4c9a88db493c649bbd584bc6ba1edc 192.168.1.202:6379
slots: (0 slots) slave
replicates 1b897eda6acfb806eb27326f151e4067f0d47f7b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
redis-cli（Redis-5）
[root@redis1 ~]# redis-cli -a 123456 --cluster check 192.168.1.201:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.1.201:6379 (feb6b432...) -> 0 keys | 5461 slots | 1 slaves.
192.168.1.205:6379 (df921d01...) -> 0 keys | 5461 slots | 1 slaves.
192.168.1.203:6379 (c2b48542...) -> 0 keys | 5462 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.1.201:6379)
M: feb6b43233fdcc01e0d5425fd03e0116b74f0833 192.168.1.201:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: df921d013a6cbf1588c1ad51809435fe39f6c25c 192.168.1.205:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
M: c2b48542344ece56e69d8ca9404ee29a48ae7b8e 192.168.1.203:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: b11dfd69bd23ae0d987a136061d7fd15b500800a 192.168.1.206:6379
slots: (0 slots) slave
replicates c2b48542344ece56e69d8ca9404ee29a48ae7b8e
S: 90d1475f064839143bbaea911ac1e449bee174da 192.168.1.204:6379
slots: (0 slots) slave
replicates feb6b43233fdcc01e0d5425fd03e0116b74f0833
S: d7797d3410c1d43b918ee2e7c8cec8810a3fd519 192.168.1.202:6379
slots: (0 slots) slave
replicates df921d013a6cbf1588c1ad51809435fe39f6c25c
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
查看集群配置文件
Redis-4.0.14：
[root@redis1 ~]# cat /apps/redis/data/nodes-6379.conf
674a2a0e6a4c9a88db493c649bbd584bc6ba1edc 192.168.1.202:6379@16379 slave 1b897eda6acfb806eb27326f151e4067f0d47f7b 0 1609748891000 4 connected
1b897eda6acfb806eb27326f151e4067f0d47f7b 192.168.1.205:6379@16379 master - 0 1609748892000 3 connected 10923-16383
6ec84046944b4b82af12d8fbc2977c54fa5f670c 192.168.1.204:6379@16379 slave 469e0c16495598b5fadba28f856203e206464b99 0 1609748892000 5 connected
4dd0b78edebcfe93c93518b5817c00cc21bef07e 192.168.1.203:6379@16379 master - 0 1609748892922 2 connected 5461-10922
9c4b8d3a34266ba89cacb80ccf170c34169a0dd9 192.168.1.206:6379@16379 slave 4dd0b78edebcfe93c93518b5817c00cc21bef07e 0 1609748891000 6 connected
469e0c16495598b5fadba28f856203e206464b99 192.168.1.201:6379@16379 myself,master - 0 1609748890000 1 connected 0-5460
vars currentEpoch 6 lastVoteEpoch 0
Redis-5.0.10：
[root@redis1 ~]# cat /apps/redis5/data/nodes-6379.conf
df921d013a6cbf1588c1ad51809435fe39f6c25c 192.168.1.205:6379@16379 master - 0 1609811160000 3 connected 10923-16383
c2b48542344ece56e69d8ca9404ee29a48ae7b8e 192.168.1.203:6379@16379 master - 0 1609811161000 2 connected 5461-10922
b11dfd69bd23ae0d987a136061d7fd15b500800a 192.168.1.206:6379@16379 slave c2b48542344ece56e69d8ca9404ee29a48ae7b8e 0 1609811162000 6 connected
90d1475f064839143bbaea911ac1e449bee174da 192.168.1.204:6379@16379 slave feb6b43233fdcc01e0d5425fd03e0116b74f0833 0 1609811162884 5 connected
d7797d3410c1d43b918ee2e7c8cec8810a3fd519 192.168.1.202:6379@16379 slave df921d013a6cbf1588c1ad51809435fe39f6c25c 0 1609811162000 4 connected
feb6b43233fdcc01e0d5425fd03e0116b74f0833 192.168.1.201:6379@16379 myself,master - 0 1609811160000 1 connected 0-5460
vars currentEpoch 6 lastVoteEpoch 0
2.4.7：测试集群读写
集群数据写入
Redis1 节点上尝试写入：
提示 foo1 这个 key 的 CRC 结果被调度到了 13431 槽位，该槽位在 192.168.1.205:6379 节点上；
[root@redis1 ~]# redis-cli
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> SET foo1 bar1
(error) MOVED 13431 192.168.1.205:6379
Redis5 节点 写入 foo1：
[root@redis5 ~]# redis-cli
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> SET foo1 bar1
OK
各节点只保存各自槽位的数据，所以 Redis1 和 Redis3 都没有刚刚写入的 foo1：
[root@redis3 ~]# redis-cli
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> KEYS foo1
(empty list or set)
集群数据读取
因为各节点只保存各自槽位的数据，所以数据的读取也只能到相应的节点上进行；
而且集群中的 Slave 节点，读写服务均不提供；
查看 Redis5 的 Slave 节点：
Redis5 的 ID 为：1b897eda6acfb806eb27326f151e4067f0d47f7b；
可以查到对应的 Slave 为 Redis2（192.168.1.202）；
[root@redis1 ~]# cat /apps/redis/data/nodes-6379.conf
674a2a0e6a4c9a88db493c649bbd584bc6ba1edc 192.168.1.202:6379@16379 slave 1b897eda6acfb806eb27326f151e4067f0d47f7b 0 1609748891000 4 connected
1b897eda6acfb806eb27326f151e4067f0d47f7b 192.168.1.205:6379@16379 master - 0 1609748892000 3 connected 10923-16383
6ec84046944b4b82af12d8fbc2977c54fa5f670c 192.168.1.204:6379@16379 slave 469e0c16495598b5fadba28f856203e206464b99 0 1609748892000 5 connected
4dd0b78edebcfe93c93518b5817c00cc21bef07e 192.168.1.203:6379@16379 master - 0 1609748892922 2 connected 5461-10922
9c4b8d3a34266ba89cacb80ccf170c34169a0dd9 192.168.1.206:6379@16379 slave 4dd0b78edebcfe93c93518b5817c00cc21bef07e 0 1609748891000 6 connected
469e0c16495598b5fadba28f856203e206464b99 192.168.1.201:6379@16379 myself,master - 0 1609748890000 1 connected 0-5460
vars currentEpoch 6 lastVoteEpoch 0
到 Redis2 读取刚刚写入到 Redis5 的 foo1：
可以查到有 foo1 的数据，但是读取还是得到 Redis5 上；
127.0.0.1:6379> KEYS foo1
1) "foo1"
127.0.0.1:6379> GET foo1
(error) MOVED 13431 192.168.1.205:6379
2.5：Redis Cluster 管理维护
2.5.1：动态添加节点
增加 Redis Node，需要与之前的 Redis 版本相同、配置一致；
添加一个 Redis Node 需要启动两台 Redis，一主一从：
- 新增 Master：Redis7，192.168.1.207；
- 新增 Slave：Redis8，192.168.1.208；
新节点配置并启动 Redis
编辑两台 Redis 服务器的 Redis 配置文件，配置 masterauth、开启集群、指定集群配置文件：
masterauth 123456
cluster-enabled yes
cluster-config-file nodes-6379.conf
启动为单机 Redis：
systemctl start redis && systemctl enable redis
添加节点到集群（Redis-3/4）
添加新的 Master 到集群中：
[root@redis1 ~]# redis-trib add-node 192.168.1.207:6379 192.168.1.201:6379
>>> Adding node 192.168.1.207:6379 to cluster 192.168.1.201:6379
>>> Performing Cluster Check (using node 192.168.1.201:6379)
M: 469e0c16495598b5fadba28f856203e206464b99 192.168.1.201:6379
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 674a2a0e6a4c9a88db493c649bbd584bc6ba1edc 192.168.1.202:6379
slots: (0 slots) slave
replicates 1b897eda6acfb806eb27326f151e4067f0d47f7b
M: 1b897eda6acfb806eb27326f151e4067f0d47f7b 192.168.1.205:6379
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: 6ec84046944b4b82af12d8fbc2977c54fa5f670c 192.168.1.204:6379
slots: (0 slots) slave
replicates 469e0c16495598b5fadba28f856203e206464b99
M: 4dd0b78edebcfe93c93518b5817c00cc21bef07e 192.168.1.203:6379
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 9c4b8d3a34266ba89cacb80ccf170c34169a0dd9 192.168.1.206:6379
slots: (0 slots) slave
replicates 4dd0b78edebcfe93c93518b5817c00cc21bef07e
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.1.207:6379 to make it join the cluster.
[OK] New node added correctly.
查看 Redis7 在集群当中的 ID：
Redis7 的 ID 为：4630b3bde39432a04e94414ec6b2582d26f7fc21
[root@redis1 ~]# redis-trib check 192.168.1.201:6379
>>> Performing Cluster Check (using node 192.168.1.201:6379)
M: 469e0c16495598b5fadba28f856203e206464b99 192.168.1.201:6379
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 674a2a0e6a4c9a88db493c649bbd584bc6ba1edc 192.168.1.202:6379
slots: (0 slots) slave
replicates 1b897eda6acfb806eb27326f151e4067f0d47f7b
M: 1b897eda6acfb806eb27326f151e4067f0d47f7b 192.168.1.205:6379
slots:10923-16383 (5461 slots) master
1 additional replica(s)
M: 4630b3bde39432a04e94414ec6b2582d26f7fc21 192.168.1.207:6379
slots: (0 slots) master
0 additional replica(s)
S: 6ec84046944b4b82af12d8fbc2977c54fa5f670c 192.168.1.204:6379
slots: (0 slots) slave
replicates 469e0c16495598b5fadba28f856203e206464b99
M: 4dd0b78edebcfe93c93518b5817c00cc21bef07e 192.168.1.203:6379
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 9c4b8d3a34266ba89cacb80ccf170c34169a0dd9 192.168.1.206:6379
slots: (0 slots) slave
replicates 4dd0b78edebcfe93c93518b5817c00cc21bef07e
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
添加新的 Slave 到集群中（需要指定它的 Master）：
[root@redis1 ~]# redis-trib add-node --slave --master-id 4630b3bde39432a04e94414ec6b2582d26f7fc21 192.168.1.208:6379 192.168.1.201:6379
>>> Adding node 192.168.1.208:6379 to cluster 192.168.1.201:6379
>>> Performing Cluster Check (using node 192.168.1.201:6379)
M: 469e0c16495598b5fadba28f856203e206464b99 192.168.1.201:6379
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 674a2a0e6a4c9a88db493c649bbd584bc6ba1edc 192.168.1.202:6379
slots: (0 slots) slave
replicates 1b897eda6acfb806eb27326f151e4067f0d47f7b
M: 1b897eda6acfb806eb27326f151e4067f0d47f7b 192.168.1.205:6379
slots:10923-16383 (5461 slots) master
1 additional replica(s)
M: 4630b3bde39432a04e94414ec6b2582d26f7fc21 192.168.1.207:6379
slots: (0 slots) master
0 additional replica(s)
S: 6ec84046944b4b82af12d8fbc2977c54fa5f670c 192.168.1.204:6379
slots: (0 slots) slave
replicates 469e0c16495598b5fadba28f856203e206464b99
M: 4dd0b78edebcfe93c93518b5817c00cc21bef07e 192.168.1.203:6379
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 9c4b8d3a34266ba89cacb80ccf170c34169a0dd9 192.168.1.206:6379
slots: (0 slots) slave
replicates 4dd0b78edebcfe93c93518b5817c00cc21bef07e
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.1.208:6379 to make it join the cluster.
Waiting for the cluster to join.
>>> Configure node as replica of 192.168.1.207:6379.
[OK] New node added correctly.
添加节点到集群（Redis-5）
添加 Master：
[root@redis1 ~]# redis-cli -a 123456 --cluster add-node 192.168.1.207:6379 192.168.1.201:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Adding node 192.168.1.207:6379 to cluster 192.168.1.201:6379
>>> Performing Cluster Check (using node 192.168.1.201:6379)
M: feb6b43233fdcc01e0d5425fd03e0116b74f0833 192.168.1.201:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: df921d013a6cbf1588c1ad51809435fe39f6c25c 192.168.1.205:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
M: c2b48542344ece56e69d8ca9404ee29a48ae7b8e 192.168.1.203:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: b11dfd69bd23ae0d987a136061d7fd15b500800a 192.168.1.206:6379
slots: (0 slots) slave
replicates c2b48542344ece56e69d8ca9404ee29a48ae7b8e
S: 90d1475f064839143bbaea911ac1e449bee174da 192.168.1.204:6379
slots: (0 slots) slave
replicates feb6b43233fdcc01e0d5425fd03e0116b74f0833
S: d7797d3410c1d43b918ee2e7c8cec8810a3fd519 192.168.1.202:6379
slots: (0 slots) slave
replicates df921d013a6cbf1588c1ad51809435fe39f6c25c
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.1.207:6379 to make it join the cluster.
[OK] New node added correctly.
查看 Redis7 在集群当中的 ID：
Redis7 的 ID：c34cbe9cba54da740cce295e318e13a1cb276d88
[root@redis2 redis5]# redis-cli -a 123456 --cluster check 192.168.1.201:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.1.201:6379 (feb6b432...) -> 0 keys | 5461 slots | 1 slaves.
192.168.1.205:6379 (df921d01...) -> 1 keys | 5461 slots | 1 slaves.
192.168.1.203:6379 (c2b48542...) -> 0 keys | 5462 slots | 1 slaves.
192.168.1.207:6379 (c34cbe9c...) -> 0 keys | 0 slots | 0 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.1.201:6379)
M: feb6b43233fdcc01e0d5425fd03e0116b74f0833 192.168.1.201:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: df921d013a6cbf1588c1ad51809435fe39f6c25c 192.168.1.205:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
M: c2b48542344ece56e69d8ca9404ee29a48ae7b8e 192.168.1.203:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: b11dfd69bd23ae0d987a136061d7fd15b500800a 192.168.1.206:6379
slots: (0 slots) slave
replicates c2b48542344ece56e69d8ca9404ee29a48ae7b8e
S: 90d1475f064839143bbaea911ac1e449bee174da 192.168.1.204:6379
slots: (0 slots) slave
replicates feb6b43233fdcc01e0d5425fd03e0116b74f0833
S: d7797d3410c1d43b918ee2e7c8cec8810a3fd519 192.168.1.202:6379
slots: (0 slots) slave
replicates df921d013a6cbf1588c1ad51809435fe39f6c25c
M: c34cbe9cba54da740cce295e318e13a1cb276d88 192.168.1.207:6379
slots: (0 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
添加 Slave：
[root@redis1 ~]# redis-cli -a 123456 --cluster add-node 192.168.1.208:6379 192.168.1.201:6379 --cluster-slave --cluster-master-id c34cbe9cba54da740cce295e318e13a1cb276d88
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Adding node 192.168.1.208:6379 to cluster 192.168.1.201:6379
>>> Performing Cluster Check (using node 192.168.1.201:6379)
M: feb6b43233fdcc01e0d5425fd03e0116b74f0833 192.168.1.201:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: df921d013a6cbf1588c1ad51809435fe39f6c25c 192.168.1.205:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
M: c2b48542344ece56e69d8ca9404ee29a48ae7b8e 192.168.1.203:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: b11dfd69bd23ae0d987a136061d7fd15b500800a 192.168.1.206:6379
slots: (0 slots) slave
replicates c2b48542344ece56e69d8ca9404ee29a48ae7b8e
S: 90d1475f064839143bbaea911ac1e449bee174da 192.168.1.204:6379
slots: (0 slots) slave
replicates feb6b43233fdcc01e0d5425fd03e0116b74f0833
S: d7797d3410c1d43b918ee2e7c8cec8810a3fd519 192.168.1.202:6379
slots: (0 slots) slave
replicates df921d013a6cbf1588c1ad51809435fe39f6c25c
M: c34cbe9cba54da740cce295e318e13a1cb276d88 192.168.1.207:6379
slots: (0 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.1.208:6379 to make it join the cluster.
Waiting for the cluster to join
>>> Configure node as replica of 192.168.1.207:6379.
[OK] New node added correctly.
重新分配槽位（Redis-3/4）
添加节点之后，需要重新分配集群中的槽位，否则新节点没有槽位就无法写入数据。
指定需要移动的槽位数量为 4096 个（16384 ÷ 4）；
指定接收槽位的节点为 Redis7；
指定槽位的源节点为 all；
[root@redis1 ~]# redis-trib reshard 192.168.1.201:6379
How many slots do you want to move (from 1 to 16384)? 4096
What is the receiving node ID? 4630b3bde39432a04e94414ec6b2582d26f7fc21
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1:all
重新分配槽位（Redis-5）
[root@redis1 ~]# redis-cli -a 123456 --cluster reshard 192.168.1.201:6379
How many slots do you want to move (from 1 to 16384)? 4096
What is the receiving node ID? c34cbe9cba54da740cce295e318e13a1cb276d88
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1: all
Do you want to proceed with the proposed reshard plan (yes/no)?yes
验证集群状态
redis-trib.rb（Redis-3/4）：
[root@redis1 ~]# redis-trib info 192.168.1.201:6379
192.168.1.201:6379 (469e0c16...) -> 0 keys | 4096 slots | 1 slaves.
192.168.1.205:6379 (1b897eda...) -> 1 keys | 4096 slots | 1 slaves.
192.168.1.207:6379 (4630b3bd...) -> 0 keys | 4096 slots | 1 slaves.
192.168.1.203:6379 (4dd0b78e...) -> 0 keys | 4096 slots | 1 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.
redis-cli（Redis-5）：
[root@redis2 ~]# redis-cli -a 123456 --cluster check 192.168.1.201:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.1.201:6379 (feb6b432...) -> 0 keys | 4096 slots | 1 slaves.
192.168.1.205:6379 (df921d01...) -> 1 keys | 4096 slots | 1 slaves.
192.168.1.203:6379 (c2b48542...) -> 0 keys | 4096 slots | 1 slaves.
192.168.1.207:6379 (c34cbe9c...) -> 0 keys | 4096 slots | 1 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.1.201:6379)
M: feb6b43233fdcc01e0d5425fd03e0116b74f0833 192.168.1.201:6379
slots:[1365-5460] (4096 slots) master
1 additional replica(s)
M: df921d013a6cbf1588c1ad51809435fe39f6c25c 192.168.1.205:6379
slots:[12288-16383] (4096 slots) master
1 additional replica(s)
M: c2b48542344ece56e69d8ca9404ee29a48ae7b8e 192.168.1.203:6379
slots:[6827-10922] (4096 slots) master
1 additional replica(s)
S: b11dfd69bd23ae0d987a136061d7fd15b500800a 192.168.1.206:6379
slots: (0 slots) slave
replicates c2b48542344ece56e69d8ca9404ee29a48ae7b8e
S: 90d1475f064839143bbaea911ac1e449bee174da 192.168.1.204:6379
slots: (0 slots) slave
replicates feb6b43233fdcc01e0d5425fd03e0116b74f0833
S: d7797d3410c1d43b918ee2e7c8cec8810a3fd519 192.168.1.202:6379
slots: (0 slots) slave
replicates df921d013a6cbf1588c1ad51809435fe39f6c25c
S: 52724938d32b56c974249fa33a82f2d2ee54f7bb 192.168.1.208:6379
slots: (0 slots) slave
replicates c34cbe9cba54da740cce295e318e13a1cb276d88
M: c34cbe9cba54da740cce295e318e13a1cb276d88 192.168.1.207:6379
slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
2.5.2：动态删除节点
添加节点的时候，是先添加节点到集群，然后分配槽位；删除节点的操作与其正好相反，是先将节点上的槽位迁移到集群中的其他节点上，然后再将其删除；
如果一个节点上的槽位没有被完全迁移，是无法将其删除的。
迁移槽位（Redis-3/4）
查看要删除的节点中有多少个槽位：
这里假设要删除 Redis3（192.168.1.203），其中的槽位数量为 4096；
[root@redis1 ~]# redis-trib info 192.168.1.201:6379
192.168.1.201:6379 (469e0c16...) -> 0 keys | 4096 slots | 1 slaves.
192.168.1.205:6379 (1b897eda...) -> 1 keys | 4096 slots | 1 slaves.
192.168.1.207:6379 (4630b3bd...) -> 0 keys | 4096 slots | 1 slaves.
192.168.1.203:6379 (4dd0b78e...) -> 0 keys | 4096 slots | 1 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.
查看接收槽位的节点的 ID：
假设要将 Redis3 的槽位分配给 Redis7，记录二者的 ID：
Redis3：4dd0b78edebcfe93c93518b5817c00cc21bef07e
Redis7：4630b3bde39432a04e94414ec6b2582d26f7fc21
实际生产环境中应该是这样的场景：
Redis3 因硬件老化等原因，需要更换服务器；
此时先将新的服务器以 Master 的形式加入集群；
然后将 Redis3 的数据导出，确保槽位中无数据；
将 Redis3 的槽位迁移到新服务器；
最后移除 Redis3，向新服务器中导入数据；
[root@redis1 ~]# redis-trib check 192.168.1.201:6379
>>> Performing Cluster Check (using node 192.168.1.201:6379)
M: 469e0c16495598b5fadba28f856203e206464b99 192.168.1.201:6379
slots:1365-5460 (4096 slots) master
1 additional replica(s)
S: 674a2a0e6a4c9a88db493c649bbd584bc6ba1edc 192.168.1.202:6379
slots: (0 slots) slave
replicates 1b897eda6acfb806eb27326f151e4067f0d47f7b
S: 86664f23678b53c6adc838ede087cdca4080a85b 192.168.1.208:6379
slots: (0 slots) slave
replicates 4630b3bde39432a04e94414ec6b2582d26f7fc21
M: 1b897eda6acfb806eb27326f151e4067f0d47f7b 192.168.1.205:6379
slots:12288-16383 (4096 slots) master
1 additional replica(s)
M: 4630b3bde39432a04e94414ec6b2582d26f7fc21 192.168.1.207:6379
slots:0-1364,5461-6826,10923-12287 (4096 slots) master
1 additional replica(s)
S: 6ec84046944b4b82af12d8fbc2977c54fa5f670c 192.168.1.204:6379
slots: (0 slots) slave
replicates 469e0c16495598b5fadba28f856203e206464b99
M: 4dd0b78edebcfe93c93518b5817c00cc21bef07e 192.168.1.203:6379
slots:6827-10922 (4096 slots) master
1 additional replica(s)
S: 9c4b8d3a34266ba89cacb80ccf170c34169a0dd9 192.168.1.206:6379
slots: (0 slots) slave
replicates 4dd0b78edebcfe93c93518b5817c00cc21bef07e
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
重新分配槽位，将 Redis3 中的槽位迁移 Redis7 上：
迁移槽位前，需要保证槽位中没有数据；
[root@redis1 ~]# redis-trib reshard 192.168.1.201:6379
How many slots do you want to move (from 1 to 16384)? 4096
What is the receiving node ID? 4630b3bde39432a04e94414ec6b2582d26f7fc21
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1:4dd0b78edebcfe93c93518b5817c00cc21bef07e
Source node #2:done
如果迁移失败，使用此命令修复集群：
redis-trib.rb fix 192.168.7101:6379
验证槽位迁移完成：
Redis3 中已无槽位；
[root@redis1 ~]# redis-trib info 192.168.1.201:6379
192.168.1.201:6379 (469e0c16...) -> 0 keys | 4096 slots | 1 slaves.
192.168.1.205:6379 (1b897eda...) -> 0 keys | 4096 slots | 1 slaves.
192.168.1.207:6379 (4630b3bd...) -> 0 keys | 8192 slots | 2 slaves.
192.168.1.203:6379 (4dd0b78e...) -> 0 keys | 0 slots | 0 slaves.
[OK] 0 keys in 4 masters.
0.00 keys per slot on average.
迁移槽位（Redis-5）
Redis3 的 ID：c2b48542344ece56e69d8ca9404ee29a48ae7b8e
Redis7 的 ID：c34cbe9cba54da740cce295e318e13a1cb276d88
[root@redis1 ~]# redis-cli -a 123456 --cluster reshard 192.168.1.201:6379
How many slots do you want to move (from 1 to 16384)? 4096
What is the receiving node ID? c34cbe9cba54da740cce295e318e13a1cb276d88
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1: c2b48542344ece56e69d8ca9404ee29a48ae7b8e
Source node #2: done
Do you want to proceed with the proposed reshard plan (yes/no)? yes
验证槽位迁移完成：
Redis3 中已经没有槽位，而 Redis7 的槽位数位 8192；
[root@redis2 ~]# redis-cli -a 123456 --cluster check 192.168.1.201:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.1.201:6379 (feb6b432...) -> 0 keys | 4096 slots | 1 slaves.
192.168.1.205:6379 (df921d01...) -> 1 keys | 4096 slots | 1 slaves.
192.168.1.203:6379 (c2b48542...) -> 0 keys | 0 slots | 0 slaves.
192.168.1.207:6379 (c34cbe9c...) -> 0 keys | 8192 slots | 2 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.1.201:6379)
M: feb6b43233fdcc01e0d5425fd03e0116b74f0833 192.168.1.201:6379
slots:[1365-5460] (4096 slots) master
1 additional replica(s)
M: df921d013a6cbf1588c1ad51809435fe39f6c25c 192.168.1.205:6379
slots:[12288-16383] (4096 slots) master
1 additional replica(s)
M: c2b48542344ece56e69d8ca9404ee29a48ae7b8e 192.168.1.203:6379
slots: (0 slots) master
S: b11dfd69bd23ae0d987a136061d7fd15b500800a 192.168.1.206:6379
slots: (0 slots) slave
replicates c34cbe9cba54da740cce295e318e13a1cb276d88
S: 90d1475f064839143bbaea911ac1e449bee174da 192.168.1.204:6379
slots: (0 slots) slave
replicates feb6b43233fdcc01e0d5425fd03e0116b74f0833
S: d7797d3410c1d43b918ee2e7c8cec8810a3fd519 192.168.1.202:6379
slots: (0 slots) slave
replicates df921d013a6cbf1588c1ad51809435fe39f6c25c
S: 52724938d32b56c974249fa33a82f2d2ee54f7bb 192.168.1.208:6379
slots: (0 slots) slave
replicates c34cbe9cba54da740cce295e318e13a1cb276d88
M: c34cbe9cba54da740cce295e318e13a1cb276d88 192.168.1.207:6379
slots:[0-1364],[5461-12287] (8192 slots) master
2 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
从集群删除节点（Redis-3/4）
删除 Master：
[root@redis1 ~]# redis-trib del-node 192.168.1.201:6379 4dd0b78edebcfe93c93518b5817c00cc21bef07e
>>> Removing node 4dd0b78edebcfe93c93518b5817c00cc21bef07e from cluster 192.168.1.201:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
Master 被删除之后，其之前的 Slave 会自动成为 Redis 集群中其他 Master 的 Slave，如果不需要，也可以一并删除。
Redis7 目前有 2 个 replicas（之前 Redis3 的 Salve（Redis6）也成为 Redis7 的 Slave：
[root@redis1 ~]# redis-trib check 192.168.1.201:6379 >>> Performing Cluster Check (using node 192.168.1.201:6379) M: 469e0c16495598b5fadba28f856203e206464b99 192.168.1.201:6379 slots:2509-5460,12288-13431 (4096 slots) master 1 additional replica(s) S: 674a2a0e6a4c9a88db493c649bbd584bc6ba1edc 192.168.1.202:6379 slots: (0 slots) slave replicates 1b897eda6acfb806eb27326f151e4067f0d47f7b S: 86664f23678b53c6adc838ede087cdca4080a85b 192.168.1.208:6379 slots: (0 slots) slave replicates 4630b3bde39432a04e94414ec6b2582d26f7fc21 M: 1b897eda6acfb806eb27326f151e4067f0d47f7b 192.168.1.205:6379 slots:1365-2508,13432-16383 (4096 slots) master 1 additional replica(s) M: 4630b3bde39432a04e94414ec6b2582d26f7fc21 192.168.1.207:6379 slots:0-1364,5461-12287 (8192 slots) master 2 additional replica(s) S: 6ec84046944b4b82af12d8fbc2977c54fa5f670c 192.168.1.204:6379 slots: (0 slots) slave replicates 469e0c16495598b5fadba28f856203e206464b99 S: 9c4b8d3a34266ba89cacb80ccf170c34169a0dd9 192.168.1.206:6379 slots: (0 slots) slave replicates 4630b3bde39432a04e94414ec6b2582d26f7fc21 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
删除 Slave（Redis6）：
[root@redis1 ~]# redis-trib del-node 192.168.1.201:6379 9c4b8d3a34266ba89cacb80ccf170c34169a0dd9
>>> Removing node 9c4b8d3a34266ba89cacb80ccf170c34169a0dd9 from cluster 192.168.1.201:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
从集群删除节点（Redis-5）
删除 Master：
[root@redis2 ~]# redis-cli -a 123456 --cluster del-node 192.168.1.201:6379 c2b48542344ece56e69d8ca9404ee29a48ae7b8e
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Removing node c2b48542344ece56e69d8ca9404ee29a48ae7b8e from cluster 192.168.1.201:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
删除 Slave：
Redis3 的 Slave 为 Redis6，ID为：b11dfd69bd23ae0d987a136061d7fd15b500800a
[root@redis2 ~]# redis-cli -a 123456 --cluster del-node 192.168.1.201:6379 b11dfd69bd23ae0d987a136061d7fd15b500800a
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Removing node b11dfd69bd23ae0d987a136061d7fd15b500800a from cluster 192.168.1.201:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
验证集群状态
redis-trib.rb（Redis-3/4）：
[root@redis1 ~]# redis-trib check 192.168.1.201:6379
>>> Performing Cluster Check (using node 192.168.1.201:6379)
M: 469e0c16495598b5fadba28f856203e206464b99 192.168.1.201:6379
slots:2509-5460,12288-13431 (4096 slots) master
1 additional replica(s)
S: 674a2a0e6a4c9a88db493c649bbd584bc6ba1edc 192.168.1.202:6379
slots: (0 slots) slave
replicates 1b897eda6acfb806eb27326f151e4067f0d47f7b
S: 86664f23678b53c6adc838ede087cdca4080a85b 192.168.1.208:6379
slots: (0 slots) slave
replicates 4630b3bde39432a04e94414ec6b2582d26f7fc21
M: 1b897eda6acfb806eb27326f151e4067f0d47f7b 192.168.1.205:6379
slots:1365-2508,13432-16383 (4096 slots) master
1 additional replica(s)
M: 4630b3bde39432a04e94414ec6b2582d26f7fc21 192.168.1.207:6379
slots:0-1364,5461-12287 (8192 slots) master
1 additional replica(s)
S: 6ec84046944b4b82af12d8fbc2977c54fa5f670c 192.168.1.204:6379
slots: (0 slots) slave
replicates 469e0c16495598b5fadba28f856203e206464b99
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
redis-cli（Redis-5）：
[root@redis2 ~]# redis-cli -a 123456 --cluster check 192.168.1.201:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.1.201:6379 (feb6b432...) -> 0 keys | 4096 slots | 1 slaves.
192.168.1.205:6379 (df921d01...) -> 1 keys | 4096 slots | 1 slaves.
192.168.1.207:6379 (c34cbe9c...) -> 0 keys | 8192 slots | 1 slaves.
[OK] 1 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.1.201:6379)
M: feb6b43233fdcc01e0d5425fd03e0116b74f0833 192.168.1.201:6379
slots:[1365-5460] (4096 slots) master
1 additional replica(s)
M: df921d013a6cbf1588c1ad51809435fe39f6c25c 192.168.1.205:6379
slots:[12288-16383] (4096 slots) master
1 additional replica(s)
S: 90d1475f064839143bbaea911ac1e449bee174da 192.168.1.204:6379
slots: (0 slots) slave
replicates feb6b43233fdcc01e0d5425fd03e0116b74f0833
S: d7797d3410c1d43b918ee2e7c8cec8810a3fd519 192.168.1.202:6379
slots: (0 slots) slave
replicates df921d013a6cbf1588c1ad51809435fe39f6c25c
S: 52724938d32b56c974249fa33a82f2d2ee54f7bb 192.168.1.208:6379
slots: (0 slots) slave
replicates c34cbe9cba54da740cce295e318e13a1cb276d88
M: c34cbe9cba54da740cce295e318e13a1cb276d88 192.168.1.207:6379
slots:[0-1364],[5461-12287] (8192 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
2.5.3：模拟 Master 故障
测试集群中某个 Master 故障时，其 Slave 是否会自动提升为 Master 并接管业务；
测试过程：
- 在集群中的一台 Master 中写入测试数据，并保证 Slave 同步数据成功；
- 停止这台 Master 的 Redis 服务；
- 验证其 Slave 是否切换为 Master；
- 测试 Slave 切换为 Master 之后的数据读写；
- 重新启动原 Master 的 Redis 服务，查看集群状态；
Master 写入数据
在其中一个 Master 中写入测试数据：
在 Redis7 中写入 foo2；
[root@redis1 ~]# redis-cli
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> SET foo2 bar2
(error) MOVED 1044 192.168.1.207:6379
[root@redis7 ~]# redis-cli
127.0.0.1:6379> AUTH 123456
OK
127.0.0.1:6379> SET foo2 bar2
OK
验证 Slave 同步的数据：
Redis7 的 Slave 为 Redis8；
[root@redis8 ~]# redis-cli
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> KEYS foo2
1) "foo2"
停止 Master 的 Redis 服务
停止 Redis7 的 Redis：
[root@redis7 ~]# systemctl stop redis
验证 Slave 切换为 Master
查看 Slave 的日志：
# 失去和 Master 的连接：
2856:S 04 Jan 18:06:32.015 # Connection with master lost.
2856:S 04 Jan 18:06:32.015 * Caching the disconnected master state.
2856:S 04 Jan 18:06:32.297 * Connecting to MASTER 192.168.1.207:6379
2856:S 04 Jan 18:06:32.298 * MASTER <-> SLAVE sync started
2856:S 04 Jan 18:06:32.298 # Error condition on socket for SYNC: Connection refused
# 标记 Master 的状态为“失败”，并在延迟时间到达时开始选举（538ms后）；
2856:S 04 Jan 18:06:47.806 * Marking node 4630b3bde39432a04e94414ec6b2582d26f7fc21 as failing (quorum reached).
2856:S 04 Jan 18:06:47.806 # Cluster state changed: fail
2856:S 04 Jan 18:06:47.823 # Start of election delayed for 538 milliseconds (rank #0, offset 308).
2856:S 04 Jan 18:06:48.438 # Starting a failover election for epoch 11.
# 在此同时，仍然会不断尝试连接 Master：
2856:S 04 Jan 18:07:02.975 * Connecting to MASTER 192.168.1.207:6379
2856:S 04 Jan 18:07:02.975 * MASTER <-> SLAVE sync started
2856:S 04 Jan 18:07:02.976 # Error condition on socket for SYNC: Connection refused
……
# 提示目前还无法 Failover，还在等待其它节点对 Redis7 的检测结果（认为无法检测到 Redis7 的节点数量还未达到大多数）
2856:S 04 Jan 18:07:07.885 # Currently unable to failover: Waiting for votes, but majority still not reached.
# 提示将在 686ms 后执行 failover 选举：
2856:S 04 Jan 18:07:48.415 # Start of election delayed for 686 milliseconds (rank #0, offset 308).
2856:S 04 Jan 18:07:48.518 # Currently unable to failover: Waiting the delay before I can start a new failover.
# Salve 提升为 Master（因为只有一个 Slave，所以直接提升为 Master）：
2856:S 04 Jan 18:07:49.132 # Starting a failover election for epoch 12.
2856:S 04 Jan 18:07:49.139 # Failover election won: I'm the new master.
2856:S 04 Jan 18:07:49.139 # configEpoch set to 12 after successful failover
2856:M 04 Jan 18:07:49.139 # Setting secondary replication ID to 26954fa49f5d2c273df73d6c9ef41237c026f534, valid up to offset: 309. New replication ID is 2a72ebfe32268
3419742a55ea11b98ab36f82987
2856:M 04 Jan 18:07:49.140 * Discarding previously cached master state.
查看 Slave 的状态，已是 master 角色：
[root@redis8 ~]# redis-cli
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> INFO Replication
# Replication
role:master
connected_slaves:0
master_replid:2a72ebfe322683419742a55ea11b98ab36f82987
master_replid2:26954fa49f5d2c273df73d6c9ef41237c026f534
master_repl_offset:308
second_repl_offset:309
repl_backlog_active:1
repl_backlog_size:104857600
repl_backlog_first_byte_offset:1
repl_backlog_histlen:308
查看当前集群状态：
做实验时，发现 Master 虽然切换成功，但是集群状态时 fail 的：
127.0.0.1:6379> CLUSTER INFO cluster_state:fail cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:12 cluster_my_epoch:12 cluster_stats_messages_ping_sent:10764 cluster_stats_messages_pong_sent:3831 cluster_stats_messages_meet_sent:7 cluster_stats_messages_auth-req_sent:10 cluster_stats_messages_update_sent:7 cluster_stats_messages_sent:14619 cluster_stats_messages_ping_received:3827 cluster_stats_messages_pong_received:3725 cluster_stats_messages_fail_received:2 cluster_stats_messages_auth-ack_received:2 cluster_stats_messages_update_received:1 cluster_stats_messages_received:7557重启 Redis8 的 Redis 进程后，日志中才出现集群 OK 的提示：
15157:M 04 Jan 18:20:18.186 # Cluster state changed: ok还未找到集群状态切换不成功的原因；
[root@redis8 ~]# redis-cli
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> CLUSTER INFO
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:12
cluster_my_epoch:12
cluster_stats_messages_ping_sent:2002
cluster_stats_messages_pong_sent:203
cluster_stats_messages_sent:2205
cluster_stats_messages_ping_received:203
cluster_stats_messages_pong_received:183
cluster_stats_messages_received:386
查看集群中的节点情况：
Redis7 的状态标记为 fail，Redis8 已为 master；
127.0.0.1:6379> CLUSTER NODES
1b897eda6acfb806eb27326f151e4067f0d47f7b 192.168.1.205:6379@16379 master - 0 1609755832019 9 connected 1365-2508 13432-16383
86664f23678b53c6adc838ede087cdca4080a85b 192.168.1.208:6379@16379 myself,master - 0 1609755832000 12 connected 0-1364 5461-12287
469e0c16495598b5fadba28f856203e206464b99 192.168.1.201:6379@16379 master - 0 1609755833042 8 connected 2509-5460 12288-13431
4630b3bde39432a04e94414ec6b2582d26f7fc21 192.168.1.207:6379@16379 master,fail - 1609755616135 1609755616135 10 disconnected
674a2a0e6a4c9a88db493c649bbd584bc6ba1edc 192.168.1.202:6379@16379 slave 1b897eda6acfb806eb27326f151e4067f0d47f7b 0 1609755831000 9 connected
6ec84046944b4b82af12d8fbc2977c54fa5f670c 192.168.1.204:6379@16379 slave 469e0c16495598b5fadba28f856203e206464b99 0 1609755830000 8 connected
验证数据读写
Redis8 读取之前的 foo2：
127.0.0.1:6379> GET foo2
"bar2"
Redis8 写入数据：
127.0.0.1:6379> SET foo4 bar4
OK
127.0.0.1:6379> GET foo4
"bar4"
Redis8 已经可以对外提供读写服务了；
恢复原 Master 的 Redis 服务
启动 Redis7 的 Redis 进程：
[root@redis7 ~]# systemctl start redis
查看 Redis7 的 Replication 状态：
成为 Redis8 的 Slave；
127.0.0.1:6379> INFO Replication
# Replication
role:slave
master_host:192.168.1.208
master_port:6379
master_link_status:up
master_last_io_seconds_ago:3
master_sync_in_progress:0
slave_repl_offset:182
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:06b3de5832fb3bd07fccf94b78a12414083300d0
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:182
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:104857600
repl_backlog_first_byte_offset:1
repl_backlog_histlen:182
查看集群状态：
[root@redis7 ~]# redis-cli
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> CLUSTER INFO
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:12
cluster_my_epoch:12
cluster_stats_messages_ping_sent:64
cluster_stats_messages_pong_sent:67
cluster_stats_messages_sent:131
cluster_stats_messages_ping_received:67
cluster_stats_messages_pong_received:64
cluster_stats_messages_update_received:2
cluster_stats_messages_received:133
查看集群中的节点情况：
cluster_stats_messages_received:133
127.0.0.1:6379> CLUSTER NODES
6ec84046944b4b82af12d8fbc2977c54fa5f670c 192.168.1.204:6379@16379 slave 469e0c16495598b5fadba28f856203e206464b99 0 1609756105000 8 connected
4630b3bde39432a04e94414ec6b2582d26f7fc21 192.168.1.207:6379@16379 myself,slave 86664f23678b53c6adc838ede087cdca4080a85b 0 1609756102000 10 connected
1b897eda6acfb806eb27326f151e4067f0d47f7b 192.168.1.205:6379@16379 master - 0 1609756106000 9 connected 1365-2508 13432-16383
469e0c16495598b5fadba28f856203e206464b99 192.168.1.201:6379@16379 master - 0 1609756106517 8 connected 2509-5460 12288-13431
86664f23678b53c6adc838ede087cdca4080a85b 192.168.1.208:6379@16379 master - 0 1609756107537 12 connected 0-1364 5461-12287
674a2a0e6a4c9a88db493c649bbd584bc6ba1edc 192.168.1.202:6379@16379 slave 1b897eda6acfb806eb27326f151e4067f0d47f7b 0 1609756105489 9 connected
验证 Redis7 的数据：
后来添加的 foo4 也同步了过来；
127.0.0.1:6379> KEYS foo*
1) "foo2"
2) "foo4"
说明，集群中某个 Master 不可用时，其 Slaves 会通过选举来产生新的 Master，并在集群中继续对外提供服务；
当原来的 Master 重新恢复后，会成为新 Master 的 Slave，并同步目前 Master 中的数据；
2.5.4：集群导入数据
应用场景：
Redis Cluster 部署完成之后，需要将之前的单机 Redis 数据导入到 Redis Cluster，因为 Redis Cluster 使用的是分槽位保存 key 的机制，因此无法使用传统的 AOF 或 RDB 导入，需要使用集群数据导入命令。
向集群中导入数据需要 Redis Cluster 中不能有与导入数据中相同的 key，否则导入不成功或中断（可以加参数 --cluster-replace 强制替换 Redis Cluster 中已有的 key）。
准备源 Redis 数据
192.168.1.106 安装 Redis，并启动为单机 Redis：
[root@node106 ~]# systemctl start redis
写入测试数据：
[root@node106 ~]# vim redis.sh
#!/bin/bash
NUM=`seq 1 1000`
PASS='123456'
for i in ${NUM};do
redis-cli -h 127.0.0.1 -a ${PASS} set key-${i} value-${i} &>/dev/null
done
echo "数据写入完成"
[root@node106 ~]# bash redis.sh
数据写入完成
[root@node106 ~]# redis-cli
127.0.0.1:6379> auth 123456
OK
127.0.0.1:6379> GET key-111
"value-111"
导入数据前的准备
导入数据之前，需要关闭集群中各节点 Redis 服务的连接密码，以及源 Redis Server 的连接密码，避免认证带来的环境不一致而无法导入。
关闭源 Redis Server 的连接密码：
[root@node106 ~]# redis-cli -a 123456 CONFIG SET requirepass ""
关闭目标 Redis Cluster 中各节点的 Redis 连接密码：
[root@redis1 ~]# redis-cli -h 192.168.1.201 -a 123456 CONFIG SET requirepass ""
[root@redis1 ~]# redis-cli -h 192.168.1.202 -a 123456 CONFIG SET requirepass ""
[root@redis1 ~]# redis-cli -h 192.168.1.204 -a 123456 CONFIG SET requirepass ""
[root@redis1 ~]# redis-cli -h 192.168.1.205 -a 123456 CONFIG SET requirepass ""
[root@redis1 ~]# redis-cli -h 192.168.1.207 -a 123456 CONFIG SET requirepass ""
[root@redis1 ~]# redis-cli -h 192.168.1.208 -a 123456 CONFIG SET requirepass ""
取消 Ruby Redis 模块配置的 Redis 连接密码：
[root@redis1 ~]# vim /usr/local/lib/ruby/gems/2.5.0/gems/redis-4.2.5/lib/redis/client.rb
class Redis
class Client
# Defaults are also used for converting string keys to symbols.
DEFAULTS = {
……
password: nil,
执行数据导入（Redis-3/4）
[root@redis1 ~]# redis-trib import --from 192.168.1.106:6379 --replace 192.168.1.201:6379
报错：
Migrating key-350 to 192.168.1.201:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY) Migrating key-906 to 192.168.1.208:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY) Migrating key-503 to 192.168.1.201:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY) Migrating key-597 to 192.168.1.205:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY) Migrating key-934 to 192.168.1.208:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY) Migrating key-617 to 192.168.1.208:6379: ERR Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)
Redis-4.0.14 组成的 Redis Cluster 数据导入测试没有成功；
执行数据导入（Redis-5）
[root@redis1 ~]# redis-cli --cluster import 192.168.1.201:6379 --cluster-from 192.168.1.106:6379 --cluster-copy
验证数据导入（Redis-5）
各节点已有 keys 存入：
[root@redis2 ~]# redis-cli --cluster info 192.168.1.201:6379
192.168.1.201:6379 (feb6b432...) -> 249 keys | 4096 slots | 1 slaves.
192.168.1.205:6379 (df921d01...) -> 250 keys | 4096 slots | 1 slaves.
192.168.1.207:6379 (c34cbe9c...) -> 502 keys | 8192 slots | 1 slaves.
[OK] 1001 keys in 3 masters.
0.06 keys per slot on average.
读取导入的 key：
[root@redis1 ~]# redis-cli
127.0.0.1:6379> GET key-222
(error) MOVED 317 192.168.1.207:6379
[root@redis7 ~]# redis-cli
127.0.0.1:6379> GET key-222
"value-222"