redis-cluster 创建及监控
集群命令
cluster info：打印集群的信息。
cluster nodes：列出集群当前已知的所有节点（node）的相关信息。
cluster meet ：将ip和port所指定的节点添加到集群当中。
cluster addslots [slot ...]：将一个或多个槽（slot）指派（assign）给当前节点。
cluster delslots [slot ...]：移除一个或多个槽对当前节点的指派。
cluster slots：列出槽位、节点信息。
cluster slaves ：列出指定节点下面的从节点信息。 比如： cluster slaves 11f9169577352c33d85ad0d1ca5f5bf0deba3209 这个实际查的是nodes.conf
cluster replicate ：将当前节点设置为指定节点的从节点。
cluster saveconfig： 手动执行命令保存保存集群的配置文件，集群默认在配置修改的时候会自动保存配置文件。
cluster keyslot ： 列出key被放置在哪个槽上。 例如：cluster keyslot 9223372036854742675
cluster flushslots： 移除指派给当前节点的所有槽，让当前节点变成一个没有指派任何槽的节点。
cluster countkeysinslot ： 返回槽目前包含的键值对数量。
cluster getkeysinslot ：返回count个槽中的键。 比如：cluster getkeysinslot 202 3
cluster setslot node 将槽指派给指定的节点，如果槽已经指派给另一个节点，那么先让另一个节点删除该槽，然后再进行指派。
cluster setslot migrating 将本节点的槽迁移到指定的节点中。
cluster setslot importing 从 node_id 指定的节点中导入槽 slot 到本节点。
cluster setslot stable 取消对槽 slot 的导入（import）或者迁移（migrate）。
cluster failover： 手动进行故障转移。
cluster forget ： 从集群中移除指定的节点，这样就无法完成握手，过期时为60s，60s后两节点又会继续完成握手。
cluster reset [HARD|SOFT]：重置集群信息，soft是清空其他节点的信息，但不修改自己的id，hard还会修改自己的id，不传该参数则使用soft方式。
cluster count-failure-reports ：列出某个节点的故障报告的长度。
cluster SET-CONFIG-EPOCH： 设置节点epoch，只有在节点加入集群前才能设置。
集群环境搭建：
搭建集群至少需要6台服务器或虚拟机，如果想要搭建测试可以根据端口号模拟服务器虚拟运行6个redis实例。
（1）下载redis安装包，编译安装，指定目录/usr/local/redis/
（2）创建/usr/local/redis-cluster/目录，然后把安装目录下的bin目录复制到redis-cluster/下重命名为redis01~06
（3）修改配置文件
#bind 127.0.0.1 注释，允许其他ip访问
protected-mode no # 改为no; yes ：保护模式，只允许本地链接，no ：保护模式关闭
port 7001/7002/7003/7004/7005/7006 # 六个配置文件分别配置不同端口
daemonize: yes # 后台运行
masterauth 1234 # 主机密码
requirepass 1234 # 本机密码
cluster-enabled yes # 开启集群
cluster-enabled cluster-config-file nodes-7001.conf # 集群配置文件；集群创建成功会自动生成不配置默认nodes.conf
cluster-node-timeout 15000 # 节点失联时间，超过该时间（毫秒），集群自动进行主从切换。
appendonly：yes # 开启AOF持久化
（4）创建集群，中途会有个提示，输入yes回车即可
/usr/local/redis/bin/redis-cli --cluster create --cluster-replicas 1 182.254.134.174:7001 182.254.134.174:7002 182.254.134.174:7003 182.254.134.174:7004 182.254.134.174:7005 182.254.134.174:7006 -a 1234
[root@VM-0-12-centos /]# cd /usr/local/redis
[root@VM-0-12-centos redis]# wget https://github.com/redis/redis/archive/7.0.4.tar.gz
[root@VM-0-12-centos redis]# tar -xzvf 7.0.4.tar.gz
[root@VM-0-12-centos redis]# ll
total 2932
-rw-r--r-- 1 root root 2994242 Aug 8 17:03 7.0.4.tar.gz
drwxrwxr-x 8 root root 4096 Jul 18 21:04 redis-7.0.4
[root@VM-0-12-centos redis]# cd redis-7.0.4
[root@VM-0-12-centos redis]# make && make install PREFIX=/usr/local/redis # 指定安装目录,会在redis目录下生成一个bin/目录,如果不指定安装目录会在 redis-7.0.4/src/ 生成启动文件，我这里为了自己操作方便些直接在 redis/bin 目录下操作
[root@VM-0-12-centos redis]# cd ..
[root@VM-0-12-centos redis]# ls
7.0.4.tar.gz bin redis-7.0.4
[root@VM-0-12-centos redis]# cd bin/
[root@VM-0-12-centos bin]# ls
redis-benchmark redis-check-aof redis-check-rdb redis-cli redis-sentinel redis-server
[root@VM-0-12-centos bin]# cp ../redis-7.0.4/redis.conf ./
[root@VM-0-12-centos bin]# ls
redis-benchmark redis-check-aof redis-check-rdb redis-cli redis.conf redis-sentinel redis-server
[root@VM-0-12-centos bin]# vim ./redis.conf # 修改daemonize 的值为 yes，开启后台启动
[root@VM-0-12-centos bin]# ./redis-server ./redis.conf
[root@VM-0-12-centos bin]# ./redis-cli
127.0.0.1:6379>
root@VM-0-12-centos bin]# cd ../..
[root@VM-0-12-centos local]# mkdir redis-cluster
[root@VM-0-12-centos local]# cd redis-cluster/
[root@VM-0-12-centos redis-cluster]# cp -r /usr/local/redis/bin/ ./redis01/
[root@VM-0-12-centos local]# rm -rf ./redis01/dump.rdb # 删除redis01目录下的快照文件dump.rdb
[root@VM-0-12-centos redis-cluster]# vim ./redis01/redis.conf
#bind 127.0.0.1 注释，允许其他ip访问
protected-mode no
port 7001
masterauth 1234
requirepass 1234 #最好设置一下密码
cluster-enabled yes
cluster-enabled yes 打开集群模式
cluster-config-file nodes-7001.conf 设定节点配置文件名
cluster-node-timeout 15000
appendonly：yes
[root@VM-0-12-centos redis-cluster]# cp -r ./redis01/ ./redis02/
[root@VM-0-12-centos redis-cluster]# cp -r ./redis01/ ./redis03/
[root@VM-0-12-centos redis-cluster]# cp -r ./redis01/ ./redis04/
[root@VM-0-12-centos redis-cluster]# cp -r ./redis01/ ./redis05/
[root@VM-0-12-centos redis-cluster]# cp -r ./redis01/ ./redis06/
[root@VM-0-12-centos redis-cluster]# vim ./redis01/redis.conf # 逐个修改端口号
[root@VM-0-12-centos redis-cluster]# vim start-all.sh # 创建启动文件，也可手动启动所有服务
cd redis01
./redis-server ./redis.conf
cd ..
cd redis02
./redis-server ./redis.conf
cd ..
cd redis03
./redis-server ./redis.conf
cd ..
cd redis04
./redis-server ./redis.conf
cd ..
cd redis05
./redis-server ./redis.conf
cd ..
cd redis06
./redis-server ./redis.conf
cd ..
[root@VM-0-12-centos redis-cluster]# vim stop-all.sh # 创建关闭文件，也可手动停止所有服务
cd redis01
./redis-cli -h 127.0.0.1 -p 7001 -a 1234 shutdown
cd ..
cd redis02
./redis-cli -h 127.0.0.1 -p 7002 -a 1234 shutdown
cd ..
cd redis03
./redis-cli -h 127.0.0.1 -p 7003 -a 1234 shutdown
cd ..
cd redis04
./redis-cli -h 127.0.0.1 -p 7004 -a 1234 shutdown
cd ..
cd redis05
./redis-cli -h 127.0.0.1 -p 7005 -a 1234 shutdown
cd ..
cd redis06
./redis-cli -h 127.0.0.1 -p 7006 -a 1234 shutdown
cd ..
[root@VM-0-12-centos redis-cluster]# chmod +x start-all.sh
[root@VM-0-12-centos redis-cluster]# chmod +x stop-all.sh
[root@VM-0-12-centos redis-cluster]# ./start-all.sh
[root@VM-0-12-centos redis-cluster]# ps aux | grep redis
root 17885 0.1 0.1 163152 3724 ? Ssl 17:52 0:01 ./redis-server *:6379
root 23399 0.0 0.1 165712 3664 ? Ssl 18:20 0:00 ./redis-server *:7001 [cluster]
root 23401 0.0 0.1 165712 3660 ? Ssl 18:20 0:00 ./redis-server *:7002 [cluster]
root 23403 0.0 0.1 165712 3664 ? Ssl 18:20 0:00 ./redis-server *:7003 [cluster]
root 23405 0.0 0.1 165712 3664 ? Ssl 18:20 0:00 ./redis-server *:7004 [cluster]
root 23411 0.0 0.1 171856 3664 ? Ssl 18:20 0:00 ./redis-server *:7005 [cluster]
root 23413 0.0 0.1 165712 3672 ? Ssl 18:20 0:00 ./redis-server *:7006 [cluster]
root 23495 0.0 0.0 112816 976 pts/0 R+ 18:21 0:00 grep --color=auto redis
[root@VM-0-12-centos redis-cluster]# ls
redis01 redis02 redis03 redis04 redis05 redis06 start-all.sh stop-all.sh
[root@VM-0-12-centos redis-cluster]# /usr/local/redis/bin/redis-cli --cluster create --cluster-replicas 1 182.254.134.174:7001 182.254.134.174:7002 182.254.134.174:7003 182.254.134.174:7004 182.254.134.174:7005 182.254.134.174:7006 -a 1234
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 182.254.134.174:7005 to 182.254.134.174:7001
Adding replica 182.254.134.174:7006 to 182.254.134.174:7002
Adding replica 182.254.134.174:7004 to 182.254.134.174:7003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: e9750b2aa5a23b0d5e79e98b3a61c6f97a69f01d 182.254.134.174:7001
slots:[0-5460] (5461 slots) master
M: 1194702cf2b7ef4f5d5e7854f526aa60d47f2577 182.254.134.174:7002
slots:[5461-10922] (5462 slots) master
M: b120e1a401f11a06f8d81f19df2607c71c5cf9df 182.254.134.174:7003
slots:[10923-16383] (5461 slots) master
S: 28362fb68d33eb50322f514daf4e7a447e3eccb3 182.254.134.174:7004
replicates 1194702cf2b7ef4f5d5e7854f526aa60d47f2577
S: 52b90d0718c06990de002dbe00be512c0f4eccd1 182.254.134.174:7005
replicates b120e1a401f11a06f8d81f19df2607c71c5cf9df
S: 2aedbda7dadc48e545ec0870b7d3585bb4637c47 182.254.134.174:7006
replicates e9750b2aa5a23b0d5e79e98b3a61c6f97a69f01d
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
..
>>> Performing Cluster Check (using node 182.254.134.174:7001)
M: e9750b2aa5a23b0d5e79e98b3a61c6f97a69f01d 182.254.134.174:7001
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: 1194702cf2b7ef4f5d5e7854f526aa60d47f2577 182.254.134.174:7002
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: 2aedbda7dadc48e545ec0870b7d3585bb4637c47 182.254.134.174:7006
slots: (0 slots) slave
replicates e9750b2aa5a23b0d5e79e98b3a61c6f97a69f01d
M: b120e1a401f11a06f8d81f19df2607c71c5cf9df 182.254.134.174:7003
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 28362fb68d33eb50322f514daf4e7a447e3eccb3 182.254.134.174:7004
slots: (0 slots) slave
replicates 1194702cf2b7ef4f5d5e7854f526aa60d47f2577
S: 52b90d0718c06990de002dbe00be512c0f4eccd1 182.254.134.174:7005
slots: (0 slots) slave
replicates b120e1a401f11a06f8d81f19df2607c71c5cf9df
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@VM-0-12-centos redis-cluster]#
模拟主机宕机情况，看看是否自动 slave自动升为master
[root@VM-0-12-centos redis-cluster]# ./redis01/redis-cli -p 7001 -a 1234
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:7001> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:886
cluster_stats_messages_pong_sent:887
cluster_stats_messages_sent:1773
cluster_stats_messages_ping_received:882
cluster_stats_messages_pong_received:886
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:1773
total_cluster_links_buffer_limit_exceeded:0
127.0.0.1:7001> cluster nodes
78b3bb404c37850c1cb7409b6d1fd1d994ee2582 182.254.134.174:7003@17003 master - 0 1660036015000 3 connected 10923-16383
0238999b8de448562d28eb23f7be491f0603afe5 182.254.134.174:7005@17005 slave 78b3bb404c37850c1cb7409b6d1fd1d994ee2582 0 1660036017989 3 connected
cc079b10ae2ff6e0870250beb91be5f0a8d5fda2 182.254.134.174:7002@17002 master - 0 1660036016986 2 connected 5461-10922
232fb95877a5f2b3093b1f7d94b632c10f55c976 172.17.0.12:7001@17001 myself,master - 0 1660036015000 1 connected 0-5460
532e2e3322dc71f281349d18d562a2a00d201b5c 182.254.134.174:7006@17006 slave 232fb95877a5f2b3093b1f7d94b632c10f55c976 0 1660036016000 1 connected
2e939c53fbc695e136bd0e63a07d6f86f9e25cb3 182.254.134.174:7004@17004 slave cc079b10ae2ff6e0870250beb91be5f0a8d5fda2 0 1660036015982 2 connected
重新开一个ssh远程连接,连接7002服务，关闭redis，等待15秒
[root@VM-0-12-centos redis-cluster]# ./redis02/redis-cli -p 7002 -a 1234
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:7002> shutdown
再次查看节点信息，会发现7004已经已经自动升为master
127.0.0.1:7001> cluster nodes
78b3bb404c37850c1cb7409b6d1fd1d994ee2582 182.254.134.174:7003@17003 master - 0 1660036206861 3 connected 10923-16383
0238999b8de448562d28eb23f7be491f0603afe5 182.254.134.174:7005@17005 slave 78b3bb404c37850c1cb7409b6d1fd1d994ee2582 0 1660036204840 3 connected
cc079b10ae2ff6e0870250beb91be5f0a8d5fda2 182.254.134.174:7002@17002 master,fail - 1660036178557 1660036172498 2 disconnected
232fb95877a5f2b3093b1f7d94b632c10f55c976 172.17.0.12:7001@17001 myself,master - 0 1660036203000 1 connected 0-5460
532e2e3322dc71f281349d18d562a2a00d201b5c 182.254.134.174:7006@17006 slave 232fb95877a5f2b3093b1f7d94b632c10f55c976 0 1660036207000 1 connected
2e939c53fbc695e136bd0e63a07d6f86f9e25cb3 182.254.134.174:7004@17004 master - 0 1660036207872 7 connected 5461-10922
127.0.0.1:7001>
操作中遇到的问题：
1、[ERR] Node 182.254.134.174:7001 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.
[root@VM-0-12-centos redis-cluster]# /usr/local/redis/bin/redis-cli --cluster create --cluster-replicas 1 182.254.134.174:7001 182.254.134.174:7002 182.254.134.174:7003 182.254.134.174:7004 182.254.134.174:7005 182.254.134.174:7006 -a 1234
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
[ERR] Node 182.254.134.174:7001 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.
原因：上面的命令只能在新创健集群的时候执行一次，目的是为了建立内部各个节点的对应关系，比如主从关系，这些关系仅且只能在一个集群中初始化时对应一次；
解决办法：删除aof，rdb，nodes.conf文件重新启动，我写了一个清除文件脚本 clear.sh
cd ..
cd redis02
rm -f nodes-7002.conf
rm -f nodes.conf
rm -f dump.rdb
rm -f appendonly.aof
cd ..
cd redis03
rm -f nodes-7003.conf
rm -f nodes.conf
rm -f dump.rdb
rm -f appendonly.aof
cd ..
cd redis04
rm -f nodes-7004.conf
rm -f nodes.conf
rm -f dump.rdb
rm -f appendonly.aof
cd ..
cd redis05
rm -f nodes-7005.conf
rm -f nodes.conf
rm -f dump.rdb
rm -f appendonly.aof
cd ..
cd redis06
rm -f nodes-7006.conf
rm -f nodes.conf
rm -f dump.rdb
rm -f appendonly.aof
cd ..
[root@VM-0-12-centos redis-cluster]# ./clear.sh
[root@VM-0-12-centos redis-cluster]# ./start-all.sh
/usr/local/redis/bin/redis-cli --cluster create --cluster-replicas 1 182.254.134.174:7001 182.254.134.174:7002 182.254.134.174:7003 182.254.134.174:7004 182.254.134.174:7005 182.254.134.174:7006 -a 1234
正常启动，重复代码就不粘贴了...
- 0@0 master,fail,noaddr - 1660036178557 1660036172498 2 disconnected
7004自动升为master后重新启动7002查看节点状态后发现，发现7002已脱离集群，且id都已发生了变化
127.0.0.1:7001> cluster nodes
78b3bb404c37850c1cb7409b6d1fd1d994ee2582 182.254.134.174:7003@17003 master - 0 1660039460000 3 connected 10923-16383
0238999b8de448562d28eb23f7be491f0603afe5 182.254.134.174:7005@17005 slave 78b3bb404c37850c1cb7409b6d1fd1d994ee2582 0 1660039461528 3 connected
cc079b10ae2ff6e0870250beb91be5f0a8d5fda2 :0@0 master,fail,noaddr - 1660036178557 1660036172498 2 disconnected
232fb95877a5f2b3093b1f7d94b632c10f55c976 172.17.0.12:7001@17001 myself,master - 0 1660039460000 1 connected 0-5460
532e2e3322dc71f281349d18d562a2a00d201b5c 182.254.134.174:7006@17006 slave 232fb95877a5f2b3093b1f7d94b632c10f55c976 0 1660039460000 1 connected
2e939c53fbc695e136bd0e63a07d6f86f9e25cb3 182.254.134.174:7004@17004 master - 0 1660039460525 7 connected 5461-10922
原因：7002已脱离集群；
解决办法：将该从节点剔出集群，重新加入
任意节点上执行 cluster forget命令
127.0.0.1:7001> cluster forget cc079b10ae2ff6e0870250beb91be5f0a8d5fda2 # 踢出集群
OK
127.0.0.1:7001> cluster meet 182.254.134.174 7002 # 握手加入集群
OK
127.0.0.1:7001> cluster nodes
37d2363fbc60a64a3cb5be0cb75c4b418795b02e 182.254.134.174:7002@17002 master - 0 1660040192000 0 connected
78b3bb404c37850c1cb7409b6d1fd1d994ee2582 182.254.134.174:7003@17003 master - 0 1660040195000 3 connected 10923-16383
0238999b8de448562d28eb23f7be491f0603afe5 182.254.134.174:7005@17005 slave 78b3bb404c37850c1cb7409b6d1fd1d994ee2582 0 1660040193966 3 connected
232fb95877a5f2b3093b1f7d94b632c10f55c976 172.17.0.12:7001@17001 myself,master - 0 1660040195000 1 connected 0-5460
532e2e3322dc71f281349d18d562a2a00d201b5c 182.254.134.174:7006@17006 slave 232fb95877a5f2b3093b1f7d94b632c10f55c976 0 1660040196975 1 connected
2e939c53fbc695e136bd0e63a07d6f86f9e25cb3 182.254.134.174:7004@17004 master - 0 1660040195972 7 connected 5461-10922
然再在7002上和7004重新配置主从关系,
127.0.0.1:7002> cluster replicate 2e939c53fbc695e136bd0e63a07d6f86f9e25cb3
127.0.0.1:7002> cluster nodes
232fb95877a5f2b3093b1f7d94b632c10f55c976 182.254.134.174:7001@17001 master - 0 1660040830000 1 connected 0-5460
78b3bb404c37850c1cb7409b6d1fd1d994ee2582 182.254.134.174:7003@17003 master - 0 1660040828781 3 connected 10923-16383
37d2363fbc60a64a3cb5be0cb75c4b418795b02e 172.17.0.12:7002@17002 myself,slave 2e939c53fbc695e136bd0e63a07d6f86f9e25cb3 0 1660040827000 7 connected
0238999b8de448562d28eb23f7be491f0603afe5 182.254.134.174:7005@17005 slave 78b3bb404c37850c1cb7409b6d1fd1d994ee2582 0 1660040829000 3 connected
2e939c53fbc695e136bd0e63a07d6f86f9e25cb3 182.254.134.174:7004@17004 master - 0 1660040829785 7 connected 5461-10922
532e2e3322dc71f281349d18d562a2a00d201b5c 182.254.134.174:7006@17006 slave 232fb95877a5f2b3093b1f7d94b632c10f55c976 0 1660040830789 1 connected
prometheus granafa 监控
下载
https://github.com/oliver006/redis_exporter/releases
配置添加
prometheus.yml
在prometheus.yml添加两个job，redis_exporter和redis_exporter_targets，redis_exporter_targets才是真正监控redis集群的。
## config for scraping the exporter itself
- job_name: 'redis_exporter'
static_configs:
- targets:
- prod-xxx-xxxx-redis01:9121
## config for the multiple Redis targets that the exporter will scrape
- job_name: 'redis_exporter_targets'
static_configs:
- targets:
- redis://172.16.0.122:7000
- redis://172.16.0.122:7001
- redis://172.16.0.120:7002
- redis://172.16.0.120:7003
- redis://172.16.0.129:7004
- redis://172.16.0.129:7005
metrics_path: /scrape
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: prod-xxxx-xxxx-redis01:9121
部署redis_exporter
REDIS_ADDR为你redis集群的其中一个节点即可，默认redis_exporter 使用9121端口！！
redis_exporter -redis.addr $ip:6379 -redis.password $password -web.listen-address ":9333" > /dev/null 2>&1 &
常用选项：
-redis.addr： 连接redis地址:端口
-redis.password： 连接redis的密码
-web.listen-address： 指定redis_exporter的服务端口，格式为-web.listen-address ":9121
然后重启prometheus。
grafana模板
id 763
文章地址: https://www.cnblogs.com/weifeng1463/p/16617374.html
https://blog.csdn.net/weixin_43844718/article/details/126250467