Redis Cluster 线上扩容采坑记

前言：很多文章及自带管理工具都是免密码扩容，但是线上环境怎么能少了密码呢。以下为针对带密码集群扩容的探索：

概念

去中心、去中间件，各节点平等，保存各自数据和集群状态，节点间活跃互连。

传统用一致性哈希分配数据，集群用哈希槽（hash slot）分配。 算法为CRC16。

默认分配16384个slot， 用CRC16算法取模{ CRC16(key)%16384 }计算所属slot。

最少3个主节点

migrating 与 importing

默认slots分配

image.png

部署

ruby:

yum:

/opt/rh/${RUBY_VERSION}
}/root/usr/local/share/gems/gems/${GEM_REDIS_VERSION}/lib/redis/client.rb
修改 password => nil 为 password => "abc123"

编译：

创建

redis-trib.rb create --replicas 1 172.20.133.39:7701 172.20.133.39:7702 172.20.133.39:7703 172.20.133.39:7704 172.20.133.39:7705 172.20.133.39:7706

[root@asiskskek ~]# ./redis-trib.rb create --password abc123 --replicas 1 172.20.133.39:7701 172.20.133.39:7702 172.20.133.39:7703 172.20.133.39:7704 172.20.133.39:7705 172.20.133.39:7706
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
172.20.133.39:7701
172.20.133.39:7702
172.20.133.39:7703
Adding replica 172.20.133.39:7704 to 172.20.133.39:7701
Adding replica 172.20.133.39:7705 to 172.20.133.39:7702
Adding replica 172.20.133.39:7706 to 172.20.133.39:7703
M: c76f037234f873f69b3dff981b82e37b7e98b7b2 172.20.133.39:7701
   slots:0-5460 (5461 slots) master
M: d2f155f3ea1506c9ac26c39b78925cd31278da67 172.20.133.39:7702
   slots:5461-10922 (5462 slots) master
M: 3af53debec41cf4ddace3f568538e0e5062d11a2 172.20.133.39:7703
   slots:10923-16383 (5461 slots) master
S: 12659ccc59a3b4e28a9cbbc978f19a79fdee0fdd 172.20.133.39:7704
   replicates c76f037234f873f69b3dff981b82e37b7e98b7b2
S: 87a3f5546ed9d8000e725edc7ebd879e219d351b 172.20.133.39:7705
   replicates d2f155f3ea1506c9ac26c39b78925cd31278da67
S: c61a72b37f25542f5eaa257211a31b512d763a03 172.20.133.39:7706
   replicates 3af53debec41cf4ddace3f568538e0e5062d11a2
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join..
>>> Performing Cluster Check (using node 172.20.133.39:7701)
M: c76f037234f873f69b3dff981b82e37b7e98b7b2 172.20.133.39:7701
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 3af53debec41cf4ddace3f568538e0e5062d11a2 172.20.133.39:7703
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 87a3f5546ed9d8000e725edc7ebd879e219d351b 172.20.133.39:7705
   slots: (0 slots) slave
   replicates d2f155f3ea1506c9ac26c39b78925cd31278da67
S: 12659ccc59a3b4e28a9cbbc978f19a79fdee0fdd 172.20.133.39:7704
   slots: (0 slots) slave
   replicates c76f037234f873f69b3dff981b82e37b7e98b7b2
S: c61a72b37f25542f5eaa257211a31b512d763a03 172.20.133.39:7706
   slots: (0 slots) slave
   replicates 3af53debec41cf4ddace3f568538e0e5062d11a2
M: d2f155f3ea1506c9ac26c39b78925cd31278da67 172.20.133.39:7702
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@asiskskek ~]#

维护

1. 配置优化

timeout = 3000
cluster-node-timeout 3000

慢日志记录

2. 分析统计

集群管理工具

官方自带 redis-trib.rb
半支持认证版：

下载地址： https://github.com/otherpirate/redis/blob/382867aaafeffcdd542688df49e8a1518e9738b5/src/redis-trib.rb

pr地址：
https://github.com/antirez/redis/pull/4288

redis-trib.rb command --password thepw xxxx

分析dump文件及内存使用量

https://github.com/sripathikrishnan/redis-rdb-tools

性能分析工具

https://github.com/facebookarchive/redis-faina.git

redis-cli -c -h 172.20.133.39 -p 7701 -a abc123 MONITOR | head -10000 | ./redis-faina.py --redis-version=3.0 >faina-date +"%Y%m%d_%H%M%S".txt

时间分析能力有限：由于是通过请求开始时间和下个请求开始时间间隔计算请求执行时间，并不精确，特别是请求量很少的时候

3. 命令防身：

cluster 命令

CLUSTER INFO   # 打印集群的信息
CLUSTER NODES   # 列出集群当前已知的所有节点(node),以及这些节点的相关信息。

节点

CLUSTER MEET     # 将 ip 和 port 所指定的节点添加到集群当中，让它成为集群的一份子。
CLUSTER FORGET    # 从集群中移除 node_id 指定的节点。
CLUSTER REPLICATE    # 将当前节点设置为 node_id 指定的节点的从节点。
CLUSTER SAVECONFIG   # 将节点的配置文件保存到硬盘里面。
CLUSTER ADDSLOTS  [slot ...]   # 将一个或多个槽（slot）指派（assign）给当前节点。
CLUSTER DELSLOTS  [slot ...]   # 移除一个或多个槽对当前节点的指派。
CLUSTER FLUSHSLOTS   # 移除指派给当前节点的所有槽，让当前节点变成一个没有指派任何槽的节点。
CLUSTER SETSLOT  NODE    # 将槽 slot 指派给 node_id 指定的节点。
CLUSTER SETSLOT  MIGRATING    # 将本节点的槽 slot 迁移到 node_id 指定的节点中。
CLUSTER SETSLOT  IMPORTING    # 从 node_id 指定的节点中导入槽 slot 到本节点。
CLUSTER SETSLOT  STABLE   # 取消对槽 slot 的导入（import）或者迁移（migrate）。

键

CLUSTER KEYSLOT    # 计算键 key 应该被放置在哪个槽上。
CLUSTER COUNTKEYSINSLOT    # 返回槽 slot 目前包含的键值对数量。
CLUSTER GETKEYSINSLOT     # 返回 count 个 slot 槽中的键。

新增

CLUSTER SLAVES node-id   # 返回一个master节点的slaves 列表

查看集群节点情况

redis-cli -c -h 172.20.133.39 -p 7701 -a abc123 cluster nodes

添加master节点：

redis-trib.rb add-node 172.20.133.39:7706(新节点) 172.20.133.39:7701(已有节点)

添加slave节点

redis-trib.rb add-node --slave --master-id xxxxxx  172.20.133.37:7707(添加slave) 172.20.133.37:7701(已有节点)

删除节点

redis-trib.rb del-node 172.20.133.37:7701(已有节点)  172.20.133.37:7707(删除节点)

迁移：

redis-trib.rb reshard 172.20.133.39:7701
4096
id
all

删除槽

redis-cli -c -h 172.20.133.39 -p 7701 -a abc123 cluster delslots 5461

在线扩容

核心概念：

扩容的核心就是迁移slot并同步迁移key-value，不管是用工具还是集群命令，核心都是以下几步:

> 在迁移目的节点执行cluster setslot  IMPORTING 命令，指明需要迁移的slot和迁移源节点。
> 在迁移源节点执行cluster setslot  MIGRATING 命令，指明需要迁移的slot和迁移目的节点。
> 在迁移源节点执行cluster getkeysinslot获取该slot的key列表。
> 在迁移源节点执行对每个key执行migrate命令，该命令会同步把该key迁移到目的节点。
> 在迁移源节点反复执行cluster getkeysinslot命令，直到该slot的列表为空。
> 在迁移源节点和目的节点执行cluster setslot  NODE ，完成迁移操作。

前提:

迁移与redis版本及是否开启认证有关：
1. 未开启认证，任何方式都可以
2. 开启认证，只有4.0.7之后的版本支持migrate auth
3. 最新稳定版4.0.9的redis-trib.rb未支持migrate auth，坊间未找到支持版，这是个给redis贡献代码的机会
4. 有密码最简单的扩容办法就是迁移的时候去掉密码,前提是业务代码要支持，去掉密码能正常使用。想折腾就写脚本实现带密码用集群命令实现，见附4

https://github.com/antirez/redis/commit/47717222b64b9437b12d76f39ac694f7454e3c7c

本次commit之后开始支持 cluster migate auth，但是好像忘了更新redis-trib.rb，应该用不了多久就会支持

image

1. 环境：

Centos 6.8
Redis 4.0.9 ---- { redis version(<4.0.7) doesn't support migrate command in the cluster with auth mode }
Ruby 2.0.5
redis gem 3.3.3 ---- { 要小于4.0 版本，否则会报：[ERR] Calling MIGRATE: ERR Syntax error, try CLIENT (LIST | KILL ip:port | GETNAME | SETNAME connection-name) ，但是 v3.3.3 don't support redis cluter with password }

2. 扩容过程：

2.1. 添加节点

添加master节点：

将7707添加进集群

redis-trib.rb add-node 172.20.133.39:7707 172.20.133.39:7701

[root@asiskskek ~]# ./redis-trib.rb add-node --password abc123 172.20.133.39:7707 172.20.133.39:7701
>>> Adding node 172.20.133.39:7707 to cluster 172.20.133.39:7701
>>> Performing Cluster Check (using node 172.20.133.39:7701)
M: c76f037234f873f69b3dff981b82e37b7e98b7b2 172.20.133.39:7701
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 3af53debec41cf4ddace3f568538e0e5062d11a2 172.20.133.39:7703
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 87a3f5546ed9d8000e725edc7ebd879e219d351b 172.20.133.39:7705
   slots: (0 slots) slave
   replicates d2f155f3ea1506c9ac26c39b78925cd31278da67
S: 12659ccc59a3b4e28a9cbbc978f19a79fdee0fdd 172.20.133.39:7704
   slots: (0 slots) slave
   replicates c76f037234f873f69b3dff981b82e37b7e98b7b2
S: c61a72b37f25542f5eaa257211a31b512d763a03 172.20.133.39:7706
   slots: (0 slots) slave
   replicates 3af53debec41cf4ddace3f568538e0e5062d11a2
M: d2f155f3ea1506c9ac26c39b78925cd31278da67 172.20.133.39:7702
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 172.20.133.39:7707 to make it join the cluster.
[OK] New node added correctly.
[root@asiskskek ~]#

检查集群状态

[root@asiskskek ~]# ./redis-trib.rb check --password abc123 172.20.133.39:7701    
>>> Performing Cluster Check (using node 172.20.133.39:7701)
M: c76f037234f873f69b3dff981b82e37b7e98b7b2 172.20.133.39:7701
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 3af53debec41cf4ddace3f568538e0e5062d11a2 172.20.133.39:7703
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 87a3f5546ed9d8000e725edc7ebd879e219d351b 172.20.133.39:7705
   slots: (0 slots) slave
   replicates d2f155f3ea1506c9ac26c39b78925cd31278da67
M: c713e819cf41b1c79faab18c93398c510dfc314d 172.20.133.39:7707
   slots: (0 slots) master
   0 additional replica(s)
S: 12659ccc59a3b4e28a9cbbc978f19a79fdee0fdd 172.20.133.39:7704
   slots: (0 slots) slave
   replicates c76f037234f873f69b3dff981b82e37b7e98b7b2
S: c61a72b37f25542f5eaa257211a31b512d763a03 172.20.133.39:7706
   slots: (0 slots) slave
   replicates 3af53debec41cf4ddace3f568538e0e5062d11a2
M: d2f155f3ea1506c9ac26c39b78925cd31278da67 172.20.133.39:7702
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@asiskskek ~]#

添加slave节点

将7707添加进集群作为7706的slave节点

redis-trib.rb add-node --slave --master-id 7707's-id  172.20.133.39:7708 172.20.133.39:7701

[root@asiskskek ~]# ./redis-trib.rb add-node --password abc123 --slave --master-id c713e819cf41b1c79faab18c93398c510dfc314d  172.20.133.39:7708 172.20.133.39:7701
>>> Adding node 172.20.133.39:7708 to cluster 172.20.133.39:7701
>>> Performing Cluster Check (using node 172.20.133.39:7701)
M: c76f037234f873f69b3dff981b82e37b7e98b7b2 172.20.133.39:7701
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 3af53debec41cf4ddace3f568538e0e5062d11a2 172.20.133.39:7703
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 87a3f5546ed9d8000e725edc7ebd879e219d351b 172.20.133.39:7705
   slots: (0 slots) slave
   replicates d2f155f3ea1506c9ac26c39b78925cd31278da67
M: c713e819cf41b1c79faab18c93398c510dfc314d 172.20.133.39:7707
   slots: (0 slots) master
   0 additional replica(s)
S: 12659ccc59a3b4e28a9cbbc978f19a79fdee0fdd 172.20.133.39:7704
   slots: (0 slots) slave
   replicates c76f037234f873f69b3dff981b82e37b7e98b7b2
S: c61a72b37f25542f5eaa257211a31b512d763a03 172.20.133.39:7706
   slots: (0 slots) slave
   replicates 3af53debec41cf4ddace3f568538e0e5062d11a2
M: d2f155f3ea1506c9ac26c39b78925cd31278da67 172.20.133.39:7702
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 172.20.133.39:7708 to make it join the cluster.
Waiting for the cluster to join.
>>> Configure node as replica of 172.20.133.39:7707.
[OK] New node added correctly.
[root@asiskskek ~]#

2.2. 迁移槽位

如果有密码认证，先批量去掉密码，附2

redis-trib.rb reshard 172.20.133.39:7701
4096 (分配给新master的slots数)
masterid
all

[root@asiskskek ~]# ./redis-trib.rb reshard --password abc123 172.20.133.39:7701 
>>> Performing Cluster Check (using node 172.20.133.39:7701)
M: c76f037234f873f69b3dff981b82e37b7e98b7b2 172.20.133.39:7701
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 3af53debec41cf4ddace3f568538e0e5062d11a2 172.20.133.39:7703
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 87a3f5546ed9d8000e725edc7ebd879e219d351b 172.20.133.39:7705
   slots: (0 slots) slave
   replicates d2f155f3ea1506c9ac26c39b78925cd31278da67
M: c713e819cf41b1c79faab18c93398c510dfc314d 172.20.133.39:7707
   slots: (0 slots) master
   1 additional replica(s)
S: 12659ccc59a3b4e28a9cbbc978f19a79fdee0fdd 172.20.133.39:7704
   slots: (0 slots) slave
   replicates c76f037234f873f69b3dff981b82e37b7e98b7b2
S: d0fb6ebab8ea6d795917f5b0a385ff2736e7c9b9 172.20.133.39:7708
   slots: (0 slots) slave
   replicates c713e819cf41b1c79faab18c93398c510dfc314d
S: c61a72b37f25542f5eaa257211a31b512d763a03 172.20.133.39:7706
   slots: (0 slots) slave
   replicates 3af53debec41cf4ddace3f568538e0e5062d11a2
M: d2f155f3ea1506c9ac26c39b78925cd31278da67 172.20.133.39:7702
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 4096  <=====
What is the receiving node ID? c713e819cf41b1c79faab18c93398c510dfc314d  
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1:all      <=====

Ready to move 4096 slots.
  Source nodes:
    M: c76f037234f873f69b3dff981b82e37b7e98b7b2 172.20.133.39:7701
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
    M: 3af53debec41cf4ddace3f568538e0e5062d11a2 172.20.133.39:7703
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
    M: d2f155f3ea1506c9ac26c39b78925cd31278da67 172.20.133.39:7702
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
  Destination node:
    M: c713e819cf41b1c79faab18c93398c510dfc314d 172.20.133.39:7707
   slots: (0 slots) master
   1 additional replica(s)
  Resharding plan:
    Moving slot 5461 from d2f155f3ea1506c9ac26c39b78925cd31278da67
    Moving slot 5462 from d2f155f3ea1506c9ac26c39b78925cd31278da67
    ...
    Moving slot 6825 from d2f155f3ea1506c9ac26c39b78925cd31278da67
    Moving slot 6826 from d2f155f3ea1506c9ac26c39b78925cd31278da67
    Moving slot 0 from c76f037234f873f69b3dff981b82e37b7e98b7b2
    Moving slot 1 from c76f037234f873f69b3dff981b82e37b7e98b7b2
    ...
    Moving slot 1363 from c76f037234f873f69b3dff981b82e37b7e98b7b2
    Moving slot 1364 from c76f037234f873f69b3dff981b82e37b7e98b7b2
    Moving slot 10923 from 3af53debec41cf4ddace3f568538e0e5062d11a2
    Moving slot 10924 from 3af53debec41cf4ddace3f568538e0e5062d11a2
    ...
    Moving slot 12286 from 3af53debec41cf4ddace3f568538e0e5062d11a2
    Moving slot 12287 from 3af53debec41cf4ddace3f568538e0e5062d11a2
Do you want to proceed with the proposed reshard plan (yes/no)? yes
Moving slot 5461 from 172.20.133.39:7702 to 172.20.133.39:7707: 
Moving slot 5462 from 172.20.133.39:7702 to 172.20.133.39:7707: 
...
Moving slot 6825 from 172.20.133.39:7702 to 172.20.133.39:7707: 
Moving slot 6826 from 172.20.133.39:7702 to 172.20.133.39:7707: 
Moving slot 0 from 172.20.133.39:7701 to 172.20.133.39:7707: 
Moving slot 1 from 172.20.133.39:7701 to 172.20.133.39:7707: 
...
Moving slot 1363 from 172.20.133.39:7701 to 172.20.133.39:7707: 
Moving slot 1364 from 172.20.133.39:7701 to 172.20.133.39:7707: 
Moving slot 10923 from 172.20.133.39:7703 to 172.20.133.39:7707: 
Moving slot 10924 from 172.20.133.39:7703 to 172.20.133.39:7707: 
...
Moving slot 12286 from 172.20.133.39:7703 to 172.20.133.39:7707: 
Moving slot 12287 from 172.20.133.39:7703 to 172.20.133.39:7707: 
[root@asiskskek ~]#

检查集群状态

[root@asiskskek ~]# ./redis-trib.rb check --password abc123 172.20.133.39:7701        
>>> Performing Cluster Check (using node 172.20.133.39:7701)
M: c76f037234f873f69b3dff981b82e37b7e98b7b2 172.20.133.39:7701
   slots:1365-5460 (4096 slots) master
   1 additional replica(s)
M: 3af53debec41cf4ddace3f568538e0e5062d11a2 172.20.133.39:7703
   slots:12288-16383 (4096 slots) master
   1 additional replica(s)
S: 87a3f5546ed9d8000e725edc7ebd879e219d351b 172.20.133.39:7705
   slots: (0 slots) slave
   replicates d2f155f3ea1506c9ac26c39b78925cd31278da67
M: c713e819cf41b1c79faab18c93398c510dfc314d 172.20.133.39:7707
   slots:0-1364,5461-6826,10923-12287 (4096 slots) master
   1 additional replica(s)
S: 12659ccc59a3b4e28a9cbbc978f19a79fdee0fdd 172.20.133.39:7704
   slots: (0 slots) slave
   replicates c76f037234f873f69b3dff981b82e37b7e98b7b2
S: d0fb6ebab8ea6d795917f5b0a385ff2736e7c9b9 172.20.133.39:7708
   slots: (0 slots) slave
   replicates c713e819cf41b1c79faab18c93398c510dfc314d
S: c61a72b37f25542f5eaa257211a31b512d763a03 172.20.133.39:7706
   slots: (0 slots) slave
   replicates 3af53debec41cf4ddace3f568538e0e5062d11a2
M: d2f155f3ea1506c9ac26c39b78925cd31278da67 172.20.133.39:7702
   slots:6827-10922 (4096 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@asiskskek ~]#

异常处理：

1. gem install redis

ERROR: Loading command: install (LoadError)
cannot load such file -- zlib
ERROR: While executing gem ... (NoMethodError)
undefined method `invoke_with_build_args' for nil:NilClass

解决：

编译安装的ruby缺少库文件，从ruby源码编译安装：
ruby目录：
cd ext/zlib
修改Makefile：
$(top_srcdir)/include/ruby.h  -->  zlib.o: ../../include/ruby.h
ruby extconf.rb
make && make install
如果有openssl 报错，处理方法类似：
Makefile 添加 top_srcdir = ../..

2. redis-trib.rb xxx

[WARNING] Node 172.20.133.39:7701 has slots in migrating state (0).
[WARNING] Node 172.20.133.39:7709 has slots in importing state (0).
[WARNING] The following slots are open: 0

解决：

redis-cli -h 172.20.133.39 -p 7701 -a abc123 cluster setslot 0 stable
redis-cli -h 172.20.133.39 -p 7709 -a abc123 cluster setslot 0 stable

[ERR] Calling MIGRATE: ERR Syntax error, try CLIENT (LIST | KILL ip:port | GETNAME | SETNAME connection-name)

解决:

需小于4.0的版本

gem uninstall redis
gem install redis -v 3.3.3

3. redis-trib reshard ip:port

认证失败

Do you want to proceed with the proposed reshard plan (yes/no)? yes
Moving slot 1365 from 172.20.133.39:7701 to 172.20.133.39:7705:
[ERR] ERR Target instance replied with error: NOAUTH Authentication required.

解决：

目前只能去掉密码

redis-cli -c -h 172.20.133.39 -p $i -a abc123 config set masterauth ""
redis-cli -c -h 172.20.133.39 -p $i -a abc123 config set requirepass ""
见附2

4. redis-trib reshard --password thepw ip:port

语法错误

Do you want to proceed with the proposed reshard plan (yes/no)? yes
Moving slot 1365 from 172.20.133.39:7701 to 172.20.133.39:7705:
[ERR] Calling MIGRATE: ERR syntax error

附：

脚本：

1. 批量向redis插入数据：

1. 单shell脚本版：

#!/bin/bash
for ((i=0;i<1000000;i++))
do
echo -en "hello" | redis-cli -c -h 172.20.133.39 -p 7701 -a coocaa set name$i $i >> keyset.log
done

1. pipeline方式，高效版：

#!/usr/bin/python
for i in range(1000000):
    print 'set name'+str(i),'helloworld'

python 1.py > redis_commands.txt

#!/bin/bash

while read CMD; do
  # each command begins with *{number arguments in command}\r\n
  XS=($CMD); printf "*${#XS[@]}\r\n"
  # for each argument, we append ${length}\r\n{argument}\r\n
  for X in $CMD; do printf "\$${#X}\r\n$X\r\n"; done
done < redis_commands.txt

bash 2.sh > redis_data.txt

cat redis_data.txt | redis-cli -c -h 172.20.133.39 -p 7701 -a abc123 --pipe

集群需要连每个master执行一遍。

2. 批量密码操作

#!/bin/bash
for ((i=7701;i<7713;i++))
do
# 增加认证
redis-cli -c -h 172.20.133.39 -p $i -a abc123 config set masterauth abc123
redis-cli -c -h 172.20.133.39 -p $i -a abc123 config set requirepass abc123
# 取消认证
#redis-cli -c -h 172.20.133.39 -p $i -a abc123 config set masterauth ""
#redis-cli -c -h 172.20.133.39 -p $i -a abc123 config set requirepass ""
done

3.redis重启脚本

#!/bin/sh

#
# Simple Redis init.d script conceived to work on Linux systems
# as it does use of the /proc filesystem.

REDISPORT=7701
EXEC=/usr/local/bin/redis-server
CLIEXEC=/usr/local/bin/redis-cli

PIDFILE=/tmp/redis_${REDISPORT}.pid
CONF="/etc/redis/${REDISPORT}.conf"

case "$1" in
    start)
        if [ -f $PIDFILE ]
        then
                echo "$PIDFILE exists, process is already running or crashed"
        else
                echo "Starting Redis server..."
                $EXEC $CONF
        fi
        ;;
    stop)
        if [ ! -f $PIDFILE ]
        then
                echo "$PIDFILE does not exist, process is not running"
        else
                PID=$(cat $PIDFILE)
                echo "Stopping ..."
                $CLIEXEC -p $REDISPORT -a abc123 -h 172.20.133.39 shutdown
                while [ -x /proc/${PID} ]
                do
                    echo "Waiting for Redis to shutdown ..."
                    sleep 1
                done
                echo "Redis stopped"
        fi
        ;;
    *)
        echo "Please use start or stop as first argument"
        ;;
esac

4.redis迁移脚本

#!/bin/bash
source_ip=$1
source_port=$2
target_ip=$3
target_port=$4
startSlot=$5
endSlot=$6
password=$7

#目标节点执行importing命令，目标节点将从源节点迁移slot

for slot in `seq ${startSlot} ${endSlot}`  
do  
    redis-cli -c -h ${target_ip} -p ${target_port} -a ${password} cluster setslot ${slot} IMPORTING `redis-cli -c -h ${source_ip} -p ${source_port} -a ${password} cluster nodes | grep ${source_ip} | grep ${source_port} | awk '{print $1}'`  
done

#源节点执行migrating命令，源节点将向目标节点迁移slot

for slot in `seq ${startSlot} ${endSlot}`  
do  
    redis-cli -c -h ${source_ip} -p ${source_port} -a ${password} cluster setslot ${slot} MIGRATING `redis-cli -c -h ${source_ip} -p ${source_port} -a ${password} cluster nodes | grep ${target_ip} | grep ${target_port} | awk '{print $1}'`  
done


for slot in `seq ${startSlot} ${endSlot}`  
do  
    while [ 1 -eq 1 ]  
    do  

#源节点执行getkeysinslot命令，从slot中取出count个键值对的key

        allkeys=`redis-cli -c -h ${source_ip} -p ${source_port} -a ${password} cluster getkeysinslot ${slot} 20`  
#        if [ !-z ${allkeys} ]  
        if [ -z "${allkeys}" ]
        then  

#源节点和目标节点执行setslot命令，将slot分配给目标节点

            redis-cli -c -h ${source_ip} -p ${source_port} -a ${password} cluster setslot ${slot} NODE `redis-cli -c -h ${source_ip} -p ${source_port} -a ${password} cluster nodes | grep ${target_ip} | grep ${target_port} | awk '{print $1}'`  
            redis-cli -c -h ${target_ip} -p ${target_port} -a ${password} cluster setslot ${slot} NODE `redis-cli -c -h ${source_ip} -p ${source_port} -a ${password} cluster nodes | grep ${target_ip} | grep ${target_port} | awk '{print $1}'`  
            break  
        else  
            for key in ${allkeys}  
            do  
                echo "slot ${slot} key ${key}"  

#源节点执行migrate命令，将key迁移到目标节点

                redis-cli -c -h ${source_ip} -p ${source_port} -a ${password} MIGRATE ${target_ip} ${target_port} ${key} 0 5000 AUTH ${password}
            done  
        fi
    done
done