Redis5集群搭建
前言
随着微服务架构的不断通用化，作为微服务缓存的redis也必须进行响应的分布式部署，共享缓存。
环境
- vmwarestation 14.*
- centos7.0
- redis5.04
过程
-
下载
wget http://download.redis.io/releases/redis-2.8.17.tar.gz 或者 官网下载 -
编译、安装（src）目录
make && make install常见错误：
如果报错提示缺少gcc，则安装gcc ： yum install -y gcc 如果报错显示：jemalloc/jemalloc.h: No such file or directory，则在make时添加参数 make MALLOC=libc 之后再make install -
创建配置文件
创建cluster-test文件夹，并创建几个节点文件夹（我创建了6个）[root@localhost soft]# cd cluster-test/ [root@localhost cluster-test]# ll total 0 drwxr-xr-x. 2 root root 69 Apr 24 19:43 7000 drwxr-xr-x. 2 root root 69 Apr 24 19:48 7001 drwxr-xr-x. 2 root root 69 Apr 24 19:48 7002 drwxr-xr-x. 2 root root 69 Apr 24 19:48 7003 drwxr-xr-x. 2 root root 69 Apr 24 19:48 7004 drwxr-xr-x. 2 root root 69 Apr 24 19:48 7005 [root@localhost cluster-test]#创建配置文件：redis.conf,并增加如下配置
[root@localhost 7000]# cat redis.conf bind 192.168.254.131#绑定ip port 7000#端口 dir /opt/soft/cluster-test/7000#文件地址 daemonize yes#后台运行 cluster-enabled yes#开启集群 cluster-config-file nodes-7000.conf#集群配置文件 cluster-node-timeout 15000#集群节点失效时长 appendonly yes#持久化方式 [root@localhost 7000]#拷贝配置文件到相应的文件夹，并做相应的修改。
-
启动节点redis
[root@localhost 7000]# redis-server /opt/soft/cluster-test/7000/redis.conf [root@localhost 7000]# redis-server /opt/soft/cluster-test/7001/redis.conf [root@localhost 7000]# redis-server /opt/soft/cluster-test/7002/redis.conf [root@localhost 7000]# redis-server /opt/soft/cluster-test/7003/redis.conf [root@localhost 7000]# redis-server /opt/soft/cluster-test/7004/redis.conf [root@localhost 7000]# redis-server /opt/soft/cluster-test/7005/redis.conf查看是否启动成功：ps -ef | grep redis-server
[root@localhost 7000]# ps -ef | grep redis-server root 2128 1 0 19:43 ? 00:00:00 redis-server 192.168.254.131:7000 [cluster] root 2141 1 0 19:48 ? 00:00:00 redis-server 192.168.254.131:7001 [cluster] root 2148 1 0 19:48 ? 00:00:00 redis-server 192.168.254.131:7002 [cluster] root 2153 1 0 19:48 ? 00:00:00 redis-server 192.168.254.131:7003 [cluster] root 2160 1 0 19:48 ? 00:00:00 redis-server 192.168.254.131:7004 [cluster] root 2165 1 0 19:48 ? 00:00:00 redis-server 192.168.254.131:7005 [cluster] root 2170 2020 0 19:48 pts/0 00:00:00 grep --color=auto redis-server -
创建集群
旧版本的集群创建方式：[root@localhost src]# ./redis-trib.rb create --replicas 1 192.168.254.131:7000 192.168.254.131:7001 192.168.254.131:7002 192.168.254.131:7003 192.168.254.131:7004 192.168.254.131:7005 WARNING: redis-trib.rb is not longer available! You should use redis-cli instead. All commands and features belonging to redis-trib.rb have been moved to redis-cli. In order to use them you should call redis-cli with the --cluster option followed by the subcommand name, arguments and options. Use the following syntax: redis-cli --cluster SUBCOMMAND [ARGUMENTS] [OPTIONS] Example: redis-cli --cluster create 192.168.254.131:7000 192.168.254.131:7001 192.168.254.131:7002 192.168.254.131:7003 192.168.254.131:7004 192.168.254.131:7005 --cluster-replicas 1 To get help about all subcommands, type: redis-cli --cluster help可以看到新版本下创建集群已经改为了使用 redis-cli --cluster的方式。
[root@localhost 7002]# redis-cli --cluster create 192.168.254.131:7000 192.168.254.131:7001 192.168.254.131:7002 192.168.254.131:7003 192.168.254.131:7004 192.168.254.131:7005 --cluster-replicas 1 >>> Performing hash slots allocation on 6 nodes... Master[0] -> Slots 0 - 5460 Master[1] -> Slots 5461 - 10922 Master[2] -> Slots 10923 - 16383 Adding replica 192.168.254.131:7004 to 192.168.254.131:7000 Adding replica 192.168.254.131:7005 to 192.168.254.131:7001 Adding replica 192.168.254.131:7003 to 192.168.254.131:7002 >>> Trying to optimize slaves allocation for anti-affinity [WARNING] Some slaves are in the same host as their master M: 7d68d4f7653b3358b9d2e10e42f5a105a5d67c46 192.168.254.131:7000 slots:[0-5460] (5461 slots) master M: cfe0ad38eb291229b250f27dddfeb0dd0c2581a3 192.168.254.131:7001 slots:[5461-10922] (5462 slots) master M: 9a0f7b1cf8283525b3ad1e099021ed0639dc81a6 192.168.254.131:7002 slots:[10923-16383] (5461 slots) master S: 51666c396b86df64f5d206ca1e63a47251bebf44 192.168.254.131:7003 replicates 7d68d4f7653b3358b9d2e10e42f5a105a5d67c46 S: 4b8be39dd071399da1749b47fbd8a2a539f795cc 192.168.254.131:7004 replicates cfe0ad38eb291229b250f27dddfeb0dd0c2581a3 S: 5f865575b54fc9ee89b63c412833f023c4776002 192.168.254.131:7005 replicates 9a0f7b1cf8283525b3ad1e099021ed0639dc81a6 Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join ..... >>> Performing Cluster Check (using node 192.168.254.131:7000) M: 7d68d4f7653b3358b9d2e10e42f5a105a5d67c46 192.168.254.131:7000 slots:[0-5460] (5461 slots) master 1 additional replica(s) S: 51666c396b86df64f5d206ca1e63a47251bebf44 192.168.254.131:7003 slots: (0 slots) slave replicates 7d68d4f7653b3358b9d2e10e42f5a105a5d67c46 M: cfe0ad38eb291229b250f27dddfeb0dd0c2581a3 192.168.254.131:7001 slots:[5461-10922] (5462 slots) master 1 additional replica(s) M: 9a0f7b1cf8283525b3ad1e099021ed0639dc81a6 192.168.254.131:7002 slots:[10923-16383] (5461 slots) master 1 additional replica(s) S: 5f865575b54fc9ee89b63c412833f023c4776002 192.168.254.131:7005 slots: (0 slots) slave replicates 9a0f7b1cf8283525b3ad1e099021ed0639dc81a6 S: 4b8be39dd071399da1749b47fbd8a2a539f795cc 192.168.254.131:7004 slots: (0 slots) slave replicates cfe0ad38eb291229b250f27dddfeb0dd0c2581a3 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. -
连接测试
[root@localhost cluster-test]# redis-cli -c -h 192.168.254.131 -p 7000 192.168.254.131:7000> info # Server redis_version:5.0.4 -----------------------版本 redis_git_sha1:00000000 ----------加密 redis_git_dirty:0 ------不知道 redis_build_id:b7228e50c523d596 --------build-id redis_mode:cluster ----------模式：集群 os:Linux 3.10.0-862.14.4.el7.x86_64 x86_64 -----------------系统内核 arch_bits:64 multiplexing_api:epoll atomicvar_api:atomic-builtin gcc_version:4.8.5 -----gcc版本 process_id:2128 ----进程id run_id:3cb0cdaca501e48afbe9433e680935d838597856 tcp_port:7000 ---端口 uptime_in_seconds:2575 ----运行时长 uptime_in_days:0 hz:10 configured_hz:10 lru_clock:12604937 executable:/opt/soft/cluster-test/redis-server ----------执行位置 config_file:/opt/soft/cluster-test/7000/redis.conf -----配置文件 # Clients -----------------客户端信息 connected_clients:1 -----连接的客户端 client_recent_max_input_buffer:2 client_recent_max_output_buffer:0 blocked_clients:0 # Memory -----内存信息 used_memory:1443312 used_memory_human:1.38M used_memory_rss:3678208 used_memory_rss_human:3.51M used_memory_peak:1443312 used_memory_peak_human:1.38M used_memory_peak_perc:100.00% used_memory_overhead:1441726 used_memory_startup:1392032 used_memory_dataset:1586 used_memory_dataset_perc:3.09% allocator_allocated:1409472 allocator_active:3640320 allocator_resident:3640320 total_system_memory:1910018048 total_system_memory_human:1.78G used_memory_lua:37888 used_memory_lua_human:37.00K used_memory_scripts:0 used_memory_scripts_human:0B number_of_cached_scripts:0 maxmemory:0 maxmemory_human:0B maxmemory_policy:noeviction allocator_frag_ratio:2.58 allocator_frag_bytes:2230848 allocator_rss_ratio:1.00 allocator_rss_bytes:0 rss_overhead_ratio:1.01 rss_overhead_bytes:37888 mem_fragmentation_ratio:2.61 mem_fragmentation_bytes:2268736 mem_not_counted_for_evict:0 mem_replication_backlog:0 mem_clients_slaves:0 mem_clients_normal:49694 mem_aof_buffer:0 mem_allocator:libc active_defrag_running:0 lazyfree_pending_objects:0 # Persistence -----持久化信息 loading:0 rdb_changes_since_last_save:0 rdb_bgsave_in_progress:0 rdb_last_save_time:1556106234 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:-1 rdb_current_bgsave_time_sec:-1 rdb_last_cow_size:0 aof_enabled:1 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_last_write_status:ok aof_last_cow_size:0 aof_current_size:0 aof_base_size:0 aof_pending_rewrite:0 aof_buffer_length:0 aof_rewrite_buffer_length:0 aof_pending_bio_fsync:0 aof_delayed_fsync:0 # Stats ---------状态 total_connections_received:2 total_commands_processed:5 instantaneous_ops_per_sec:0 total_net_input_bytes:114 total_net_output_bytes:18698 instantaneous_input_kbps:0.00 instantaneous_output_kbps:0.00 rejected_connections:0 sync_full:0 sync_partial_ok:0 sync_partial_err:0 expired_keys:0 expired_stale_perc:0.00 expired_time_cap_reached_count:0 evicted_keys:0 keyspace_hits:0 keyspace_misses:0 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:0 migrate_cached_sockets:0 slave_expires_tracked_keys:0 active_defrag_hits:0 active_defrag_misses:0 active_defrag_key_hits:0 active_defrag_key_misses:0 # Replication ----------------集群信息 role:master ------------------master connected_slaves:0 master_replid:fb37775be67657aaf26eeac00e736cefa0669f67 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0 # CPU used_cpu_sys:1.578260 used_cpu_user:1.303976 used_cpu_sys_children:0.000000 used_cpu_user_children:0.000000 # Cluster cluster_enabled:1 # Keyspace 192.168.254.131:7000> exit
查看集群状态cluster info 或者 cluster nodes
[root@localhost 7002]# redis-cli --cluster create 192.168.254.131:7000 192.168.254.131:7001 192.168.254.131:7002 192.168.254.131:7003 192.168.254.131:7004 192.168.254.131:7005 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.254.131:7004 to 192.168.254.131:7000
Adding replica 192.168.254.131:7005 to 192.168.254.131:7001
Adding replica 192.168.254.131:7003 to 192.168.254.131:7002
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 7d68d4f7653b3358b9d2e10e42f5a105a5d67c46 192.168.254.131:7000
slots:[0-5460] (5461 slots) master
M: cfe0ad38eb291229b250f27dddfeb0dd0c2581a3 192.168.254.131:7001
slots:[5461-10922] (5462 slots) master
M: 9a0f7b1cf8283525b3ad1e099021ed0639dc81a6 192.168.254.131:7002
slots:[10923-16383] (5461 slots) master
S: 51666c396b86df64f5d206ca1e63a47251bebf44 192.168.254.131:7003
replicates 7d68d4f7653b3358b9d2e10e42f5a105a5d67c46
S: 4b8be39dd071399da1749b47fbd8a2a539f795cc 192.168.254.131:7004
replicates cfe0ad38eb291229b250f27dddfeb0dd0c2581a3
S: 5f865575b54fc9ee89b63c412833f023c4776002 192.168.254.131:7005
replicates 9a0f7b1cf8283525b3ad1e099021ed0639dc81a6
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.....
>>> Performing Cluster Check (using node 192.168.254.131:7000)
M: 7d68d4f7653b3358b9d2e10e42f5a105a5d67c46 192.168.254.131:7000
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: 51666c396b86df64f5d206ca1e63a47251bebf44 192.168.254.131:7003
slots: (0 slots) slave
replicates 7d68d4f7653b3358b9d2e10e42f5a105a5d67c46
M: cfe0ad38eb291229b250f27dddfeb0dd0c2581a3 192.168.254.131:7001
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
M: 9a0f7b1cf8283525b3ad1e099021ed0639dc81a6 192.168.254.131:7002
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 5f865575b54fc9ee89b63c412833f023c4776002 192.168.254.131:7005
slots: (0 slots) slave
replicates 9a0f7b1cf8283525b3ad1e099021ed0639dc81a6
S: 4b8be39dd071399da1749b47fbd8a2a539f795cc 192.168.254.131:7004
slots: (0 slots) slave
replicates cfe0ad38eb291229b250f27dddfeb0dd0c2581a3
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
-
测试集群数据
xshell中复制一个ssh连接，一个链接到7000端口的redis，一个链接到7004的redis（目前7000为master，并且它的slave是7004）7000端口的redis
192.168.254.131:7000> set user jatham -> Redirected to slot [5474] located at 192.168.254.131:7001 OK 192.168.254.131:7001> get user "jatham" 192.168.254.131:7001>7004端口的redis
[root@localhost 7000]# redis-cli -c -h 192.168.254.131 -p 7004 192.168.254.131:7004> get user -> Redirected to slot [5474] located at 192.168.254.131:7001 ---这里它做了一个重定向到master槽的操作 "jatham" 192.168.254.131:7001>可以看到7000的slave中也查看到了7000中设置的值。那其他的master和slave节点呢？是否也能正常获取到master7000中设置的值？答案是肯定的。
[root@localhost 7000]# redis-cli -c -h 192.168.254.131 -p 7001 192.168.254.131:7001> get user "jatham" 192.168.254.131:7001>
还有一种情况，在slave节点设置值，其对应的master节点能否正常获取到值呢？测试下：7005slave节点下设置user的值
[root@localhost 7002]# redis-cli -c -h 192.168.254.131 -p 7005
192.168.254.131:7005> set user other-jatham
-> Redirected to slot [5474] located at 192.168.254.131:7001
OK
192.168.254.131:7001>
###7001master节点
192.168.254.131:7001> get user
"other-jatham"
192.168.254.131:7001>
###7000master节点
[root@localhost 7000]# redis-cli -c -h 192.168.254.131 -p 7000
192.168.254.131:7000> get user
-> Redirected to slot [5474] located at 192.168.254.131:7001
"other-jatham"
192.168.254.131:7001> -------------这里竟然到了7001
发现我在7000节点访问7005设置的值，结果转向了7005的master节点7001.而且无论在哪里获取设置的值，都会重定向到[5471]slot。
注意：设置值的时候有一个这样的提示
Redirected to slot [5474] located at 192.168.254.131:7001
说明数据存储到了这个槽内。
- 原理
做了这些操作，是不是对其中的一些实现很疑惑，反正我是挺疑惑的，贴一个链接
官方文档：官方教学
原理：redis分槽原理
其他版本配置方式：redis个版本配置方式