离线安装Redis并构建RedisCluster集群

一、简介

公司要求在内网环境中部署一套Redis集群，根据和开发的商议决定采用redis官方推荐的RedisCluster方式搭建；

系统OS： CentOS6.5
集群环境：三台主机6个节点

主机名	IP地址	端口
RedisCluster-1	192.168.10.111	7000 / 7001
RedisCluster-2	192.168.10.112	7002 / 7003
RedisCluster-3	192.168.10.113	7004 / 7005

软件依赖：需要zlib，zlib-devel，openssl，openssl-devel，gcc，gcc-c++支持
注：如果没有则到https://pkgs.org/ 网站查找相关依赖包进行安装
redisCluster依赖：ruby、rubygems、redis.gem、tcl等软件支持
注：如果为外网环境则可直接yum -y install ruby rubygems tcl进行安装

二、离线安装ruby

下载ruby-2.3.1并上传至服务器
https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.1.tar.gz

# 解压tar包
$ tar zxf ruby-2.3.1.tar.gz
$ cd ruby-2.3.1
# 配置并编译
$ ./configure —prefix=/opt/ruby
$ make && make install
# 配置环境变量
$ echo “PATH=$PATH:/opt/ruby/bin” >> /etc/profile
$ source /etc/profile
$ ruby -v
ruby 1.8.7 (2013-06-27 patchlevel 374) [x86_64-linux]

三、离线安装rubygems

下载rubygems-2.7.6.tgz并上传至服务器
https://rubygems.org/rubygems/rubygems-2.7.6.tgz

# 解压到/opt目录
$ tar zxf rubygems-2.7.6.tgz -C /opt
$ cd /opt/rubygems-2.7.6
# 配置rubygem
$ ruby setup.rb #会将rubygems安装至上一步ruby的安装目录的lib目录下
# 配置环境变量，使其支持gem命令
$ echo “PATH=$PATH:/opt/rubygems-2.7.6/bin” >> /etc/profile
$ source /etc/profile
# 验证gem是否可以使用
$ gem -v
   2.7.6

四、离线配置rubygems的redisapi

下载redis的gem并上传至服务器
https://rubygems.org/downloads/redis-3.3.0.gem

gem install -l redis-3.3.0.gem
gem list redis

*** LOCAL GEMS ***

redis (3.3.0)

五、离线安装tcl

下载tcl8并上传至服务器
https://sourceforge.net/projects/tcl/files/Tcl/8.6.0/

# 解压tcl8.6.1软件包
$ tar zxf tcl8.6.1-src.tar.gz
$ cd tcl8.6.0/unix
# 配置并编译
$ ./configure —prefix=/opt/tcl
$ make && make install
$ make install-private-headers
# 配置环境软连接及环境变量使其支持tcl
$ ln -v -sf tclsh8.6 /opt/tcl/bin/tclsh
$ chmod -v 755 /opt/tcl/lib/libtcl8.6.so
$ echo “PATH=$PATH:/opt/tcl/bin” >> /etc/profile
$ sourch /etc/profile

六、离线安装redis

下载redis3.2.0并上传至服务器
http://download.redis.io/releases/redis-3.2.0.tar.gz

$ tar zxf redis-3.2.0.tar.gz -C /opt
$ cd /opt/redis-3.2.0/src
# make安装redis(如果不修改Makefile文件则默认安装/usr/local/bin下)
$ make
$ make test
$ make install
# 安装完成后会自动在/usr/local/bin下生成redis运行文件
# 将集群控制脚本复制到/usr/local/bin下
$ cp /opt/redis-3.2.0/src/redis-trib.rb /usr/local/bin

注： 以上所有操作需在三台机器都执行一遍

七、编写redis配置文件并启动实例

1、在三台主机创建redis-cluster目录

# 192.168.10.111
$ mkdir -p /etc/redis-cluster/{7000,7001}
# 192.168.10.112
$ mkdir -p /etc/redis-cluster/{7002,7003}
# 192.168.10.113
$ mkdir -p /etc/redis-cluster/{7004,7005}

2、对应各个实例创建redis.conf配置文件

# 192.168.10.111
$ cd /etc/redis-cluster/7000
$ touch redis.con
$ cat redis.conf
#redis后台运行
daemonize    yes
# pidfile文件对应目录
pidfile  /var/run/redis_7000.pid
# 指定端口
port 7000
# 绑定地址
bind 192.168.10.111
# 开启集群  把注释#去掉
cluster-enabled  yes
# 集群的配置  配置文件首次启动会自动生成
cluster-config-file  nodes_7000.conf
# 请求超时  设置5秒够了
cluster-node-timeout  5000
# aof日志开启  有需要就开启，它会每次写操作都记录一条日志
appendonly  yes

将该配置文件复制到各主机对应的目录中并修改文件中的下列几项

2、启动redis实例

# 192.168.10.111
$ redis-server /etc/redis-cluster/7000/redis.conf
$ redis-server /etc/redis-cluster/7001/redis.conf
$ ps -ef | grep redis-server
root       5494      1  0 06:31 ?        00:00:00 redis-server 192.168.10.111:7000 [cluster]
root       5498      1  0 06:31 ?        00:00:00 redis-server 192.168.10.111:7001 [cluster]
# 192.168.10.112
$ redis-server /etc/redis-cluster/7002/redis.conf
$ redis-server /etc/redis-cluster/7003/redis.conf
$ ps -ef | grep redis-server
root      13356      1  0 06:33 ?        00:00:00 redis-server 192.168.10.112:7002 [cluster]
root      13360      1  0 06:33 ?        00:00:00 redis-server 192.168.10.112:7003 [cluster]
# 192.168.10.113
$redis-server /etc/redis-cluster/7004/redis.conf
$ redis-server /etc/redis-cluster/7005/redis.conf
$ ps -ef | grep redis-server
root       1854      1  0 06:34 ?        00:00:00 redis-server 192.168.10.113:7004 [cluster]
root       1858      1  0 06:34 ?        00:00:00 redis-server 192.168.10.113:7005 [cluster]

可编写redis服务启动脚本用server来管理

# 
$ vim /etc/init.d/redis-7000
#!/bin/sh
#
# Simple Redis init.d script conceived to work on Linux systems
# as it does use of the /proc filesystem.
# chkconfig:   2345 90 10
# description:  Redis is a persistent key-value database

export PATH=$PATH:/usr/local/bin/

REDISPORT=7000
PIDFILE=/var/run/redis_$REDISPORT.pid
CONF="/etc/redis-cluster/$REDISPORT/redis.conf"

case "$1" in
    status)
    if [ ! -f $PIDFILE ]
    then
        echo "Redis stopped..."
    else
        ps -ef | grep redis-server | grep $REDISPORT
    fi
    ;;
    start)
        if [ -f $PIDFILE ]
        then
                echo "$PIDFILE exists, process is already running or crashed"
        else
                echo "Starting Redis server..."
                redis-server $CONF
        fi
        ;;
    stop)
        if [ ! -f $PIDFILE ]
        then
                echo "$PIDFILE does not exist, process is not running"
        else
                PID=$(cat $PIDFILE)
                echo "Stopping ..."
                ps -ef | grep redis-server | grep $REDISPORT | awk '{print $2}' | xargs kill
                while [ -x /proc/${PID} ]
                do
                    echo "Waiting for Redis to shutdown ..."
                    sleep 1
                done
                echo "Redis stopped"
        fi
        ;;
    restart|force-reload)
        ${0} stop
        ${0} start
        ;;
    *)
        echo "Please use start or stop as first argument"
        ;;
esac

$ chkconfig --add redis-7000
$ chkconfig redis-7000 on
# 将服务脚本复制五份放到各自/etc/init.d/下并修改REDISPORT对应的值

###八、启动rediscluster集群
1、使用ruby自带的redis-trib.rb命令在192.168.10.111上构建

# --replicas 则指定了为Redis Cluster中的每个Master节点配备几个Slave节点
$ redis-trib.rb create --replicas 1 192.168.10.111:7000 192.168.10.111:7001 192.168.10.112:7002 192.168.10.112:7003
192.168.10.113:7004 192.168.10.113:7005
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.10.113:7004
192.168.10.112:7002
192.168.10.111:7000
Adding replica 192.168.10.112:7003 to 192.168.10.113:7004
Adding replica 192.168.10.113:7005 to 192.168.10.112:7002
Adding replica 192.168.10.111:7001 to 192.168.10.111:7000
M: 985750026ac9434fab986b54a2a258089d171617 192.168.10.111:7000
   slots:10923-16383 (5461 slots) master
S: bd77f34d038d05e9a586dacd447bc10d7472ccb1 192.168.10.111:7001
   replicates 985750026ac9434fab986b54a2a258089d171617
M: 598a585000c07354483019cd3561fb1c6ddcb755 192.168.10.112:7002
   slots:5461-10922 (5462 slots) master
S: c0ab83ae1a8ef4b9c450ae6538ee568fa2a1d1a9 192.168.10.112:7003
   replicates 3b18cf1ca717226e73664a0025cd572b798655bc
M: 3b18cf1ca717226e73664a0025cd572b798655bc 192.168.10.113:7004
   slots:0-5460 (5461 slots) master
S: f140afda21eba172b8a07229c78f090d9b062e8e 192.168.10.113:7005
   replicates 598a585000c07354483019cd3561fb1c6ddcb755
Can I set the above configuration? (type 'yes' to accept): yes    #在此处输入“yes”
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join..
>>> Performing Cluster Check (using node 192.168.10.111:7000)
M: 985750026ac9434fab986b54a2a258089d171617 192.168.10.111:7000
   slots:10923-16383 (5461 slots) master
M: bd77f34d038d05e9a586dacd447bc10d7472ccb1 192.168.10.111:7001
   slots: (0 slots) master
   replicates 985750026ac9434fab986b54a2a258089d171617
M: 598a585000c07354483019cd3561fb1c6ddcb755 192.168.10.112:7002
   slots:5461-10922 (5462 slots) master
M: c0ab83ae1a8ef4b9c450ae6538ee568fa2a1d1a9 192.168.10.112:7003
   slots: (0 slots) master
   replicates 3b18cf1ca717226e73664a0025cd572b798655bc
M: 3b18cf1ca717226e73664a0025cd572b798655bc 192.168.10.113:7004
   slots:0-5460 (5461 slots) master
M: f140afda21eba172b8a07229c78f090d9b062e8e 192.168.10.113:7005
   slots: (0 slots) master
   replicates 598a585000c07354483019cd3561fb1c6ddcb755
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

启动后系统会自动分配主从
###九、rediscluster命令使用
1.命令介绍

redis-server #实例启动脚本，一般结合redis.conf文件使用
redis-cli #redis客户端程序
redis-trib.rb #集群控制脚本，用于构建或查询集群信息

2.redis-cli使用介绍

$ redis-cli -h 主机名 -p端口
# 查看群集状态命令（下列命令在redis-cli终端执行）
> cluster info      #打印集群信息
> cluster nodes     #列出集群当前节点信息

3.redis-trib.rb使用介绍

# 创建集群
$ redis-trib.rb create --replicas 1 xxx.xxx.xxx.xxx:xxx …
# 查看集群状态信息
$ redis-trib.rb check IP地址:端口

###十、单节点故障解决方案

1.用redis-trib.rb check命令找到具体哪台节点挂掉；

$ redis-trib.rb check IP地址:端口

2.登入到该节点主机查看对应节点进程是否还存在，如果存在则kill

$ ps -ef | grep redis

3.进入配置文件存放目录，重启redis节点

$ cd /data/redis-cluster/
$ redis-server $端口目录/redis.conf
$ ps -ef | grep redis           #查看进程是否起来
root  1667  1  0 08:24 ?    00:00:38 redis-server 172.50.18.116:6379 [cluster]
root     1671   1  0 08:24 ?    00:00:38 redis-server 172.50.18.116:6380 [cluster]
$ redis-trib.rb check $IP:端口    
#查看是否自动加入集群

4.注：如果自动加入失败请上网查看redis-trib.rb加入集群命令

十一、集群重启解决方案

1. 登入各个主机进入集群配置目录删除集群相关文件（一下命令需在三台机器都执行）

   $ cd /data/redis-cluster
   # 删除集群配置文件，保留6379/6380配置文件及数据文件；
   $ rm -rf appendonly.aof nodes_* 

2. 杀死残留redis相关进程

   $ killall -9 redis-server           #杀死所以redis-server进程
   $ ps -ef | grep redis           #查看是否存留redis进程

3.使用redis-trib命令重启集群（下列命令只需在一台机器执行即可）

$ redis-trib.rb create --replicas 1 172.50.18.116:6379 172.50.18.116:6380 172.50.18.5:6379 172.50.18.5:6380 172.50.17.79:6389 172.50.17.79:6380
# 查看输出信息，确认所以节点都自动分配成功

转载于:https://blog.51cto.com/wuyunkeji/2114013