Codis（Redis）集群的搭建
准备工作
安装go，查看https://www.jianshu.com/p/2ad2ac4676fb
安装godeo，查看https://www.jianshu.com/p/ea65738eeffb
安装python3，查看https://www.jianshu.com/p/0c06a83a90c7
所有准备工作就绪，开始进入正题
离线安装
安装codis【git clone https://github.com/CodisLabs/codis.git -b release3.2】
Linux下安装时注意【mkreleasehdr.sh、version】两个shell文件可能为格式为Windows格式，需改成UNIX格式
mkdir -p ${GOPATH}/src/github.com/CodisLabs
mv codis ${GOPATH}/src/github.com/CodisLabs
配置环境变量
export CODIS_HOME=${GOPATH}/src/github.com/CodisLabs/codis
export PATH=${CODIS_HOME}/bin:$PATH
赋予执行权限
cd ${CODIS_HOME}
find . -name '*.sh' | xargs chmod +x
find . -name '*.py' | xargs chmod +x
Linux下安装时注意【mkreleasehdr.sh、version】两个shell文件可能为格式为Windows格式，需改成UNIX格式
安装了dos2unix命令使用下列命令，没有的使用notepad++或其他工具自行转换
如果已经为UNIX格式，则不用进行转换
find . -name '*.sh' | xargs dos2unix
find . -name 'version' | xargs dos2unix
find . -name '*.py' | xargs dos2unix
编译源码
make MALLOC=libc
make gotest
编译之后在bin目录中会生成以下文件
[ocetl@einvoice123 bin]$ ll
total 120700
drwxr-x--- 4 ocetl ocetl 4096 Aug 31 14:22 assets
-rwxr-x--- 1 ocetl ocetl 23185474 Aug 31 14:21 codis-admin
-rwxr-x--- 1 ocetl ocetl 24483711 Aug 31 14:21 codis-dashboard
-rwxr-x--- 1 ocetl ocetl 22196925 Aug 31 14:22 codis-fe
-rwxr-x--- 1 ocetl ocetl 20919717 Aug 31 14:21 codis-ha
-rwxr-x--- 1 ocetl ocetl 26735238 Aug 31 14:21 codis-proxy
-rwxr-x--- 1 ocetl ocetl 2682479 Aug 31 14:21 codis-server
-rwxr-x--- 1 ocetl ocetl 274460 Aug 31 14:21 redis-benchmark
-rwxr-x--- 1 ocetl ocetl 405557 Aug 31 14:21 redis-cli
-rwxr-x--- 1 ocetl ocetl 2682479 Aug 31 14:21 redis-sentinel
-rw-r----- 1 ocetl ocetl 95 Aug 31 14:21 version
为了方便管理，为codis执行文件创建单独目录，并将bin目录移动到该文件夹
mkdir ~/app/codis
cp bin -r ~/app/codis
配置和启动各组件
-
配置codis-dashboard
打开dashboard配置文件修改配置，端口为admin_addr的端口
cd ${CODIS_HOME}/config
cp dashboard.toml dashboard-18080.toml
vi dashboard-18080.toml
修改下列属性值
coordinator_name = "zookeeper"
coordinator_addr = "vbtstmdb04:2181,vbdevapp02:2181,vbdevmdb02:2181"
product_name = "codis-yxd"
product_auth = "yxd"
admin_addr = "0.0.0.0:18080"
创建日志目录
mkdir ${CODIS_HOME}/logs
启动脚本
PORT=18080
NCPU=4
CONFIG=${CODIS_HOME}/config/dashboard-${PORT}.toml
LOG=${CODIS_HOME}/logs/dashboard-${PORT}.log
PID=${CODIS_HOME}/logs/dashboard-${PORT}.pid
LEVEL="WARN"
codis-dashboard --ncpu=${NCPU} --config=${CONFIG} --log=${LOG} --pidfile=${PID} --log-level=${LEVEL} &
关闭脚本
PORT=18080
PID=${CODIS_HOME}/logs/dashboard-${PORT}.pid
cat ${PID} | xargs kill
-
codis-proxy
修改proxy配置文件，端口为admin_addr的端口
cd ${CODIS_HOME}/config
cp proxy.toml proxy-11080.toml
vi proxy-11080.toml
修改下列属性值
product_name = "codis-yxd"
product_auth = "yxd"
# session_auth客户端连接密码【jodis代码连接使用密码】
session_auth = "proxy-yxd"
admin_addr = "0.0.0.0:11080"
proxy_addr = "0.0.0.0:19000"
jodis_name = "zookeeper"
jodis_addr = "vbtstmdb04:2181,vbdevapp02:2181,vbdevmdb02:2181"
# 与redis中databases保持一致
backend_number_databases = 1
启动脚本
PORT=11080
NCPU=4
CONFIG=${CODIS_HOME}/config/proxy-${PORT}.toml
LOG=${CODIS_HOME}/logs/proxy-${PORT}.log
PID=${CODIS_HOME}/logs/proxy-${PORT}.pid
LEVEL="WARN"
codis-proxy --ncpu=${NCPU} --config=${CONFIG} --log=${LOG} --pidfile=${PID} --log-level=${LEVEL} &
关闭脚本
PORT=11080
PID=${CODIS_HOME}/logs/proxy-${PORT}.pid
cat ${PID} | xargs kill
-
codis-server
cd ${CODIS_HOME}/config
mkdir ${CODIS_HOME}/data
# Redis密码配置与product_auth一致
# 使用codis-ha时，一个组中多个节点不显示[slaveof]设置主备
vi master-6379.conf
#------------------------------Master配置------------------------------
bind 0.0.0.0
port 6379
dir /sngrp/ocetl/support/gopath/src/github.com/CodisLabs/codis/data
pidfile ../logs/redis-6379.pid
logfile ../logs/redis-6379.log
dbfilename dump-6379.rdb
daemonize yes
databases 1
masterauth "yxd"
requirepass "yxd"
#### Master关闭backsave[rdb] ####
save ""
stop-writes-on-bgsave-error no
vi slave-6380.conf
#--------------------slave配置--------------------
bind 0.0.0.0
port 6380
dir /sngrp/ocetl/support/gopath/src/github.com/CodisLabs/codis/data
pidfile ../logs/redis-6380.pid
logfile ../logs/redis-6380.log
dbfilename dump-6380.rdb
daemonize yes
databases 1
masterauth "yxd"
requirepass "yxd"
#### rdb配置 ####
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error no
slaveof 127.0.0.1 6379
slave-read-only yes
# 启动
codis-server ${CODIS_HOME}/config/master-6379.conf
codis-server ${CODIS_HOME}/config/slave-6380.conf
# 停止
redis-cli -p 6379 <-
codis-fe
cd ${CODIS_HOME}/config
vi codis-8080.json
[
{
"name": "codis-yxd",
"dashboard": "127.0.0.1:18080"
}
]
启动脚本
PORT=8080
NCPU=4
ASSETS=${CODIS_HOME}/bin/assets
CONFIG=${CODIS_HOME}/config/codis-${PORT}.json
LOG=${CODIS_HOME}/logs/fe-${PORT}.log
PID=${CODIS_HOME}/logs/fe-${PORT}.pid
LEVEL="WARN"
codis-fe --ncpu=${NCPU} --assets-dir=${ASSETS} --log=${LOG} --pidfile=${PID} --log-level=${LEVEL} --dashboard-list=${CONFIG} --listen=0.0.0.0:${PORT} &
关闭脚本
PORT=8080
PID=${CODIS_HOME}/logs/fe-${PORT}.pid
cat ${PID} | xargs kill
fe管理界面访问地址（将ip换成你实际安装的ip）
http://fe-ip:8080/
-
codis-admin
codis-admin需要在fe主机上执行，【此命令的操作也可以在界面http://fe-ip:8080完成】
添加代理【多个代理时，修改create-proxy -x的ip:port】
代理掉线后需重新添加代理
codis-admin --dashboard=127.0.0.1:18080 --create-proxy -x 192.168.11.61:11080
codis-admin --dashboard=127.0.0.1:18080 --create-proxy -x 135.188.12.76:11080
codis-admin --dashboard=127.0.0.1:18080 --create-proxy -x 135.188.12.77:11080
- 添加组
codis-admin --dashboard=127.0.0.1:18080 --create-group --gid=1
codis-admin --dashboard=127.0.0.1:18080 --create-group --gid=2
codis-admin --dashboard=127.0.0.1:18080 --create-group --gid=3
每个主备为一组
给组添加/删除Redis服务【group-addr】【group-del：删除时不带datacenter参数】
codis-admin --dashboard=127.0.0.1:18080 --group-add --gid=1 --datacenter=center1 --addr=192.168.11.65:6379
codis-admin --dashboard=127.0.0.1:18080 --group-add --gid=1 --datacenter=center1 --addr=192.168.11.65:6380
codis-admin --dashboard=127.0.0.1:18080 --group-add --gid=2 --datacenter=center2 --addr=135.188.12.76:6379
codis-admin --dashboard=127.0.0.1:18080 --group-add --gid=2 --datacenter=center2 --addr=135.188.12.76:6380
codis-admin --dashboard=127.0.0.1:18080 --group-add --gid=3 --datacenter=center3 --addr=135.188.12.77:6379
codis-admin --dashboard=127.0.0.1:18080 --group-add --gid=3 --datacenter=center3 --addr=135.188.12.77:6380
- 初始化slots并设置server group服务的slot 范围
codis-admin --dashboard=127.0.0.1:18080 --slot-action --create-range --beg=0 --end=340 --gid=1
codis-admin --dashboard=127.0.0.1:18080 --slot-action --create-range --beg=341 --end=682 --gid=2
codis-admin --dashboard=127.0.0.1:18080 --slot-action --create-range --beg=683 --end=1023 --gid=3
-
ha
PORT=18080
INTERVAL=3
DASHBOARD=127.0.0.1:${PORT}
LOG=${CODIS_HOME}/logs/ha-${PORT}.log
LEVEL="WARN"
codis-ha --log=${LOG} --log-level=${LEVEL} --interval=${INTERVAL} --dashboard=${DASHBOARD} &
所有工作都已经准备完成，接下来就是将编译好的codis分发到各个主机上
-
jodis客户端连接
public static final String zkAddr = "vbtstmdb04,vbdevapp02,vbdevmdb02:2181";
public static final int zkSessionTimeoutMs = 30000;
public static final String zkProxyDir = "/jodis/codis-yxd";
public static void main(String[] args) {
JedisResourcePool jedisPool = RoundRobinJedisPool.create().curatorClient(zkAddr, zkSessionTimeoutMs)
.zkProxyDir(zkProxyDir).build();
for (int i = 0; i < 1024; i++) {
try (Jedis jedis = jedisPool.getResource()) {
jedis.set("test:hash:foo:" + i, "bar" + i);
String value = jedis.get("test:hash:foo:" + i);
System.out.println(value);
}
}
}
在线安装
还在搭建