迁移数据到Redis Cluster
目录
一、数据迁移脚本树形图
二、数据丢失问题
三、执行脚本redis_data_migration.sh
四、全局参数配置文件redis_data_migration.conf
五、原项目对应Redis服务器IP&PORT文件
六、Redis Cluster IP&PORT文件
一、数据迁移脚本树形图
| /home/work/wufei/migration ├── bin │ └── redis_data_migration.sh ├── conf │ └── redis_data_migration.conf ├── ip_port │ ├── migration_redis_ip_port.bim │ ├── migration_redis_ip_port.im │ ├── redis_cluster_ip_port.bim │ ├── redis_cluster_ip_port.im │ └── redis_cluster_ip_port.inf ├── del │ ├── migration_del.log │ └── migration_del.success │ └── migration_del.error ├── hash │ ├── migration_hash.log │ └── migration_hash.success │ └── migration_hash.exists │ └── migration_hash.error ├── list │ ├── migration_list.error │ ├── migration_list.log │ └── migration_list.success ├── Set │ ├── migration_Set.exists │ ├── migration_Set.log │ └── migration_Set.success │ └── migration_Set.error └── sSet │ ├── migration_sSet.log │ └── migration_sSet.success │ └── migration_sSet.error ├── expire │ ├── migration_expire.error │ ├── migration_expire.log │ └── migration_expire.success └── string ├── migration_string.log └── migration_string.success └── migration_string.error |
1. bin/redis_data_migration.sh：执行脚本（支持string、hash、list、Set集合、有序集合(sorted set)迁移数据到Redis Cluster，还支持批量删除key值，添加过期时间等）
2. conf/redis_data_migration.conf：全局参数配置文件
2.1原key值前缀、key值操作类型[string/hash/list/Set/sSet/del]、公有路径、原Redis服务器IP&PORT文件、Redis Cluster IP&PORT文件、Redis Cluster 连接节点IP、Redis Cluster 连接PORT、老 Redis 连接密码[jy34@MLshuoMG]，没有设为空等参数，KEY值过期设置等
2.2 过期设置类型：
1 EXPIRE key seconds //将key的生存时间设置为ttl秒
2 PEXPIRE key milliseconds //将key的生成时间设置为ttl毫秒
3 EXPIREAT key timestamp //将key的过期时间设置为timestamp所代表的的秒数的时间戳
4 PEXPIREAT key milliseconds-timestamp //将key的过期时间设置为timestamp所代表的的毫秒数的时间戳
3. ip_port/migration_redis_ip_port.[project]：原项目对应Redis服务器IP&PORT文件（如higo_bim：migration_redis_ip_port.bim）
4. ip_port/redis_cluster_ip_port.[project]：Redis Cluster IP&PORT文件（如higo_bim：redis_cluster_ip_port.bim；inf环境：redis_cluster_ip_port.inf）
5. string、hash、list、Set、sSet、expire目录为在redis_data_migration.conf配置key值操作类型后生成的对应目录，并生成对应的日志文件：
5.1 migration_[key_type].log：日志文件；
| 类型 |
举个栗子 |
| string |
IP:PORT=>xx.xx.1.24:6379##key=>wealthservice:bim_image_id_xxxxx##value=>a:10:{s:2:"id";i:113113;s:8:"image_id";i:xxxxx;s:10:"image_path";s:62:"http://xxxx.xxxx.com/xxx/54654b016368aa6f0d13b2efd3d45160.jpg";s:10:"image_hash";s:28:"Ft7VHRmd5S5ZvKdTqkvVz0LVqnmo";s:11:"image_width";i:900;s:12:"image_height";i:1200;s:10:"image_size";i:289858;s:12:"image_status";i:1;s:11:"image_mtime";s:19:"2018-12-02 17:54:31";s:11:"image_ctime";s:19:"2018-12-02 17:54:31";}##status=>success. |
| hash |
IP:PORT=>xx.xx.1.24:6379##key=>wealthservice:xxx_new_xxxxx##field=>2018112121_2805xxxxx_new##value=>0.04##hexists=>0##status=>success. |
| list |
IP:PORT=>xx.xx.0.47:6489##key=>xxxxim:ImMsgQueueShop:2:sync472xxx525##value=>{"msg_id":"1567414277","uid":3,"shop_id":0}##status=>lrem error. IP:PORT=>xx.xx.0.40:6479##key=>xxxxim:ImMsgQueueShop:2:sync425xxx979##value=>{"msg_id":1746206575,"uid":"1200882445","shop_id":0}##status=>success. IP:PORT=>xx.xx.0.47:6489##key=>xxxxim:ImMsgQueueShop:2:sync426xxx901##value=>{"msg_id":"1588692513","uid":1200762040,"shop_id":0}##status=>lrem error. |
| Set |
IP:PORT=>xx.xx.1.22:6379##key=>wealthservice:t_bim_fw_followers_openIds##member=>orwuH1HRTMhk_td3EpWrSW4fX-Io##sismember=>0##status=>success. |
| sSet | IP:PORT=>xx.xx.1.24:6379##key=>bimuserservice:t_bim_most_publish_2018-12-11##score=>3##member=>2318608xxxxx653901##status=>success. |
5.2 migration_[key_type].success：迁移成功的执行命令日志；
| 类型 |
举个栗子 |
| string |
redis-cli -h xx.xx.2.35 -c -p 7001 setnx wealthservice:bim_image_id_xxxxx 'a:10:{s:2:"id";i:113113;s:8:"image_id";i:2814692718xxxxx980;s:10:"image_path";s:62:"http://xxxx.xxxx.com/xxx/54654b016368aa6f0d13b2efd3d45160.jpg";s:10:"image_hash";s:28:"Ft7VHRmd5S5ZvKdTqkvVz0LVqnmo";s:11:"image_width";i:900;s:12:"image_height";i:1200;s:10:"image_size";i:289858;s:12:"image_status";i:1;s:11:"image_mtime";s:19:"2018-12-02 17:54:31";s:11:"image_ctime";s:19:"2018-12-02 17:54:31";}' |
| hash |
redis-cli -h xx.xx.2.35 -c -p 7001 hset wealthservice:bim_new_2463250059xxxxx923 2018112121_280532297xxxxx6946711_new '0.04' |
| list |
redis-cli -h xx.xx.2.35 -c -p 7001 lrem higoim:ImMsgQueueShop:2:sync42662xx01 0 '{"msg_id":"1624xxx093","uid":1200xxx315,"shop_id":0}' |
| Set |
redis-cli -h xx.xx.2.35 -c -p 7001 sadd wealthservice:t_bim_fw_followers_openIds 'orwuH1HRTMhk_td3EpWrSW4fX-Io' |
| sSet | redis-cli -h xx.xx.2.35 -c -p 7001 zadd bimuserservice:t_bim_most_publish_2018-12-11 3 '231860805xxxxx3901' |
5.3 migration_[key_type].error：迁移失败的执行命令日志（同5.2）；
5.4 migration_[key_type].exists：迁移中重复值的执行命令日志，如hash中字段是否存在，Set中member元素是否是集合 key 的成员（同5.2）；
二、数据丢失问题
migration_[key_type].[status]日志文件很好的解决了执行失败等数据丢失问题，在提前双写或者老的业务停止的前提下，迁移过程绝对不会丢失任何数据，只要老的Redis中有的，Redis Cluster中就一定会有。
三、执行脚本redis_data_migration.sh
#!/bin/bash
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# Program : Redis 数据迁移脚本 #
# Redis Cluster : redis-4.0.10 #
# Author : [email protected] #
# Date : 2018-12-11 #
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# 更新时间：2018-12-24；更新人：WUFEI；更新内容：增加Redis过期限制方法
# 读取配置文件中的所有变量，设置为全局变量
conf_file="../conf/redis_data_migration.conf"
# 获取Redis Cluster中key值前缀
cluster_prefix=`sed '/^cluster_prefix=/!d;s/.*=//' ${conf_file}`
# 获取原key值前缀
key_prefix=`sed '/^key_prefix=/!d;s/.*=//' ${conf_file}`
# 获取key值类型
key_type=`sed '/^key_type=/!d;s/.*=//' ${conf_file}`
# 获取公有路径
public_path=`sed '/^public_path=/!d;s/.*=//' ${conf_file}`
# 获取原Redis服务器IP&PORT文件
ip_port_file=`sed '/^ip_port_file=/!d;s/.*=//' ${conf_file}`
# 获取所有Redis Cluster节点IP&PORT文件
cluster_ip_file=`sed '/^cluster_ip_file=/!d;s/.*=//' ${conf_file}`
# 获取Redis Cluster IP
cluster_ip=`sed '/^cluster_ip=/!d;s/.*=//' ${conf_file}`
# 获取Redis Cluster PORT
cluster_port=`sed '/^cluster_port=/!d;s/.*=//' ${conf_file}`
# Redis Cluster 连接
cluster_cli="redis-cli -h ${cluster_ip} -c -p ${cluster_port}"
# 获取老 Redis 连接密码
redis_auth=`sed '/^redis_auth=/!d;s/.*=//' ${conf_file}`
# 获取过期过期设置类型
expire_ypte=`sed '/^expire_ypte=/!d;s/.*=//' ${conf_file}`
# 获取过期类型对应形式的时间
expire_ttl=`sed '/^expire_ttl=/!d;s/.*=//' ${conf_file}`
# 创建相关目录
if [ ! -d "${public_path}/${key_type}" ]
then
mkdir -p ${public_path}/${key_type}
fi
# 判断是否存在密码
if [ -z ${redis_auth} ]
then
# 如果密码为空设置判断值为0
redis_auth_judge=0
else
# 如果密码不为空设置判断值为1
redis_auth_judge=1
fi
# 迁移string类型的数据
function migration_string(){
while read ip_port
do
old_ip=`echo $ip_port | awk '{print $1}'`
old_port=`echo $ip_port | awk '{print $2}'`
# 判断老的Redis连接是否需要添加'-a'参数
if [ 1 -eq ${redis_auth_judge} ]; then
old_redis_cli="redis-cli -h ${old_ip} -p ${old_port} -a ${redis_auth}"
else
old_redis_cli="redis-cli -h ${old_ip} -p ${old_port}"
fi
# 循环获取KEY名
${old_redis_cli} keys "${key_prefix}*" | while read old_key
do
# 判断是否为空
if [[ -z ${old_key} ]]
then
continue
fi
# 获取KEY名对应VALUE值
old_value=`${old_redis_cli} get ${old_key}`
# 只有在 key 不存在时设置 key 的值。
${cluster_cli} setnx ${cluster_prefix}${old_key} "${old_value}"
import_0k=$?
if [ 0 -eq "${import_0k}" ]; then
echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##value=>${old_value}##status=>success." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${cluster_cli} setnx ${cluster_prefix}${old_key} '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.success
else
echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##value=>${old_value}##status=>error." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${cluster_cli} setnx ${cluster_prefix}${old_key} '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.error
fi
done
done < ${public_path}/${ip_port_file}
}
# 迁移hash类型的数据
function migration_hash(){
while read ip_port
do
old_ip=`echo $ip_port | awk '{print $1}'`
old_port=`echo $ip_port | awk '{print $2}'`
# 判断老的Redis连接是否需要添加'-a'参数
if [ 1 -eq ${redis_auth_judge} ]; then
old_redis_cli="redis-cli -h ${old_ip} -p ${old_port} -a ${redis_auth}"
else
old_redis_cli="redis-cli -h ${old_ip} -p ${old_port}"
fi
# 循环获取KEY名
${old_redis_cli} keys "${key_prefix}*" | while read old_key
do
# 判断是否为空
if [[ -z ${old_key} ]]
then
continue
fi
# 循环获取所有哈希表中的字段
${old_redis_cli} hkeys ${old_key} | while read old_h_keys
do
# 获取给定字段的值
old_h_value=`${old_redis_cli} hget ${old_key} ${old_h_keys}`
# 判断Redis Cluster中是否存在
hexists_num=`${cluster_cli} hexists ${cluster_prefix}${old_key} "${old_h_keys}"`
if [ 0 -eq "${hexists_num}" ];then
# 如果不存在则插入
${cluster_cli} hset ${cluster_prefix}${old_key} "${old_h_keys}" "${old_h_value}"
import_0k=$?
if [ 0 -eq "${import_0k}" ]; then
echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##field=>${old_h_keys}##value=>${old_h_value}##hexists=>${hexists_num}##status=>success." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${cluster_cli} hset ${cluster_prefix}${old_key} ${old_h_keys} '${old_h_value}'" >> ${public_path}/${key_type}/migration_${key_type}.success
else
echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##field=>${old_h_keys}##value=>${old_h_value}##hexists=>${hexists_num}##status=>error." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${cluster_cli} hset ${cluster_prefix}${old_key} ${old_h_keys} '${old_h_value}'" >> ${public_path}/${key_type}/migration_${key_type}.error
fi
else
echo "${cluster_cli} hset ${cluster_prefix}${old_key} ${old_h_keys} '${old_h_value}'" >> ${public_path}/${key_type}/migration_${key_type}.exists
echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##field=>${old_h_keys}##value=>${old_h_value}##hexists=>${hexists_num}##status=>is exists." >> ${public_path}/${key_type}/migration_${key_type}.log
fi
done
done
done < ${public_path}/${ip_port_file}
}
# 迁移list类型的数据
function migration_list(){
while read ip_port
do
old_ip=`echo $ip_port | awk '{print $1}'`
old_port=`echo $ip_port | awk '{print $2}'`
# 判断老的Redis连接是否需要添加'-a'参数
if [ 1 -eq ${redis_auth_judge} ]; then
old_redis_cli="redis-cli -h ${old_ip} -p ${old_port} -a ${redis_auth}"
else
old_redis_cli="redis-cli -h ${old_ip} -p ${old_port}"
fi
# 循环获取KEY名
${old_redis_cli} keys "${key_prefix}*" | while read old_key
do
# 判断是否为空
if [[ -z ${old_key} ]]
then
continue
fi
# 获取列表所以的元素
${old_redis_cli} lrange ${old_key} 0 -1 | while read old_value
do
# 判断元素是否为空
if [[ -z ${old_value} ]]
then
continue
fi
# 移除列表与${old_value}想等的所有元素
${cluster_cli} lrem ${cluster_prefix}${old_key} 0 "${old_value}"
import_0k=$?
if [ 0 -eq "${import_0k}" ]; then
${cluster_cli} rpush ${cluster_prefix}${old_key} "${old_value}"
import_1k=$?
if [ 0 -eq "${import_1k}" ]; then
echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##value=>${old_value}##status=>success." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${cluster_cli} lrem ${cluster_prefix}${old_key} 0 '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.success
echo "${cluster_cli} rpush ${cluster_prefix}${old_key} '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.success
else
echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##value=>${old_value}##status=>rpush error." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${cluster_cli} lrem ${cluster_prefix}${old_key} 0 '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.error
echo "${cluster_cli} rpush ${cluster_prefix}${old_key} '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.error
fi
else
echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##value=>${old_value}##status=>lrem error." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${cluster_cli} lrem ${cluster_prefix}${old_key} 0 '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.error
echo "${cluster_cli} rpush ${cluster_prefix}${old_key} '${old_value}'" >> ${public_path}/${key_type}/migration_${key_type}.error
fi
done
done
done < ${public_path}/${ip_port_file}
}
# 迁移Set集合类型的数据
function migration_Set(){
while read ip_port
do
old_ip=`echo $ip_port | awk '{print $1}'`
old_port=`echo $ip_port | awk '{print $2}'`
# 判断老的Redis连接是否需要添加'-a'参数
if [ 1 -eq ${redis_auth_judge} ]; then
old_redis_cli="redis-cli -h ${old_ip} -p ${old_port} -a ${redis_auth}"
else
old_redis_cli="redis-cli -h ${old_ip} -p ${old_port}"
fi
# 循环获取KEY名
${old_redis_cli} keys "${key_prefix}*" | while read old_key
do
# 判断是否为空
if [[ -z ${old_key} ]]
then
continue
fi
# 返回集合中的所有成员
${old_redis_cli} smembers ${old_key} | while read old_member
do
# 判断member是否为空
if [[ -z ${old_member} ]]
then
continue
fi
# 判断 member 元素是否是Redis Cluster集合 key 的成员
sismember_judge=`${cluster_cli} sismember ${cluster_prefix}${old_key} "${old_member}"`
if [[ 0 -eq "${sismember_judge}" ]];then
# 如果不存在则插入
${cluster_cli} sadd ${cluster_prefix}${old_key} "${old_member}"
import_0k=$?
if [ 0 -eq "${import_0k}" ]; then
echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##member=>${old_member}##sismember=>${sismember_judge}##status=>success." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${cluster_cli} sadd ${cluster_prefix}${old_key} '${old_member}'" >> ${public_path}/${key_type}/migration_${key_type}.success
else
echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##member=>${old_member}##sismember=>${sismember_judge}##status=>error." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${cluster_cli} sadd ${cluster_prefix}${old_key} '${old_member}'" >> ${public_path}/${key_type}/migration_${key_type}.error
fi
else
echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##member=>${old_member}##sismember=>${sismember_judge}##status=>exists." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${cluster_cli} sadd ${cluster_prefix}${old_key} '${old_member}'" >> ${public_path}/${key_type}/migration_${key_type}.exists
fi
done
done
done < ${public_path}/${ip_port_file}
}
# 迁移有序集合类型的数据
function migration_sSet(){
while read ip_port
do
old_ip=`echo $ip_port | awk '{print $1}'`
old_port=`echo $ip_port | awk '{print $2}'`
# 判断老的Redis连接是否需要添加'-a'参数
if [ 1 -eq ${redis_auth_judge} ]; then
old_redis_cli="redis-cli -h ${old_ip} -p ${old_port} -a ${redis_auth}"
else
old_redis_cli="redis-cli -h ${old_ip} -p ${old_port}"
fi
# 循环获取KEY名
${old_redis_cli} keys "${key_prefix}*" | while read old_key
do
# 判断是否为空
if [[ -z ${old_key} ]]
then
continue
fi
# 返回集合中的所有成员
${old_redis_cli} zrange ${old_key} 0 -1 | while read old_member
do
# 判断member是否为空
if [[ -z ${old_member} ]]
then
continue
fi
# 返回有序集中，成员的分数值
zscore_key_member=`${old_redis_cli} zscore ${old_key} "${old_member}"`
if [[ 0 -le "${zscore_key_member}" ]];then
${cluster_cli} zadd ${cluster_prefix}${old_key} ${zscore_key_member} "${old_member}"
import_0k=$?
if [ 0 -eq "${import_0k}" ]; then
echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##score=>${zscore_key_member}##member=>${old_member}##status=>success." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${cluster_cli} zadd ${cluster_prefix}${old_key} ${zscore_key_member} '${old_member}'" >> ${public_path}/${key_type}/migration_${key_type}.success
else
echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##score=>${zscore_key_member}##member=>${old_member}##status=>error." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${cluster_cli} zadd ${cluster_prefix}${old_key} ${zscore_key_member} '${old_member}'" >> ${public_path}/${key_type}/migration_${key_type}.error
fi
else
echo "IP:PORT=>${old_ip}:${old_port}##key=>${cluster_prefix}${old_key}##score=>${zscore_key_member}##member=>${old_member}##status=>error." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${cluster_cli} zadd ${cluster_prefix}${old_key} ${zscore_key_member} '${old_member}'" >> ${public_path}/${key_type}/migration_${key_type}.error
fi
done
done
done < ${public_path}/${ip_port_file}
}
# 批量删除数据
function migration_del(){
while read ip_port
do
cluster_ip=`echo $ip_port | awk '{print $1}'`
cluster_port=`echo $ip_port | awk '{print $2}'`
redis_cluster_cli="redis-cli -h ${cluster_ip} -c -p ${cluster_port}"
# 循环获取KEY名
${redis_cluster_cli} keys "${cluster_prefix}${key_prefix}*" | while read cluster_key
do
# 判断是否为空
if [[ -z ${cluster_key} ]]
then
continue
fi
# 删除key值
${redis_cluster_cli} del "${cluster_key}"
import_0k=$?
if [ 0 -eq "${import_0k}" ]; then
echo "IP:PORT=>${cluster_ip}:${cluster_port}##key=>${cluster_key}##status=>del success." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${redis_cluster_cli} del ${cluster_key}" >> ${public_path}/${key_type}/migration_${key_type}.success
else
echo "IP:PORT=>${cluster_ip}:${cluster_port}##key=>${cluster_key}##status=>del error." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${redis_cluster_cli} del ${cluster_key}" >> ${public_path}/${key_type}/migration_${key_type}.error
fi
done
done < ${public_path}/${cluster_ip_file}
}
# KEY值过期限制
function migration_expire(){
# 1 EXPIRE key seconds //将key的生存时间设置为ttl秒
# 2 PEXPIRE key milliseconds //将key的生成时间设置为ttl毫秒
# 3 EXPIREAT key timestamp //将key的过期时间设置为timestamp所代表的的秒数的时间戳
# 4 PEXPIREAT key milliseconds-timestamp //将key的过期时间设置为timestamp所代表的的毫秒数的时间戳
# 1、2两种方式是设置一个过期的时间段，就是咱们处理验证码最常用的策略，设置三分钟或五分钟后失效，把分钟数转换成秒或毫秒存储到Redis中。
# 3、4两种方式是指定一个过期的时间 ，比如优惠券的过期时间是某年某月某日，只是单位不一样。
# # 返回值
# # 一个整数值1或0，如果成功地为该键设置了超时时间，返回 1,如果键不存在或无法设置超时时间，返回 0.
while read ip_port
do
cluster_ip=`echo $ip_port | awk '{print $1}'`
cluster_port=`echo $ip_port | awk '{print $2}'`
redis_cluster_cli="redis-cli -h ${cluster_ip} -c -p ${cluster_port}"
# 循环获取KEY名
${redis_cluster_cli} keys "${cluster_prefix}${key_prefix}*" | while read cluster_key
do
# 判断是否为空
if [[ -z ${cluster_key} ]]
then
continue
fi
# 判断是否已经设置了过期时间
judge_ttl=`${redis_cluster_cli} ttl ${cluster_key}`
if [[ ${judge_ttl} -gt 0 ]]
then
continue
fi
# 判断过期设置类型,并设置过期时间
${redis_cluster_cli} ${expire_ypte} ${cluster_key} ${expire_ttl}
import_0k=$?
if [ 0 -eq "${import_0k}" ]; then
echo "IP:PORT=>${cluster_ip}:${cluster_port}##key=>${cluster_key}##status=>${expire_ypte} success." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${redis_cluster_cli} ${expire_ypte} ${cluster_key} ${expire_ttl}" >> ${public_path}/${key_type}/migration_${key_type}.success
else
echo "IP:PORT=>${cluster_ip}:${cluster_port}##key=>${cluster_key}##status=>${expire_ypte} error." >> ${public_path}/${key_type}/migration_${key_type}.log
echo "${redis_cluster_cli} ${expire_ypte} ${cluster_key} ${expire_ttl}" >> ${public_path}/${key_type}/migration_${key_type}.error
fi
done
done < ${public_path}/${cluster_ip_file}
}
function main(){
if [ ${key_type} = 'string' ]
then
# 迁移string类型的数据
migration_string
elif [ ${key_type} = 'hash' ]
then
# 迁移hash类型的数据
migration_hash
elif [ ${key_type} = 'list' ]
then
# 迁移list类型的数据
migration_list
elif [ ${key_type} = 'Set' ]
then
# 迁移Set集合类型的数据
migration_Set
elif [ ${key_type} = 'sSet' ]
then
# 迁移有序集合类型的数据
migration_sSet
elif [ ${key_type} = 'del' ]
then
# 批量删除数据
migration_del
elif [ ${key_type} = 'expire' ]
then
# KEY值过期限制
migration_expire
else
# 配置文件错误
echo "配置文件${conf_file}参数错误，请仔细核对相应参数!"
fi
}
main
四、全局参数配置文件redis_data_migration.conf
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# Program : Redis 数据迁移配置文件 #
# Author : [email protected] #
# Date : 2018-12-11 #
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# Redis Cluster中key值前缀
cluster_prefix=PUSH:1.106102
# 原key值前缀
key_prefix=
# key值操作类型
key_type=del
# 公有路径
public_path=/home/work/wufei/migration
# 原Redis服务器IP&PORT文件
ip_port_file=ip_port/migration_redis_ip_port.push
# Redis Cluster IP&PORT文件
cluster_ip_file=ip_port/redis_cluster_ip_port.push
# Redis Cluster 连接节点IP
cluster_ip=xx.xx.2.35
# Redis Cluster 连接PORT
cluster_port=7001
# 老 Redis 连接密码(jy34@MLshuoMG)，没有设为空
redis_auth=xxxx@MLxxxxMG
# 过期设置类型
# 1 EXPIRE key seconds //将key的生存时间设置为ttl秒
# 2 PEXPIRE key milliseconds //将key的生成时间设置为ttl毫秒
# 3 EXPIREAT key timestamp //将key的过期时间设置为timestamp所代表的的秒数的时间戳
# 4 PEXPIREAT key milliseconds-timestamp //将key的过期时间设置为timestamp所代表的的毫秒数的时间戳
# expire_ypte可设置为expire | pexpire | expireat | pexpireat
# expire_ttl设置为expire_ypte所选择形式的对应时间类型
expire_ypte=expire
expire_ttl=2592000五、原项目对应Redis服务器IP&PORT文件
（如higo_bim：migration_redis_ip_port.bim）
xx.xx.1.15 6379
xx.xx.1.16 6379
xx.xx.1.18 6379
xx.xx.1.19 6379
xx.xx.1.20 6379
xx.xx.1.21 6379
xx.xx.1.22 6379
xx.xx.1.24 6379六、Redis Cluster IP&PORT文件
（如higo_bim：redis_cluster_ip_port.bim）
xx.xx.0.12 7121
xx.xx.0.66 7121
xx.xx.0.13 7121
xx.xx.0.71 7121
xx.xx.0.67 7121