推荐非常好用的redis数据迁移工具redis-shake,解决redis数据迁移常见的痛点问题:
1. 源库目标库版本不一致
2. 为实现平滑迁移,要求能做增量同步
3. 能快速实现多库批量同时迁移
4. 迁移速度非常快,基本实现秒级迁移
项目地址 https://github.com/alibaba/RedisShake
针对我的项目,要迁移多个业务服务器的同时,做业务相关redis数据迁移和增量同步。只需要为每个迁移任务新建单独的文件夹和迁移配置文件,然后分别执行。示例python代码如下:
def runShell(command):
print(command)
os.system(command)
def shake(server, output):
id = server[0]
ip = server[1]
redisId = server[2]
runShell('mkdir -p /home/csdn/redis-tool/redis-shake-{}'.format(id))
runShell('cp -rf /home/csdn/redis-tool/redis-shake /home/csdn/redis-tool/redis-shake-{}/redis-shake'.format(id))
runShell('cp -rf /home/csdn/redis-tool/redis-shake.conf /home/csdn/redis-tool/redis-shake-{}/redis-shake.conf'.format(id))
runShell("sed -i 's/source.address = 127.0.0.1:20441/source.address = {}:6379/g' /home/csdn/redis-tool/redis-shake-{}/redis-shake.conf".format(ip, id))
runShell("sed -i 's/target.address = 127.0.0.1:20551/target.address = {}.redis.rds.aliyuncs.com:6379/g' /home/csdn/redis-tool/redis-shake-{}/redis-shake.conf".format(redisId, id))
runShell("sed -i 's/system_profile = 9310/system_profile = {}/g' /home/csdn/redis-tool/redis-shake-{}/redis-shake.conf".format(9300 + int(id), id))
runShell("sed -i 's/http_profile = 9320/http_profile = {}/g' /home/csdn/redis-tool/redis-shake-{}/redis-shake.conf".format(9400 + int(id), id))
runShell("cd /home/csdn/redis-tool/redis-shake-{} && ./redis-shake -conf=./redis-shake.conf -type=sync > ./redis-shake.log".format(id, id, id))
output.put('redis sync {} fininshed'.format(id))
def shakeBatch(fromId, toId):
servers = sql.executeSQL('select a.id, a.inner_host, b.redis_id from server_list a, db_server b where a.id=b.db_id and a.id between {} and {}'.format(fromId, toId))
output = mp.Queue()
processes = [mp.Process(target=shake, args=(server, output)) for server in servers]
for p in processes:
p.start()
for p in processes:
p.join()
results = [output.get() for p in processes]