由于业务发展原因,原有memcache(以下均称mem)缓存负载过高,所以需要把默认数据缓存由mem改为redis,并且旧redis也要迁移
原有mem配置共计5台,权重均等.
//memcache 缓存
'cache' => array(
'class' => 'system.caching.CMemCache',
'useMemcached' => true,
'keyPrefix' => 'ifh_house',
'servers' => array(
array('host' => 'ip', 'port' => 11211, 'weight' => 20,),
array('host' => 'ip', 'port' => 11211, 'weight' => 20,),
array('host' => 'ip', 'port' => 11211, 'weight' => 20,),
array('host' => 'ip', 'port' => 11211, 'weight' => 20,),
array('host' => 'ip', 'port' => 11211, 'weight' => 20,),
),
),
旧redis配置3台cluster模式 3.2.6
ip1:7000 myself,master 0-5460
ip2:6379 slave
ip3:6379 master 5461-10922
ip1:7001
ip4:6379 master 10923-16383
ip1:7002 slave
新redis配置5台cluster模式5.0.7
ip1:7000@17000 master - 0 0-3276
ip2:7001@17001 slave
ip2:7000@17000 master 3277-6553
ip3:7001@17001 slave
ip3:7000@17000 master - 6554-9829
ip4:7001@17001 slave
ip4:7000@17000 myself,master 9830-13106
ip5:7001@17001 slave
ip5:7000@17000 master 13107-16383
ip1:7001@17001 slave
修改要求:
第一条也是最重要的肯定是不能影响现有业务访问
第二条就是尽量少改现有代码和逻辑
修改大致思路:
1.mem读的时候判断redis有无此数据,没有则写缓存统一10分钟(根据自己业务情况).
2.在线查看新redis请求情况(monitor)查看key写入量(keys *)总体信息(info)
3.上线一段时间后,把默认cache切换成redis.
4.回滚方案(配置文件临时注释,或新切分支,已备快速上线)
遇到问题:
1.yii 1.0默认CredisCache 不支持cluster模式,所以借用preids创建连接对象
'redis'];
/**
* @var Redis the Redis instance
*/
protected $_cache=null;
/**
* @var array list of servers
*/
protected $_servers=array();
/**
* @var string list of servers
*/
public $servers=array('host'=>'127.0.0.1','port'=>6379);
/**
* Initializes this application component.
* This method is required by the {@link IApplicationComponent} interface.
* It creates the redis instance and adds redis servers.
* @throws CException if redis extension is not loaded
*/
public function init()
{
parent::init();
$this->getRedis();
}
/**
* @return mixed the redis instance (or redisd if {@link useRedisd} is true) used by this component.
*/
public function getRedis()
{
if($this->_cache!==null)
return $this->_cache;
else{
require_once 'framework/plugins/predis/autoload.php';
Yii::log('Opening Redis connection',CLogger::LEVEL_TRACE);
return $this->_cache= new Predis\Client($this->servers, self::$options);
}
}
/**
* Retrieves a value from cache with a specified key.
* This is the implementation of the method declared in the parent class.
* @param string $key a unique key identifying the cached value
* @return string the value stored in cache, false if the value is not in the cache or expired.
*/
protected function getValue($key)
{
return $this->_cache->get($key);
}
/**
* Retrieves multiple values from cache with the specified keys.
* @param array $keys a list of keys identifying the cached values
* @return array a list of cached values indexed by the keys
* @since 1.0.8
*/
protected function getValues($keys)
{
if(!empty($keys) && is_array($keys)){
$result = [];
foreach ($keys as $key =>$val){
$result[$key] = $this->_cache->get($val);
}
return $result;
}
// return $this->_cache->mget($keys);
}
/**
* Stores a value identified by a key in cache.
* This is the implementation of the method declared in the parent class.
*
* @param string $key the key identifying the value to be cached
* @param string $value the value to be cached
* @param integer $expire the number of seconds in which the cached value will expire. 0 means never expire.
* @return boolean true if the value is successfully stored into cache, false otherwise
*/
protected function setValue($key,$value,$expire)
{
if($expire>0)
return $this->_cache->setex($key,$expire,$value);
else
return $this->_cache->set($key,$value);
}
/**
* Stores a value identified by a key into cache if the cache does not contain this key.
* This is the implementation of the method declared in the parent class.
*
* @param string $key the key identifying the value to be cached
* @param string $value the value to be cached
* @param integer $expire the number of seconds in which the cached value will expire. 0 means never expire.
* @return boolean true if the value is successfully stored into cache, false otherwise
*/
protected function addValue($key,$value,$expire)
{
if($expire>0){
if($this->_cache->setnx($key,$expire,$value))
return $this->_cache->expire($key,$expire);
return false;
}else
return $this->_cache->setnx($key,$value);
}
/**
* Deletes a value with the specified key from cache
* This is the implementation of the method declared in the parent class.
* @param string $key the key of the value to be deleted
* @return boolean if no error happens during deletion
*/
protected function deleteValue($key)
{
return $this->_cache->del($key);
}
/**
* Deletes all values from cache.
* This is the implementation of the method declared in the parent class.
* @return boolean whether the flush operation was successful.
* @since 1.1.5
*/
protected function flushValues()
{
return $this->_cache->flush();
}
/**
* call unusual method
* */
public function __call($method,$args){
return call_user_func_array(array($this->_cache,$method),$args);
}
/**
* Returns whether there is a cache entry with a specified key.
* This method is required by the interface ArrayAccess.
* @param string $id a key identifying the cached value
* @return boolean
*/
public function offsetExists($id)
{
return $this->_cache->exists($id);
}
}
2.yii 1.0 利用predis写入数据时会影响旧的redis使用
public function get($id)
{
$value = $this->getValue($this->generateUniqueKey($id));
/*=========================调试信息start==============================*/
//key保持与memecache一致
/* $tempKey = md5($this->keyPrefix.$id);
//存储 redis 10分钟
$redisVal = RedisUtil::get($tempKey,$this->redisServers);
if($redisVal == false){
$res = RedisUtil::setex($tempKey,$value,600,$this->redisServers);
}*/
/*==========================调试信息end===============================*/
if($value===false || $this->serializer===false)
return $value;
if($this->serializer===null)
$value=unserialize($value);
else
$value=call_user_func($this->serializer[1], $value);
if(is_array($value) && (!$value[1] instanceof ICacheDependency || !$value[1]->getHasChanged()))
{
Yii::trace('Serving "'.$id.'" from cache','system.caching.'.get_class($this));
return $value[0];
}
else
return false;
}
RedisUtil类实例化时,会影响旧的redis,RedisUtil代码如下
class RedisUtil {
public static $options = ['cluster' => 'redis'];
public static function get($key,$servers){
$client = self::initClient($servers);
return $client->get($key);
}
private static function initClient($servers){
static $predis = null;//防止重复建立连接
if($predis == null){
$predis = new Predis\Client($servers, self::$options);
}
return $predis;
}
}
到此迁移代码基本搞定,接下来开始准备迁移redis数据,经过查找找到一个很好用的工具redis-migrate-tool github地址:https://github.com/tanruixing88/redis-migrate-tool注意:这个地址的工具支持由低版本到高版本迁移.
git clone https://github.com/tanruixing88/redis-migrate-tool.git
cd redis-migrate-tool
autoreconf -fvi
./configure
make
src/redis-migrate-tool -h
rmt.conf 内容如下 (从一个redis迁移到另一个redis节点):
[source]
type: redis cluster
servers :
-ip1:7000
-ip2:6379
-ip3:6379
[target]
type: redis cluster
servers:
-ip1:7000
-ip2:7000
-ip3:7000
-ip4:7000
-ip5:7000
[common]
listen: 0.0.0.0:8888
迁移命令
/rec/redis-migrate-tool/src/redis-migrate-tool -c …/rmt.conf -o log -d
会用到的其他redis命令:
[house@houseCache_58 ~]$ redis-cli -p 7000 cluster nodes //查看redis节点信息
redis-cli -h ip地址 -p 7000 //链接redis
monitor 查看请求信息
key * 查看所有key
info 查看基本信息
这个进程会在后台一直运行,等数据追平后,让业务方在低峰期将老的redis库停止写入,将连接方式改到这个新库上即可(建议redis主从用sentinel来做高可用)。 确认没问题后,然后由DBA kill 掉 redis-migrate-tool进程,整个redis迁移全过程结束
到此redis数据迁移完成,代码切换完成并上线后杀掉8888 进程即可.