redisObject结构包含一个lru属性,记录对象被程序访问的情况。
该属性有两种使用方式:
- LRU:Least Recently Used,按最近的访问时间淘汰;
- LFU:least frequntly used,按最近的访问频率淘汰;
// server.h
#define LFU_INIT_VAL 5
// object.c
/* Set the LRU to the current lruclock (minutes resolution), or
* alternatively the LFU counter. */
if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {
o->lru = (LFUGetTimeInMinutes()<<8) | LFU_INIT_VAL;
} else {
o->lru = LRU_CLOCK();
}
可以从代码中看到,方式的选择由maxmemory_policy配置决定。
LRU方式
// server.h
#define LRU_CLOCK_RESOLUTION 1000 /* LRU clock resolution in ms */
#define LRU_CLOCK_MAX ((1<lru */
/* Return the LRU clock, based on the clock resolution. This is a time
* in a reduced-bits format that can be used to set and check the
* object->lru field of redisObject structures. */
unsigned int getLRUClock(void) {
return (mstime()/LRU_CLOCK_RESOLUTION) & LRU_CLOCK_MAX;
}
#define atomicGet(var,dstvar) do { \
dstvar = atomic_load_explicit(&var,memory_order_relaxed); \
} while(0)
// evict.c
/* This function is used to obtain the current LRU clock.
* If the current resolution is lower than the frequency we refresh the
* LRU clock (as it should be in production servers) we return the
* precomputed value, otherwise we need to resort to a system call. */
unsigned int LRU_CLOCK(void) {
unsigned int lruclock;
if (1000/server.hz <= LRU_CLOCK_RESOLUTION) {
atomicGet(server.lruclock,lruclock);
} else {
lruclock = getLRUClock();
}
return lruclock;
}
// server.c
/* We have just LRU_BITS bits per object for LRU information.
* So we use an (eventually wrapping) LRU clock.
*
* Note that even if the counter wraps it's not a big problem,
* everything will still work but some object will appear younger
* to Redis. However for this to happen a given object should never be
* touched for all the time needed to the counter to wrap, which is
* not likely.
*
* Note that you can change the resolution altering the
* LRU_CLOCK_RESOLUTION define. */
unsigned int lruclock = getLRUClock();
atomicSet(server.lruclock,lruclock);
在 LRU 模式下, lru 字段存储的是Redis时钟server.lruclock。 这个原子变量是一个24bit 的整数,默认是 Unix 时间戳对 2^24 取模的结果,大约 97 天清零一次 。
当某个 key被访问一次,它的对象头的 lru值就会被更新为 server.lruclock。
根据前后时间的时间差,可以算出多久没有访问这个对象了。即使clock已经被清零过了,也可以进行计算。
// evict.c
/* Given an object returns the min number of milliseconds the object was never
* requested, using an approximated LRU algorithm. */
unsigned long long estimateObjectIdleTime(robj *o) {
unsigned long long lruclock = LRU_CLOCK();
if (lruclock >= o->lru) {
return (lruclock - o->lru) * LRU_CLOCK_RESOLUTION;
} else {
return (lruclock + (LRU_CLOCK_MAX - o->lru)) *
LRU_CLOCK_RESOLUTION;
}
}
下面命令可以查看对象的时间:
127.0.0.1:6379> set msg "hello world"
OK
127.0.0.1:6379> object idletime msg
(integer) 16
127.0.0.1:6379> object idletime msg
(integer) 109
LFU方式
在LFU模式下,lru字段用来存储两个值,分别是ldt(last decrement time)16bit和logc(logistic counter)8bit。
// db.c
/* Update LFU when an object is accessed.
* Firstly, decrement the counter if the decrement time is reached.
* Then logarithmically increment the counter, and update the access time. */
void updateLFU(robj *val) {
unsigned long counter = LFUDecrAndReturn(val);
counter = LFULogIncr(counter);
val->lru = (LFUGetTimeInMinutes()<<8) | counter;
}
ldt 是 16 个 bit,用来存储上一次 logc 的更新时间。因为只有16 个 bit,它取的是分钟时间戳对 2^16 进行取模,大约每隔 45 天就会清零一次。同 LRU 模式一样,我们也可以使用这个逻辑计算出对象的空闲时间,只不过精度是分钟级别的。
// evict.c
/* Return the current time in minutes, just taking the least significant
* 16 bits. The returned time is suitable to be stored as LDT (last decrement
* time) for the LFU implementation. */
unsigned long LFUGetTimeInMinutes(void) {
return (server.unixtime/60) & 65535;
}
logc 是 8 个 bit,用来存储访问频次,因为 8 个 bit 能表示的最大整数值为 255,存储频次肯定远远不够,所以这 8 个 bit 存储的是频次的对数值,并且这个值还会随时间衰减,如果它的值比较小,那么就很容易被回收。为了确保新创建的对象不被回收,新对象的这 8 个 bit 会被初始化为一个大于零的值 LFU_INIT_VAL(默认是=5) 。
// evict.c
/* Given an object last access time, compute the minimum number of minutes
* that elapsed since the last access. Handle overflow (ldt greater than
* the current 16 bits minutes time) considering the time as wrapping
* exactly once. */
unsigned long LFUTimeElapsed(unsigned long ldt) {
unsigned long now = LFUGetTimeInMinutes();
if (now >= ldt) return now-ldt;
return 65535-ldt+now;
}
#define RAND_MAX 0x7fff
/* Logarithmically increment a counter. The greater is the current counter value
* the less likely is that it gets really implemented. Saturate it at 255. */
uint8_t LFULogIncr(uint8_t counter) {
if (counter == 255) return 255;
double r = (double)rand()/RAND_MAX;
double baseval = counter - LFU_INIT_VAL;
if (baseval < 0) baseval = 0;
double p = 1.0/(baseval*server.lfu_log_factor+1);
if (r < p) counter++;
return counter;
}
/* If the object decrement time is reached decrement the LFU counter but
* do not update LFU fields of the object, we update the access time
* and counter in an explicit way when the object is really accessed.
* And we will times halve the counter according to the times of
* elapsed time than server.lfu_decay_time.
* Return the object frequency counter.
*
* This function is used in order to scan the dataset for the best object
* to fit: as we check for the candidate, we incrementally decrement the
* counter of the scanned objects if needed. */
unsigned long LFUDecrAndReturn(robj *o) {
unsigned long ldt = o->lru >> 8;
unsigned long counter = o->lru & 255;
// 计算经过了多少段server.lfu_decay_time
unsigned long num_periods = server.lfu_decay_time ? LFUTimeElapsed(ldt) / server.lfu_decay_time : 0;
if (num_periods)
counter = (num_periods > counter) ? 0 : counter - num_periods;
return counter;
}
从这里我们可以看出,每经过一段server.lfu_decay_time时间会导致logc的计数值减一,也就是频次减半。另外频次不是每次都能增加的,而是采用随机算法进行增加,频次越高,频次增加的概率就越低。
logc不是在对象被访问时更新的,而是在redis的淘汰逻辑进行时更新的,每次淘汰都是采用随机策略, 随机挑选若干个 key,更新这个 key 的“热度”,淘汰掉“热度”最低的 key。因为 Redis 采用的是随机算法,如果 key 比较多的话,那么 ldt 更新得可能会比较慢。不 过既然它是分钟级别的精度,也没有必要更新得过于频繁。
总而言之,就是通过logc和ldt相互结合,可以获得某个键的使用频率。
下面命令可以查看对象的频次:
127.0.0.1:6379> set lfustr "hello world"
OK
127.0.0.1:6379> object freq lfustr
(integer) 0
127.0.0.1:6379> get lfustr
"hello world"
127.0.0.1:6379> object freq lfustr
(integer) 1
当服务器打开了maxmemory选项,且服务器内存占用超过maxmemory时,空转时长较长的那些对象会优先被服务器释放。