memcached源码学习-items操作

        今天主要总结items相关的操作,items的操作分布比较多,定义和实现在memcachd.h/c、thread.h/c、items.h/c都有,感觉完全可以放在items.h/c中。这里就对所有的这些操作(除去stats部分)进行一个简单的总结。

        首先对数据结构、ITEM_*和一些变量进行一个简单的说明,这里先建立一个宏观的概念,理解了它们的用途对后续阅读程序有很大的帮助。

/**
 * Structure for storing items within memcached.
 */
typedef struct _stritem {
    struct _stritem *next;      // next, prev在LRU队列中使用
    struct _stritem *prev;
    struct _stritem *h_next;    /* hash chain next */     // 用于hashtable中
    rel_time_t      time;       /* least recent access */ // 最近访问时间
    rel_time_t      exptime;    /* expire time */         // 过期时间
    int             nbytes;     /* size of data */        // 数据的长度(数据格式等下面介绍)
    unsigned short  refcount;   // 引用计数
    uint8_t         nsuffix;    /* length of flags-and-length string */
    uint8_t         it_flags;   /* ITEM_* above */
    uint8_t         slabs_clsid;/* which slab class we're in */
    uint8_t         nkey;       /* key length, w/terminating null and padding */
    /* this odd type prevents type-punning issues when we do
     * the little shuffle to save space when not using CAS. */
    union {
        uint64_t cas;
        char end;
    } data[];
    /* if it_flags & ITEM_CAS we have 8 bytes CAS */
    /* then null-terminated key */
    /* then " flags length\r\n" (no terminating null) */
    /* then data with terminating \r\n (no terminating null; it's binary!) */
} item;

        从上面的定义可以知道data字段的格式如下:

               if it_flags & ITEM_CAS, 8bytes      # 如果ITEM_CAS标志设置时,这里有8字节的数据

               key<null>                                       # 键值字符串,以null结尾,nkey长度不包括null字符

               flags-length\r\n                               # flags-and-length字符串,以\r\n结尾,nsuffix长度包括\r\n

               value\r\n                                         # 数据域,以\r\n结尾,nbytes长度包括\r\n

         知道了具体的data字段的格式,ITEM_*宏就很容易理解了,主要考虑ITEM_CAS标志设置时,后面的key,flags-length及value字段的存取都需要后移8字节。

// 获取cas值,即当ITEM_CAS标志设置时,返回8字节的cas值,否则返回0
#define ITEM_get_cas(i) (((i)->it_flags & ITEM_CAS) ? \
        (i)->data->cas : (uint64_t)0)

// 设置cas值,同上类似
#define ITEM_set_cas(i,v) { \
    if ((i)->it_flags & ITEM_CAS) { \
        (i)->data->cas = v; \
    } \
}
// 获得key值,如果ITEM_CAS标志设置时,需要后移8字节
#define ITEM_key(item) (((char*)&((item)->data)) \
         + (((item)->it_flags & ITEM_CAS) ? sizeof(uint64_t) : 0))

// 获得suffix字段,同上类似
#define ITEM_suffix(item) ((char*) &((item)->data) + (item)->nkey + 1 \
         + (((item)->it_flags & ITEM_CAS) ? sizeof(uint64_t) : 0))
// 获得数据域
#define ITEM_data(item) ((char*) &((item)->data) + (item)->nkey + 1 \
         + (item)->nsuffix \
         + (((item)->it_flags & ITEM_CAS) ? sizeof(uint64_t) : 0))

// 获得总结构的大小
#define ITEM_ntotal(item) (sizeof(struct _stritem) + (item)->nkey + 1 \
         + (item)->nsuffix + (item)->nbytes \
         + (((item)->it_flags & ITEM_CAS) ? sizeof(uint64_t) : 0))

        下面3个宏,定义了item的状态:

#define ITEM_LINKED 1
#define ITEM_CAS 2
#define ITEM_SLABBED 4

        ITEM_LINKED表示item处于hashtable和LRU队列中,分别在do_item_link(将item放入hashtable和LRU队列)和do_item_unlink(将item从hashtable和LRU队列移除)中设置和置位,后续的对it_flags &ITEM_LINKED的判断,根据是否在hashtable和LRU队列中,从而能够进行或禁止某些操作,如:如果item在hashtable和LRU队列,就不能执行free_item操作(assert((it->it_flags & ITEM_LINKED) == 0)保证),必须首先do_item_unlink。

       Item_CAS是根据用户命令行参数决定的,CAS表示check and set,只有cas字段一致时(check)才执行后续操作(set),否则操作失败。

        ITEM_SLABBED,这个标志只有在item_free函数中,slabs_free函数调用前设置,说明只有将其放入slabs对应的slabclass_t的freelist指针数组中时,才设置此标志,即设置此标志的item应该为free状态,it_flags & ITEM_SLABBED即可以以此来理解。

        下面说明LRU队列,主要包括heads和tails,sizes数组只是为了stats时使用,记录了每个对应的heads中item的数量:

#define LARGEST_ID POWER_LARGEST 
static item *heads[LARGEST_ID];         // LRU head buckets, 每个与slabclass对应,双向链表(每次插入放最前面)
static item *tails[LARGEST_ID];         // LRU tail buckets, 双向链表,记录了LRU队列的最后一个元素(oldest)

        slabclass定义为:

#define MAX_NUMBER_OF_SLAB_CLASSES (POWER_LARGEST + 1)
static slabclass_t slabclass[MAX_NUMBER_OF_SLAB_CLASSES];   // slabclass数组

        感觉 #define LARGEST_ID POWER_LARGEST  改为    #define LARGEST_ID POWER_LARGEST+1 更合理一些,这样heads就与slabclass一一对应起来了,虽然slabclass及heads并不是每个位置都使用了。因为slabclass[0]并没有使用,所以heads[0]和tails[0]也没有使用(使用方法heads[slabs_clsid],slabs_clsid非0)。
       

        全局设置setting中存在一个oldest_live的字段,表示在oldest_live时间之前的数据失效,比如flush_all操作(memcached的文本协议),设置oldest_live为当前时间,这样之前的数据都失效了,LRU算法执行时,就可以复用这些item了,这里称它为 截止时间

        还有就是对memcached中引用计数的理解,只有do_item_get(或do_item_get_nocheck)时引用计数++,do_item_remove时引用计数--,其它地方不涉及具体的操作。

        只有item_link_q和item_unlink_q(名字后面有个_q)两个函数才操作heads和tails数组,完成对LRU队列的维护。


        下面贴出代码,并进行相应的注释。

/**
 * Generates the variable-sized part of the header for an object.
 *
 * key     - The key
 * nkey    - The length of the key(include the null terminator)
 * flags   - key flags
 * nbytes  - Number of bytes to hold value and addition CRLF terminator
 * suffix  - Buffer for the "VALUE" line suffix (flags, size).
 * nsuffix - The length of the suffix is stored here.
 *
 * Returns the total size of the header.
 */
static size_t item_make_header(const uint8_t nkey, const int flags, const int nbytes,
                     char *suffix, uint8_t *nsuffix) {
    /* suffix is defined at 40 chars elsewhere.. */
    *nsuffix = (uint8_t) snprintf(suffix, 40, " %d %d\r\n", flags, nbytes - 2);
    return sizeof(item) + nkey + *nsuffix + nbytes;
}

/* 分配一个item结构 */
item *do_item_alloc(char *key, const size_t nkey, const int flags, const rel_time_t exptime, const int nbytes) {
    uint8_t nsuffix;
    item *it = NULL;
    char suffix[40];
    size_t ntotal = item_make_header(nkey + 1, flags, nbytes, suffix, &nsuffix);
    if (settings.use_cas) {
        ntotal += sizeof(uint64_t);         // if( (it_flags & ITEM_CAS)!=0), 8bytes CAS
    }

    unsigned int id = slabs_clsid(ntotal);  // 返回对应的slab的clsid
    if (id == 0)                            // 查找失败
        return 0;

    /* do a quick check if we have any expired items in the tail.. */
    int tries = 50;
    item *search;
    rel_time_t oldest_live = settings.oldest_live;

    for (search = tails[id];
         tries > 0 && search != NULL;
         tries--, search=search->prev) {
        if (search->refcount == 0 &&            // 引用计数为0
            ((search->time < oldest_live) ||    // dead by flush,比oldest_live旧的已失效
             (search->exptime != 0 && search->exptime < current_time))) { // 过期
            it = search;
            /* I don't want to actually free the object, just steal
             * the item to avoid to grab the slab mutex twice ;-)
             */
            STATS_LOCK();
            stats.reclaimed++;
            STATS_UNLOCK();
            itemstats[id].reclaimed++;
            it->refcount = 1;           // 后面会重用此item
            
            // 调整old item与新的item之间的差值,ITEM_ntotal(it)与ntotal计算方式相同
            slabs_adjust_mem_requested(it->slabs_clsid, ITEM_ntotal(it), ntotal);
            
            do_item_unlink(it);                 // 从hashtable和LRU队列摘除item
            /* Initialize the item block: */
            it->slabs_clsid = 0;
            it->refcount = 0;
            break;
        }
    }
 
    // 上面查找过程没有找到,则首先分配一个,如果分配失败,则通过LRU算法尝试摘除一些item
    // slabs_alloc:从slabclass[id]中分配一个size大小的trunk,错误时返回NULL(0)
    if (it == NULL && (it = slabs_alloc(ntotal, id)) == NULL) {
        /*
        ** Could not find an expired item at the tail, and memory allocation
        ** failed. Try to evict some items!
        */

        /* If requested to not push old items out of cache when memory runs out,
         * we're out of luck at this point...
         */

        // 通过M指定不使用LRU算法淘汰old item
        // -M: return error on memory exhausted (rather than removing items)
        if (settings.evict_to_free == 0) {
            itemstats[id].outofmemory++;
            return NULL;
        }

        /*
         * try to get one off the right LRU
         * don't necessarily unlink the tail because it may be locked: refcount>0
         * search up from tail an item with refcount==0 and unlink it; give up after 50
         * tries
         */

        if (tails[id] == 0) {
            itemstats[id].outofmemory++;
            return NULL;
        }
        tries = 50; // 最多尝试50次
        for (search = tails[id]; tries > 0 && search != NULL; tries--, search=search->prev) 
        {
            if (search->refcount == 0) // 引用计数为0
            {
                if (search->exptime == 0 || search->exptime > current_time) 
                {
                    itemstats[id].evicted++;
                    itemstats[id].evicted_time = current_time - search->time;
                    if (search->exptime != 0)
                        itemstats[id].evicted_nonzero++;
                    STATS_LOCK();
                    stats.evictions++;
                    STATS_UNLOCK();
                } 
                else 
                {
                    itemstats[id].reclaimed++;
                    STATS_LOCK();
                    stats.reclaimed++;
                    STATS_UNLOCK();
                }
                do_item_unlink(search); // 从hashtable及LRU队列摘除item项
                
                break;                  // 找到第一个引用计数为0的就结束此循环
            }
        }
        // 这里重新分配,如果上面循环回收了item结构,这里能够分配成功
        // 否则,分配会失败,然后强制回收时间锁定(refcount>0)时间过长(超过3小时)的item
        it = slabs_alloc(ntotal, id);   
        if (it == 0) 
        {
            itemstats[id].outofmemory++;
            /* Last ditch effort. There is a very rare bug which causes
             * refcount leaks. We've fixed most of them, but it still happens,
             * and it may happen in the future.
             * We can reasonably assume no item can stay locked for more than
             * three hours, so if we find one in the tail which is that old,
             * free it anyway.
             */
            tries = 50;
            for (search = tails[id]; tries > 0 && search != NULL; tries--, search=search->prev) 
            {
                if (search->refcount != 0 && search->time + TAIL_REPAIR_TIME < current_time) 
                {
                    itemstats[id].tailrepairs++;
                    search->refcount = 0;
                    do_item_unlink(search);
                    break;
                }
            }
            it = slabs_alloc(ntotal, id);
            if (it == 0) {
                return NULL;
            }
        }
    }

    assert(it->slabs_clsid == 0);

    it->slabs_clsid = id;

    assert(it != heads[it->slabs_clsid]);

    it->next = it->prev = it->h_next = 0;
    it->refcount = 1;     /* the caller will have a reference */
    DEBUG_REFCNT(it, '*');
    it->it_flags = settings.use_cas ? ITEM_CAS : 0; //? 0
    it->nkey = nkey;                                // key由null结尾,nkey长度不包括null字符
    it->nbytes = nbytes;                            // value由\r\n结尾,nbytes长度包括\r\n
    memcpy(ITEM_key(it), key, nkey);
    it->exptime = exptime;
    memcpy(ITEM_suffix(it), suffix, (size_t)nsuffix);
    it->nsuffix = nsuffix;                          // suffix由\r\n结尾,nsuffix长度包括\r\n
    return it;
}

void item_free(item *it) {
    size_t ntotal = ITEM_ntotal(it);
    unsigned int clsid;
    assert((it->it_flags & ITEM_LINKED) == 0);
    assert(it != heads[it->slabs_clsid]);
    assert(it != tails[it->slabs_clsid]);
    assert(it->refcount == 0);

    /* so slab size changer can tell later if item is already free or not */
    clsid = it->slabs_clsid;
    it->slabs_clsid = 0;
    it->it_flags |= ITEM_SLABBED;
    DEBUG_REFCNT(it, 'F');
    // 将ptr指向的item结构放入slabclass[clsid]的slabclass_t的freelist数组中
    slabs_free(it, ntotal, clsid);
}

/**
 * Returns true if an item will fit in the cache (its size does not exceed
 * the maximum for a cache entry.)
 */
bool item_size_ok(const size_t nkey, const int flags, const int nbytes) {
    char prefix[40];
    uint8_t nsuffix;

    size_t ntotal = item_make_header(nkey + 1, flags, nbytes,
                                     prefix, &nsuffix);
    if (settings.use_cas) {
        ntotal += sizeof(uint64_t);
    }

    return slabs_clsid(ntotal) != 0;
}

// 把该item插入LRU队列
static void item_link_q(item *it) { /* item is the new head */
    item **head, **tail;
    assert(it->slabs_clsid < LARGEST_ID);
    assert((it->it_flags & ITEM_SLABBED) == 0);

    head = &heads[it->slabs_clsid];
    tail = &tails[it->slabs_clsid];
    assert(it != *head);
    assert((*head && *tail) || (*head == 0 && *tail == 0));
    it->prev = 0;
    it->next = *head;
    if (it->next) it->next->prev = it;
    *head = it;
    if (*tail == 0) *tail = it;
    sizes[it->slabs_clsid]++;
    return;
}

// 从LRU队列删除此item
static void item_unlink_q(item *it) {
    item **head, **tail;
    assert(it->slabs_clsid < LARGEST_ID);
    head = &heads[it->slabs_clsid];
    tail = &tails[it->slabs_clsid];
    // 头部
    if (*head == it) {
        assert(it->prev == 0);
        *head = it->next;
    }
    // 尾部
    if (*tail == it) {
        assert(it->next == 0);
        *tail = it->prev;
    }
    assert(it->next != it);
    assert(it->prev != it);

    // 改变双向链表指针
    if (it->next) it->next->prev = it->prev;
    if (it->prev) it->prev->next = it->next;
    sizes[it->slabs_clsid]--;
    return;
}

// 将item放入hashtable和LRU队列
int do_item_link(item *it) {
    MEMCACHED_ITEM_LINK(ITEM_key(it), it->nkey, it->nbytes);
    assert((it->it_flags & (ITEM_LINKED|ITEM_SLABBED)) == 0);
    it->it_flags |= ITEM_LINKED;    // 设置ITEM_LINKED标志
    it->time = current_time;        // 最近访问时间为当前时间
    assoc_insert(it);               // 将item指针插入hash表中

    STATS_LOCK();
    stats.curr_bytes += ITEM_ntotal(it);
    stats.curr_items += 1;
    stats.total_items += 1;
    STATS_UNLOCK();

    /* Allocate a new CAS ID on link. */
    ITEM_set_cas(it, (settings.use_cas) ? get_cas_id() : 0);// 调用get_cas_id()给item的cas_id赋值

    // 把该item插入LRU队列(heads, tails, sizes)
    item_link_q(it);

    return 1;
}

// 从hashtable及LRU队列摘除item项
void do_item_unlink(item *it) {
    MEMCACHED_ITEM_UNLINK(ITEM_key(it), it->nkey, it->nbytes);
    if ((it->it_flags & ITEM_LINKED) != 0) 
    {
        it->it_flags &= ~ITEM_LINKED;
        STATS_LOCK();
        stats.curr_bytes -= ITEM_ntotal(it);
        stats.curr_items -= 1;
        STATS_UNLOCK();
        assoc_delete(ITEM_key(it), it->nkey);// 从hashtable中删除key对应的item
        item_unlink_q(it);                   // 从LRU队列删除此item
        if (it->refcount == 0)               // 引用计数为1时,删除此item
            item_free(it);
    }
}
// 减少引用计数refcount,引用计数为0的时候,就将其释放
void do_item_remove(item *it) 
{
    MEMCACHED_ITEM_REMOVE(ITEM_key(it), it->nkey, it->nbytes);
    assert((it->it_flags & ITEM_SLABBED) == 0);
    if (it->refcount != 0) 
    {
        it->refcount--;         // 减少引用计数
        DEBUG_REFCNT(it, '-');
    }
    if (it->refcount == 0 && (it->it_flags & ITEM_LINKED) == 0) 
    {
        item_free(it);
    }
}
// 更新item时间戳
// 先调用item_unlink_q(),更新了时间以后,再调用item_link_q(),
// 将其重新连接到LRU队列之中,即让该item移到LRU队列的最前
void do_item_update(item *it) {
    MEMCACHED_ITEM_UPDATE(ITEM_key(it), it->nkey, it->nbytes);
    if (it->time < current_time - ITEM_UPDATE_INTERVAL) {
        assert((it->it_flags & ITEM_SLABBED) == 0);

        if ((it->it_flags & ITEM_LINKED) != 0) {
            item_unlink_q(it);
            it->time = current_time;
            item_link_q(it);
        }
    }
}
// 调用do_item_unlink()解除原有it的连接,再调用do_item_link()连接到新的new_it
int do_item_replace(item *it, item *new_it) {
    MEMCACHED_ITEM_REPLACE(ITEM_key(it), it->nkey, it->nbytes,
                           ITEM_key(new_it), new_it->nkey, new_it->nbytes);
    assert((it->it_flags & ITEM_SLABBED) == 0);

    do_item_unlink(it);             // 从hashtable及LRU队列摘除it
    return do_item_link(new_it);    // 将new_it插入hashtable和LRU队列
}
/** wrapper around assoc_find which does the lazy expiration logic */
item *do_item_get(const char *key, const size_t nkey) {
    item *it = assoc_find(key, nkey);   // 从hashtable查找key
    int was_found = 0;

    if (settings.verbose > 2) {
        if (it == NULL) {
            fprintf(stderr, "> NOT FOUND %s", key);
        } else {
            fprintf(stderr, "> FOUND KEY %s", ITEM_key(it));
            was_found++;
        }
    }
    // 命中,且item存取时间比截止时间早,已失效
    if (it != NULL && settings.oldest_live != 0 && settings.oldest_live <= current_time &&
        it->time <= settings.oldest_live) {
        do_item_unlink(it);           /* MTSAFE - cache_lock held */
        it = NULL;
    }

    if (it == NULL && was_found) {
        fprintf(stderr, " -nuked by flush");
        was_found--;
    }

    // 命中,且item已过期
    if (it != NULL && it->exptime != 0 && it->exptime <= current_time) {
        do_item_unlink(it);           /* MTSAFE - cache_lock held */
        it = NULL;
    }

    if (it == NULL && was_found) {
        fprintf(stderr, " -nuked by expire");
        was_found--;
    }

    if (it != NULL) {
        it->refcount++;
        DEBUG_REFCNT(it, '+');
    }

    if (settings.verbose > 2)
        fprintf(stderr, "\n");

    return it;
}

/** returns an item whether or not it's expired. */
item *do_item_get_nocheck(const char *key, const size_t nkey) {
    item *it = assoc_find(key, nkey);
    if (it) {
        it->refcount++;
        DEBUG_REFCNT(it, '+');
    }
    return it;
}

/* expires items that are more recent than the oldest_live setting. */// 执行flush操作,即将所有当前有效的item清空(flush)
 void do_item_flush_expired(void) {
    int i;
    item *iter, *next;
    if (settings.oldest_live == 0)
        return;
    for (i = 0; i < LARGEST_ID; i++) {
        /* The LRU is sorted in decreasing time order, and an item's timestamp
         * is never newer than its last access time, so we only need to walk
         * back until we hit an item older than the oldest_live time.
         * The oldest_live checking will auto-expire the remaining items.
         */
        for (iter = heads[i]; iter != NULL; iter = next) {
            if (iter->time >= settings.oldest_live) {     // 比截止时间新的数据
                next = iter->next;
                if ((iter->it_flags & ITEM_SLABBED) == 0) {
                    do_item_unlink(iter);
                }
            } else {  
               // 比截止时间旧的数据已经在之前回收了,
// 由于是按照时间降序排列的(越新越靠前),所以碰到第一个older的数据就停止 /* We've hit the first old item. Continue to the next queue. */ break; } } }}

下面的函数都是上层调用的,主要与memcached的协议关系比较紧密,同时保证了临界资源的互斥访问(cache_lock),最好提前熟悉一下memcached的文本协议。

/*
 * Allocates a new item.
 */
// 新分配一个item结构
item *item_alloc(char *key, size_t nkey, int flags, rel_time_t exptime, int nbytes) {
    item *it;
    pthread_mutex_lock(&cache_lock);

    // do_item_alloc:新分配一个item结构,首先快速扫描是否存在过期,存在则复用item结构,
    // 如果没有,就新分配一个,如果分配失败,则使用LRU算法进行回收
    it = do_item_alloc(key, nkey, flags, exptime, nbytes);
    
    pthread_mutex_unlock(&cache_lock);
    return it;
}

/*
 * Returns an item if it hasn't been marked as expired,
 * lazy-expiring as needed.
 */
 // 查询key值的item
item *item_get(const char *key, const size_t nkey) {
    item *it;
    pthread_mutex_lock(&cache_lock);
    it = do_item_get(key, nkey);      // 查询item,如果成功,这里会增加引用计数
    pthread_mutex_unlock(&cache_lock);
    return it;
}
/*
 * Decrements the reference count on an item and adds it to the freelist if
 * needed.
 */
void item_remove(item *item) {
    pthread_mutex_lock(&cache_lock);
    do_item_remove(item);           // 减少item的引用计数,如果降为0,则释放
    pthread_mutex_unlock(&cache_lock);
}

/*
 * Links an item into the LRU and hashtable.
 */
// 将item放入hashtable和LRU队列
int item_link(item *item) {
    int ret;

    pthread_mutex_lock(&cache_lock);
    ret = do_item_link(item);
    pthread_mutex_unlock(&cache_lock);
    return ret;
}
/*
 * Unlinks an item from the LRU and hashtable.
 */
// 从hashtable和LRU队列去除item
void item_unlink(item *item) {
    pthread_mutex_lock(&cache_lock);
    do_item_unlink(item);
    pthread_mutex_unlock(&cache_lock);
}

/*
 * Replaces one item with another in the hashtable.
 * Unprotected by a mutex lock since the core server does not require
 * it to be thread-safe.
 */
int item_replace(item *old_it, item *new_it) {
    // 替换,首先将old_it从hashtable和LRU删除,再讲new_it加入hashtable和LRU
    return do_item_replace(old_it, new_it);
}

/*
 * Moves an item to the back of the LRU queue.
 */
void item_update(item *item) {
    pthread_mutex_lock(&cache_lock);
    do_item_update(item);
    pthread_mutex_unlock(&cache_lock);
}

/*
 * Stores an item in the cache (high level, obeys set/add/replace semantics)
 */
enum store_item_type store_item(item *item, int comm, conn* c) {
    enum store_item_type ret;

    pthread_mutex_lock(&cache_lock);
    ret = do_store_item(item, comm, c);
    pthread_mutex_unlock(&cache_lock);
    return ret;
}

/*
 * Stores an item in the cache according to the semantics of one of the set
 * commands. In threaded mode, this is protected by the cache lock.
 *
 * Returns the state of storage.
 */
enum store_item_type do_store_item(item *it, int comm, conn *c) {
    char *key = ITEM_key(it);
    item *old_it = do_item_get(key, it->nkey);  // 查询key值的item
    enum store_item_type stored = NOT_STORED;

    item *new_it = NULL;
    int flags;

    if (old_it != NULL && comm == NREAD_ADD) { // 已经存在
        /* add only adds a nonexistent item, but promote to head of LRU */
        do_item_update(old_it);
    } else if (!old_it && (comm == NREAD_REPLACE
        || comm == NREAD_APPEND || comm == NREAD_PREPEND))
    {
        /* replace only replaces an existing value; don't store */
    } else if (comm == NREAD_CAS) {
        /* validate cas operation */
        if(old_it == NULL) {
            // LRU expired
            stored = NOT_FOUND;
            pthread_mutex_lock(&c->thread->stats.mutex);
            c->thread->stats.cas_misses++;
            pthread_mutex_unlock(&c->thread->stats.mutex);
        }
        else if (ITEM_get_cas(it) == ITEM_get_cas(old_it)) {// cas(check and set)一致时,才操作
            // cas validates
            // it and old_it may belong to different classes.
            // I'm updating the stats for the one that's getting pushed out
            pthread_mutex_lock(&c->thread->stats.mutex);
            c->thread->stats.slab_stats[old_it->slabs_clsid].cas_hits++;
            pthread_mutex_unlock(&c->thread->stats.mutex);

            item_replace(old_it, it);
            stored = STORED;
        } else {
            pthread_mutex_lock(&c->thread->stats.mutex);
            c->thread->stats.slab_stats[old_it->slabs_clsid].cas_badval++;
            pthread_mutex_unlock(&c->thread->stats.mutex);

            if(settings.verbose > 1) {
                fprintf(stderr, "CAS:  failure: expected %llu, got %llu\n",
                        (unsigned long long)ITEM_get_cas(old_it),
                        (unsigned long long)ITEM_get_cas(it));
            }
            stored = EXISTS;
        }
    } else {
        /*
         * Append - combine new and old record into single one. Here it's
         * atomic and thread-safe.
         */
        if (comm == NREAD_APPEND || comm == NREAD_PREPEND) {
            /*
             * Validate CAS
             */
            if (ITEM_get_cas(it) != 0) {
                // CAS much be equal
                if (ITEM_get_cas(it) != ITEM_get_cas(old_it)) {
                    stored = EXISTS;
                }
            }

            if (stored == NOT_STORED) {
                /* we have it and old_it here - alloc memory to hold both */
                /* flags was already lost - so recover them from ITEM_suffix(it) */

                flags = (int) strtol(ITEM_suffix(old_it), (char **) NULL, 10);

                // 新建一个item
                new_it = do_item_alloc(key, it->nkey, flags, old_it->exptime, it->nbytes + old_it->nbytes - 2 /* CRLF */);

                if (new_it == NULL) {
                    /* SERVER_ERROR out of memory */
                    if (old_it != NULL)
                        do_item_remove(old_it);

                    return NOT_STORED;
                }

                /* copy data from it and old_it to new_it */

                if (comm == NREAD_APPEND) {
                    memcpy(ITEM_data(new_it), ITEM_data(old_it), old_it->nbytes);
                    memcpy(ITEM_data(new_it) + old_it->nbytes - 2 /* CRLF */, ITEM_data(it), it->nbytes);
                } else {
                    /* NREAD_PREPEND */
                    memcpy(ITEM_data(new_it), ITEM_data(it), it->nbytes);
                    memcpy(ITEM_data(new_it) + it->nbytes - 2 /* CRLF */, ITEM_data(old_it), old_it->nbytes);
                }

                it = new_it;
            }
        }

        if (stored == NOT_STORED) {
            if (old_it != NULL)
                item_replace(old_it, it);
            else
                do_item_link(it);

            c->cas = ITEM_get_cas(it);

            stored = STORED;
        }
    }

    if (old_it != NULL)
        do_item_remove(old_it);         /* release our reference */
    if (new_it != NULL)
        do_item_remove(new_it);

    if (stored == STORED) {
        c->cas = ITEM_get_cas(it);
    }

    return stored;
}

/*
 * Flushes expired items after a flush_all call
 */
// 执行flush_all操作,清空所有items
void item_flush_expired() {
    pthread_mutex_lock(&cache_lock);
    do_item_flush_expired();
    pthread_mutex_unlock(&cache_lock);
}

总结:感觉items相关的操作并没有很高的复用价值,这个主要是理解slab和hashtable的接口及使用方法,设计的算法和数据结构并不多,本身逻辑也并不是很复杂。


你可能感兴趣的:(struct,cache,memcached,null,search,reference)