dm-cache源码浅析

最近想学习Linux IO子系统, 找了flashcache代码, 它通过内核提供的Device Mapper机制, 将一块SSD和一块普通磁盘虚拟为一个块设备, 其中SSD作为cache, 数据最终落地到普通磁盘. 这种混合存储的策略, 向上层应用(如mysql)屏蔽了底层的实现, 上层应用看到的只是一个挂载到虚拟块设备上的某种文件系统, 使用常见的文件系统接口即可读写数据, 一方面保持兼容, 一方面获得不错的性能. flashcache的代码只有几千行, 从commit log中可以看到版本迭代比较频繁, 也因此引入了较多我个人不关心的新特性. flashcache源码中作者写到借鉴了dm-cache的代码, 所以查了下资料, 竟是国人出品, sloc不足两千, 一晚上就可以看完, 正合胃口. dm-cache的使用可以参考flashcache文档, 原理见flashcache原理.

内存结构

dm-cache思路非常简单, 它把SSD作为cache, 将数据持久化到普通磁盘. 其中, SSD cache组织方式为set-associative map, 这和CPU cache的组织非常相像, 只是这里的key是cacheblock编号. cacheblock是dm-cache为了方便存取数据引入的单位, 粒度在磁盘block之上. 在通过dmsetup创建dm-cache块设备时时可以指定cacheblock的大小, 默认为8个连续的磁盘block组成一个cacheblock, 即4k字节. 上层的IO请求由Device Mapper框架切割为cacheblock大小(且对齐)的bio, 然后交由dm-cache处理. 也就是说, 不管是对SSD, 还是普通磁盘, dm-cache处理IO的单位都是cacheblock. 它在内存中的metadata为:

117 /* Cache block metadata structure */
118 struct cacheblock {
119     spinlock_t lock;    /* Lock to protect operations on the bio list */
120     sector_t block;     /* Sector number of the cached block */
121     unsigned short state;  /* State of a block */
122     unsigned long counter; /* Logical timestamp of the block’s last access */
123     struct bio_list bios;  /* List of pending bios */
124 };

其中, block字段表示当前cacheblock的起始扇区编号. 既然SSD作为cache, 针对写请求必定会有writeback和writethrough等多种选择. writeback即数据先写到SSD, 然后由后台线程在合适的时间写回磁盘. writethrough指数据同时写入磁盘和SSD. (flashcache在这基础上又增加了writearound的方式, 意思是绕过SSD cache, 数据直接写入磁盘, 在处理读请求时更新到SSD.) 不管是writeback, 还是writethrough, 数据写入磁盘(或者由磁盘读取数据更新至cache)都不可能一蹴而就, 所以每个cacheblock必定会有一个状态(state字段). 另外, cache有淘汰的概念, dm-cache支持FIFO或LRU淘汰, 所以需要为每个cacheblock保存其最后访问时间(counter字段). 最后, 为了互斥同时请求同一个cacheblock, 每个cacheblock还对应一个spinlock. 被互斥的后发请求记录在bios链表中. 在当前cacheblock上的操作完成后, dm-cache将重新提交bios链表上的bio.

接下来看下dm-cache的总控结构体cache_c:

 80 /*
 81  * Cache context
 82  */
 83 struct cache_c {
 84     struct dm_dev *src_dev;        /* Source device */
 85     struct dm_dev *cache_dev;  /* Cache device */
 86     struct dm_kcopyd_client *kcp_client; /* Kcopyd client for writing back data */
 87 
 88     struct cacheblock *cache;  /* Hash table for cache blocks */
 89     sector_t size;          /* Cache size */
 90     unsigned int bits;     /* Cache size in bits */
 91     unsigned int assoc;        /* Cache associativity */
 92     unsigned int block_size;   /* Cache block size */
 93     unsigned int block_shift;  /* Cache block size in bits */
 94     unsigned int block_mask;   /* Cache block mask */
 95     unsigned int consecutive_shift;    /* Consecutive blocks size in bits */
 96     unsigned long counter;     /* Logical timestamp of last access */
 97     unsigned int write_policy; /* Cache write policy */
 98     sector_t dirty_blocks;      /* Number of dirty blocks */
 99 
100     spinlock_t lock;        /* Lock to protect page allocation/deallocation */
101     struct page_list *pages;   /* Pages for I/O */
102     unsigned int nr_pages;     /* Number of pages */
103     unsigned int nr_free_pages;    /* Number of free pages */
104     wait_queue_head_t destroyq; /* Wait queue for I/O completion */
105     atomic_t nr_jobs;       /* Number of I/O jobs */
106     struct dm_io_client *io_client;   /* Client memory pool*/
107 
108     /* Stats */
109     unsigned long reads;       /* Number of reads */
110     unsigned long writes;      /* Number of writes */
111     unsigned long cache_hits;  /* Number of cache hits */
112     unsigned long replace;     /* Number of cache replacements */
113     unsigned long writeback;   /* Number of replaced dirty blocks */
114     unsigned long dirty;       /* Number of submitted dirty blocks */
115 };

其中, src_dev和cache_dev分别为磁盘和SSD在DM框架的抽象. cache字段为连续的cacheblock数组, 元素个数即size字段. 其余字段顾名思义, 不再赘述.

初始化

dm-cache的初始化代码相对简单, DM框架获取dmsetup参数, 传递给cache_ctr(), dm-cache通过该函数构造一个cache_c对象, 保存在dm_target.private中. dm_target结构中另一重要字段为split_io, 这个字段表示DM框架分割bio的粒度, cache_ctr()函数指定其为cacheblock大小.

上层的读写请求在IO内核路径上表示为bio, 针对Device Mapper框架虚拟出来的块设备的bio请求, DM框架通过bio的block编号找到所属的dm_targets(一个bio的请求可能横跨多个dm_target), 逐个回调dm_target.type->map, 该字段为函数指针, 在dm-cache模块加载到内核时, 由该模块的初始化函数dm_cache_init()注册为cache_map(). 也就是说, 读写请求的入口都是cache_map().

请求处理

如上所述, 读写请求的入口都是cache_map(), 其实现如下:

1202 /*
1203  * Decide the mapping and perform necessary cache operations for a bio request.
1204  */
1205 static int cache_map(struct dm_target *ti, struct bio *bio,
1206               union map_info *map_context)
1207 {
1208     struct cache_c *dmc = (struct cache_c *) ti->private;
1209     sector_t request_block, cache_block = 0, offset;
1210     int res;
1211 
1212     offset = bio->bi_sector & dmc->block_mask;
1213     request_block = bio->bi_sector � offset;
1214 
1220     if (bio_data_dir(bio) == READ) dmc->reads++;
1221     else dmc->writes++;
1222 
1223     res = cache_lookup(dmc, request_block, &cache_block);
1224     if (1 == res)  /* Cache hit; server request from cache */
1225         return cache_hit(dmc, bio, cache_block);
1226     else if (0 == res) /* Cache miss; replacement block is found */
1227         return cache_miss(dmc, bio, cache_block);
1228     else if (2 == res) { /* Entire cache set is dirty; initiate a write-back */
1229         write_back(dmc, cache_block, 1);
1230         dmc->writeback++;
1231     }
1232 
1233     /* Forward to source device */
1234     bio->bi_bdev = dmc->src_dev->bdev;
1235 
1236     return 1;
1237 }

该函数首先从ti->private中获取cache_c *dmc, 这个对象由cache_ctr()中构造. 接着获得bio所请求的起始扇区(即bio->bi_sector)所属的cacheblock的扇区编号, 保存在request_block变量. 接着通过cache_lookup()函数在dmc->cache中查找, key便是request_block. cache_lookup()代码相对简单, 不再细述.

如果cache中查找失败, 则进入cache_miss()逻辑. 其最后一个参数cache_block为cache_lookup()以某种淘汰形式找到的待替换的cacheblock的扇区编号.

1189 /* Handle cache misses */
1190 static int cache_miss(struct cache_c *dmc, struct bio* bio, sector_t cache_block) {
1191     if (bio_data_dir(bio) == READ)
1192         return cache_read_miss(dmc, bio, cache_block);
1193     else
1194         return cache_write_miss(dmc, bio, cache_block);
1195 }

cache_miss()函数判断bio是读是写, 读则调用cache_read_miss(), 否则调用cache_write_miss().

篇幅所限, 接下来我们只看下读请求未命中cache的情况, 这时cache_read_miss()将被调用.

1073 /*
1074  * Handle a read cache miss:
1075  *  Update the metadata; fetch the necessary block from source device;
1076  *  store data to cache device.
1077  */
1078 static int cache_read_miss(struct cache_c *dmc, struct bio* bio,
1079                            sector_t cache_block) {
1080     struct cacheblock *cache = dmc->cache;
1081     unsigned int offset, head, tail;
1082     struct kcached_job *job;
1083     sector_t request_block, left;
1084 
1085     offset = (unsigned int)(bio->bi_sector & dmc->block_mask);
1086     request_block = bio->bi_sector � offset;
1087 
1095     cache_insert(dmc, request_block, cache_block); /* Update metadata first */
1096 
1097     job = new_kcached_job(dmc, bio, request_block, cache_block);
1098 
1099     head = to_bytes(offset);
1100 
1101     left = (dmc->src_dev->bdev->bd_inode->i_size>>9) � request_block;
1102     if (left < dmc->block_size) {
1103         tail = to_bytes(left) � bio->bi_size � head;
1104         job->src.count = left;
1105         job->dest.count = left;
1106     } else
1107         tail = to_bytes(dmc->block_size) � bio->bi_size � head;
1108 
1109     /* Requested block is aligned with a cache block */
1110     if (0 == head && 0 == tail)
1111         job->nr_pages= 0;
1112     else /* Need new pages to store extra data */
1113         job->nr_pages = dm_div_up(head, PAGE_SIZE) + dm_div_up(tail, PAGE_SIZE);
1114     job->rw = READ; /* Fetch data from the source device */
1115 
1118     queue_job(job);
1119 
1120     return 0;
1121 }

函数首先调用cache_insert()更新cache, 设置该cacheblock.state为RESERVED. 然后调用new_kcached_job()分配一个kcached_job对象. 第1109~1104行是核心代码, 如前文所述, 上层请求的bio已经由DM框架按cacheblock单位切分, 也就是说, cache_map()所处理的每个bio请求的扇区数最大为cacheblock. 如下图所示: 第1099行获得这个bio在cacheblock中的偏移, 保存在head. 第1101行获得第request_block块扇区到磁盘最后一块扇区所跨过的扇区数. 第1103或1107行获得bio请求的数据最后一个字节离磁盘最后一字节(或者下一个cacheblock)的偏移. 第1110行, 如果0 == head且0 == tail, 说明所请求的bio正好覆盖整个cacheblock. 否则, 说明请求的bio只占cacheblock的一部分, 针对这种情况, 需要为该bio未请求的前后两部分分别分配页面. 因为dm-cache请求磁盘的单位为cacheblock大小. 第1114行指定job的读写方向为READ. 最后, 第1118行提交job.

回头看看cache_read_miss()中的1097行分配job所调用的函数new_kcached_job(), 第3个参数request_block表示bio请求在磁盘的起始扇区号, 第4个参数cache_block表示bio请求在SSD的起始扇区号.

1049 static struct kcached_job *new_kcached_job(struct cache_c *dmc, struct bio* bio,
1050                                            sector_t request_block,
1051                                            sector_t cache_block)
1052 {
1053     struct dm_io_region src, dest;
1054     struct kcached_job *job;
1055 
1056     src.bdev = dmc->src_dev->bdev;
1057     src.sector = request_block;
1058     src.count = dmc->block_size;
1059     dest.bdev = dmc->cache_dev->bdev;
1060     dest.sector = cache_block << dmc->block_shift;
1061     dest.count = src.count;
1062 
1063     job = mempool_alloc(_job_pool, GFP_NOIO);
1064     job->dmc = dmc;
1065     job->bio = bio;
1066     job->src = src;
1067     job->dest = dest;
1068     job->cacheblock = &dmc->cache[cache_block];
1069 
1070     return job;
1071 }

接下来看看job结构的定义:

126 /* Structure for a kcached job */
127 struct kcached_job {
128     struct list_head list;
129     struct cache_c *dmc;
130     struct bio *bio;   /* Original bio */
131     struct dm_io_region src;
132     struct dm_io_region dest;
133     struct cacheblock *cacheblock;
134     int rw;
135     /*
136      * When the original bio is not aligned with cache blocks,
137      * we need extra bvecs and pages for padding.
138      */
139     struct bio_vec *bvec;
140     unsigned int nr_pages;
141     struct page_list *pages;
142 };

在dm-cache中, job有3种状态, 它以list字段链入其所属于的链表, 分别为_io_jobs, _pages_jobs和_complete_jobs. 其中, io_jobs表示待执行IO的任务, page_jobs待分配页面的任务, compelet_jobs表示待收尾的任务. kcached_job的bio字段存储DM框架发给cache_map()的bio请求, src和dest分别指向磁盘和SSD的dm_io_region. cacheblock指针指向cache_c.cache数组中以请求的bio所落在的SSD磁盘上的cacheblock编号为下标的偏移. rw字段表示job当前的读写方向.

回到cache_read_miss()函数, 它在第1118行调用queue_job()提交了任务, 代码如下:

736 static void queue_job(struct kcached_job *job)
737 {
738     atomic_inc(&job->dmc->nr_jobs);
739     if (job->nr_pages > 0) /* Request pages */
740         push(&_pages_jobs, job);
741     else /* Go ahead to do I/O */
742         push(&_io_jobs, job);
743     wake();
744 }

可以看到, 如果需要为job分配页面, 则将job链入_pages_jobs链表, 否则, 链入_io_jobs链表. 然后调用wake():

299 static inline void wake(void)
300 {
301     queue_work(_kcached_wq, &_kcached_work);
302 }

wake()函数只是对queue_work()的封装, 它将_kcached_work提交到_kcached_wq. 在dm-cache模块初始化函数dm_cache_init()中, _kcached_work被注册的回调为do_work. 所以, 当_kcached_work被调度时, do_work()将被回调.

729 static void do_work(struct work_struct *ignored)
730 {
731     process_jobs(&_complete_jobs, do_complete);
732     process_jobs(&_pages_jobs, do_pages);
733     process_jobs(&_io_jobs, do_io);
734 }

可见, do_work()依次遍历_complete_jobs, _pages_jobs和_io_jobs链表中的任务, 以任务为参数, 分别回调do_complete, do_pages, do_io. 在这里, 遍历的顺序是有讲究的: 先处理_complete_jobs任务, 是因为此类任务完成后可能释放一些页面回页面内存池; 然后处理_pages_jobs任务, 因为此类任务只有获取页面后才能执行IO操作, 它从页面内存池中获取页面; 最后处理_io_jobs链表任务.

process_jobs()代码如下:

696 /*
697  * Run through a list for as long as possible.  Returns the count
698  * of successful jobs.
699  */
700 static int process_jobs(struct list_head *jobs,
701                         int (*fn) (struct kcached_job *))
702 {
703     struct kcached_job *job;
704     int r, count = 0;
705 
706     while ((job = pop(jobs))) {
707         r = fn(job);
708 
709         if (r < 0) {
710             /* error this rogue job */
711             DMERR("process_jobs: Job processing error");
712         }
713 
714         if (r > 0) {
715             /*
716              * We couldn’t service this job ATM, so
717              * push this job back onto the list.
718              */
719             push(jobs, job);
720             break;
721         }
722 
723         count++;
724     }
725 
726     return count;
727 }

它依次遍历链表, 调用回调.

回到queue_job(), 前面说过, 因为dm-cache读写SSD及磁盘的粒度为cacheblock大小, 所以如果bio请求未对其cacheblock, 或请求大小不等于cacheblock大小, 则需要为该cacheblock中, bio不关心的前后部分分配页面, 即把job提交到_pages_jobs链表. 否则, 直接提交到_io_jobs链表.

_pages_jobs链表的回调函数do_pages非常简单, 它从页面内存池获取一些页面(页面数为nr_pages), 保存在kcached_job结构的pages字段, 然后将job提交到_io_jobs链表.

针对_io_jobs链表上的任务, do_work()将以do_io回调来处理该任务.

618 static int do_io(struct kcached_job *job)
619 {
620     int r = 0;
621 
622     if (job->rw == READ) { /* Read from source device */
623         r = do_fetch(job);
624     } else { /* Write to cache device */
625         r = do_store(job);
626     }
627 
628     return r;
629 }

针对读请求, 很明显是进入do_fetch()分支.

400 /*
401  * Fetch data from the source device asynchronously.
402  * For a READ bio, if a cache block is larger than the requested data, then
403  * additional data are prefetched. Larger cache block size enables more
404  * aggressive read prefetching, which is useful for read-mostly usage.
405  * For a WRITE bio, if a cache block is larger than the requested data, the
406  * entire block needs to be fetched, and larger block size incurs more overhead.
407  * In scenaros where writes are frequent, 4KB is a good cache block size.
408  */
409 static int do_fetch(struct kcached_job *job)
410 {
411     int r = 0, i, j;
412     struct bio *bio = job->bio;
413     struct cache_c *dmc = job->dmc;
414     unsigned int offset, head, tail, remaining, nr_vecs, idx = 0;
415     struct bio_vec *bvec;
416     struct page_list *pl;
417     printk("do_fetch");
418     offset = (unsigned int) (bio->bi_sector & dmc->block_mask);
419     head = to_bytes(offset);
420     tail = to_bytes(dmc->block_size) � bio->bi_size � head;
425 
426     if (bio_data_dir(bio) == READ) { /* The original request is a READ */
427         if (0 == job->nr_pages) { /* The request is aligned to cache block */
428             r = dm_io_async_bvec(1, &job->src, READ,
429                                  bio->bi_io_vec + bio->bi_idx,
430                                  io_callback, job);
431             return r;
432         }
433 
434         nr_vecs = bio->bi_vcnt � bio->bi_idx + job->nr_pages;
435         bvec = kmalloc(nr_vecs * sizeof(*bvec), GFP_NOIO);
436         if (!bvec) {
437             DMERR("do_fetch: No memory");
438             return 1;
439         }
440 
441         pl = job->pages;
442         i = 0;
443         while (head) {
444             bvec[i].bv_len = min(head, (unsigned int)PAGE_SIZE);
445             bvec[i].bv_offset = 0;
446             bvec[i].bv_page = pl->page;
447             head -= bvec[i].bv_len;
448             pl = pl->next;
449             i++;
450         }
451 
452         remaining = bio->bi_size;
453         j = bio->bi_idx;
454         while (remaining) {
455             bvec[i] = bio->bi_io_vec[j];
456             remaining -= bvec[i].bv_len;
457             i++; j++;
458         }
459 
460         while (tail) {
461             bvec[i].bv_len = min(tail, (unsigned int)PAGE_SIZE);
462             bvec[i].bv_offset = 0;
463             bvec[i].bv_page = pl->page;
464             tail -= bvec[i].bv_len;
465             pl = pl->next;
466             i++;
467         }
468 
469         job->bvec = bvec;
470         r = dm_io_async_bvec(1, &job->src, READ, job->bvec, io_callback, job);
471         return r;
472     } else { /* The original request is a WRITE */
541     }
542 }

-如果任务没有申请页面, 即bio请求正好cacheblock对齐且请求大小正好为一个cacheblock, 则直接调用dm_io_async_bvec().
-如果任务申请了页面, 即bio请求不是cacheblock对齐, 或者请求大小不是一个cacheblock, 则通过第434~467行代码主动构造一个bio_vec *bvec, 保存在job->bvec中, 然后调用dm_io_async_bvec().

仔细比较上述两种情况调用dm_io_async_bvec()所传递的参数, 不难发现, 只有第4个参数是不一样的. 前者传递的为原来请求的bio的bvec, 后者传递的为主动构造的bvec.

dm_io_async_bvec()函数提交IO, 从磁盘(job->src)中读取数据到第4个参数, 然后回调io_callback().

382 static void io_callback(unsigned long error, void *context)
383 {
384     struct kcached_job *job = (struct kcached_job *) context;
385 
386     if (error) {
387         /* TODO */
388         DMERR("io_callback: io error");
389         return;
390     }
391 
392     if (job->rw == READ) {
393         job->rw = WRITE;
394         push(&_io_jobs, job);
395     } else
396         push(&_complete_jobs, job);
397     wake();
398 }

读请求的job->rw为READ, 将其修改为WRITE后将job提交到_io_jobs链表. _io_jobs链表元素再次由do_work()以do_io()回调. 此时, 因为job->rw为WRITE, 所以调用的函数变成了do_store().

544 /*
545  * Store data to the cache source device asynchronously.
546  * For a READ bio request, the data fetched from the source device are returned
547  * to kernel and stored in cache at the same time.
548  * For a WRITE bio request, the data are written to the cache and source device
549  * at the same time.
550  */
551 static int do_store(struct kcached_job *job)
552 {
553     int i, j, r = 0;
554     struct bio *bio = job->bio ;
555     struct cache_c *dmc = job->dmc;
556     unsigned int offset, head, tail, remaining, nr_vecs;
557     struct bio_vec *bvec;
558     offset = (unsigned int) (bio->bi_sector & dmc->block_mask);
559     head = to_bytes(offset);
560     tail = to_bytes(dmc->block_size) � bio->bi_size � head;
566 
567     if (0 == job->nr_pages) /* Original request is aligned with cache blocks */
568         r = dm_io_async_bvec(1, &job->dest, WRITE, bio->bi_io_vec + bio->bi_idx,
569                              io_callback, job);
570     else {
571         if (bio_data_dir(bio) == WRITE && head > 0 && tail > 0) {
573             nr_vecs = job->nr_pages + bio->bi_vcnt � bio->bi_idx;
574             if (offset && (offset + bio->bi_size < PAGE_SIZE)) nr_vecs++;
576             bvec = kmalloc(nr_vecs * sizeof(*bvec), GFP_KERNEL);
577             if (!bvec) {
578                 DMERR("do_store: No memory");
579                 return 1;
580             }
581 
582             i = 0;
583             while (head) {
584                 bvec[i].bv_len = min(head, job->bvec[i].bv_len);
585                 bvec[i].bv_offset = 0;
586                 bvec[i].bv_page = job->bvec[i].bv_page;
587                 head -= bvec[i].bv_len;
588                 i++;
589             }
590             remaining = bio->bi_size;
591             j = bio->bi_idx;
592             while (remaining) {
593                 bvec[i] = bio->bi_io_vec[j];
594                 remaining -= bvec[i].bv_len;
595                 i++; j++;
596             }
597             j = (to_bytes(offset) + bio->bi_size) / PAGE_SIZE;
598             bvec[i].bv_offset = (to_bytes(offset) + bio->bi_size) -
599                                 j * PAGE_SIZE;
600             bvec[i].bv_len = PAGE_SIZE � bvec[i].bv_offset;
601             bvec[i].bv_page = job->bvec[j].bv_page;
602             tail -= bvec[i].bv_len;
603             i++; j++;
604             while (tail) {
605                 bvec[i] = job->bvec[j];
606                 tail -= bvec[i].bv_len;
607                 i++; j++;
608             }
609             kfree(job->bvec);
610             job->bvec = bvec;
611         }
612 
613         r = dm_io_async_bvec(1, &job->dest, WRITE, job->bvec, io_callback, job);
614     }
615     return r;
616 }

这段代码和do_fetch()非常相像, 不再细述. 它把do_fetch()中从磁盘读取的数据, 通过dm_io_async_bvec()函数, 写入SSD(job->dest). 然后io_callback()再次被回调. 此时, 因为job->rw为WRITE, io_callback()将任务提交到_complete_jobs链表. 该链表对应的回调函数为do_complete():

673 static int do_complete(struct kcached_job *job)
674 {
675     int r = 0;
676     struct bio *bio = job->bio;
677 
680     bio_endio(bio, 0);
681 
682     if (job->nr_pages > 0) {
683         kfree(job->bvec);
684         kcached_put_pages(job->dmc, job->pages);
685     }
686 
687     flush_bios(job->cacheblock);
688     mempool_free(job, _job_pool);
689 
690     if (atomic_dec_and_test(&job->dmc->nr_jobs))
691         wake_up(&job->dmc->destroyq);
692 
693     return r;
694 }

do_complete()首先调用bio_endio(), 告诉IO子系统上层, 当前bio已经处理完成. 然后释放页面. 之后调用flush_bios()重新提交在当前bio之后所有发往同个cacheblock的bios, 最后释放job.

至此, 读请求完成. 写请求与读请求大同小异, 不表.

总结陈词

IO处理内核化是一种有效的IO优化方式. 另外, IO路径网络化(iSCSI)也是大势所趋, 如Amazon的EBS及腾讯的CBS(入门参考块存储的世界). 希望以后一窥究竟.