Redis AOF刷新策略分析

阅读更多

此文为转载，原文： http://afei2.sinaapp.com/?p=536

   redis支持使用aof来进行持久化，防止数据丢失，aof的刷新策略通过参数appendfsync控制，有三个值：always、everysec、no，默认是everysec。
   下面从源码的角度剖析一下aof的刷新策略。
   每次redis进入event循环准备执行这个event时，会调用beforeSleep方法

 

void aeMain(aeEventLoop *eventLoop) {
    eventLoop->stop = 0;
    while (!eventLoop->stop) {
        if (eventLoop->beforesleep != NULL)
            eventLoop->beforesleep(eventLoop);
        aeProcessEvents(eventLoop, AE_ALL_EVENTS);
    }
}

/* This function gets called every time Redis is entering the
 * main loop of the event driven library, that is, before to sleep
 * for ready file descriptors. */
void beforeSleep(struct aeEventLoop *eventLoop) {
    ......
    /* Write the AOF buffer on disk */
    flushAppendOnlyFile(0);
    ......
}

    上面的代码中的flushAppendOnlyFile(int force)进行实际的执行。

   

   src/aof.c

 

void flushAppendOnlyFile(int force) {
    ......
    /* Perform the fsync if needed. */
    if (server.aof_fsync == AOF_FSYNC_ALWAYS) {
        /* aof_fsync is defined as fdatasync() for Linux in order to avoid
         * flushing metadata. */
        latencyStartMonitor(latency);
        aof_fsync(server.aof_fd); /* Let's try to get this data on the disk */
        latencyEndMonitor(latency);
        latencyAddSampleIfNeeded("aof-fsync-always",latency);
        server.aof_last_fsync = server.unixtime;
    } else if ((server.aof_fsync == AOF_FSYNC_EVERYSEC &&
                server.unixtime > server.aof_last_fsync)) {
        if (!sync_in_progress) aof_background_fsync(server.aof_fd);
        server.aof_last_fsync = server.unixtime;
    }
}

   AOF_FSYNC_ALWAYS会调用aof_fsync进行同步写入，而aof_fsync在linux下就是fdatasync，

   AOF_FSYNC_EVERYSEC会调用aof_background_fsync，而aof_background_fsync会创建一个任务交给后台的bio线程进行处理。

 

 

/* Define aof_fsync to fdatasync() in Linux and fsync() for all the rest */
#ifdef __linux__
#define aof_fsync fdatasync
#else
#define aof_fsync fsync
#endif
/* Starts a background task that performs fsync() against the specified
 * file descriptor (the one of the AOF file) in another thread. */
void aof_background_fsync(int fd) {
    bioCreateBackgroundJob(REDIS_BIO_AOF_FSYNC,(void*)(long)fd,NULL,NULL);
}

     其中everysec是通过下面的逻辑来进行的，检测后台是否fsync任务在进行，如果有的话，判断上次的fsync距离现在的时间，如果大于2s，则阻塞，否则直接进行后台队列。

 

    如果上一次的fsync执行了2s多，则会阻塞执行，直到写入成功，这个时候日志中会记录下面一条记录，并且增加info中对应的aof_delayed_fsync值
 
[5750] 12 Aug 09:56:17.057 * Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down Redis.
详细逻辑如下：

/*
 * When the fsync policy is set to 'everysec' we may delay the flush if there
 * is still an fsync() going on in the background thread, since for instance
 * on Linux write(2) will be blocked by the background fsync anyway.
 * When this happens we remember that there is some aof buffer to be
 * flushed ASAP, and will try to do that in the serverCron() function.
 *
 * However if force is set to 1 we'll write regardless of the background
 * fsync.
 *
 * 但是如果上一次的fsync执行了2s多，则会阻塞执行，直到写入成功
 */
    /* With this append fsync policy we do background fsyncing.
     * If the fsync is still in progress we can try to delay
     * the write for a couple of seconds. */
    if (sync_in_progress) {
        if (server.aof_flush_postponed_start == 0) {
            /* No previous write postponinig, remember that we are
             * postponing the flush and return. */
            server.aof_flush_postponed_start = server.unixtime;
            return;
        } else if (server.unixtime - server.aof_flush_postponed_start < 2) {
            /* We were already waiting for fsync to finish, but for less
             * than two seconds this is still ok. Postpone again. */
            return;
        }
        /* Otherwise fall trough, and go write since we can't wait
         * over two seconds. */

 aof_pending_bio_fsync
/* Return the number of pending jobs of the specified type. */
bioPendingJobsOfType(REDIS_BIO_AOF_FSYNC),
serverCron中检查server.aof_flush_postponed_start，如果有的话，就追加一次flush，但是只有在上面的情况下会导致阻塞，其他情况下都会很快返回；
/* AOF postponed flush: Try at every cron cycle if the slow fsync
* completed. */
if (server.aof_flush_postponed_start) flushAppendOnlyFile(0);