Redis之EventLoop分析

以下文章基于Redis3.0以上版本源码进行分析解读

---

摘要

1. redis服务初始化分为三个阶段
   1.1. 服务配置
   1.2. 服务初始化
   1.3. 启动event loop

2. redis服务端所有的操作都会被封装为event, 主要有两个类型的event
   2.1. IO Event 封装了在线的command请求
   2.2. Time Event 为1s执行一次的计划任务, 会处理以下逻辑
   -- 2.2.1. 过期key的清理
   -- 2.2.2. 内部的调用性能统计
   -- 2.2.3. DB对象的rehash扩容
   -- 2.2.4. RDB&AOF的数据持久化(如果有必要)
   -- 2.2.5. ...及其他一些检查操作

3. event loop为单线程处理
   3.1. 所有event的处理因为是单线程顺序处理, 所以在操作DB等内存数据时是无锁的
   3.1. 在每个process循环中都尝试处理所有已加入队列的io event和time event
   3.2. io event和time event是在同一个loop processor中顺序执行
   3.3. event loop中process的时延直接决定了redis server的吞吐量

---

先分析一下Event Loop初始化过程中做了什么(代码为server.c#main的常驻进程启动逻辑)

1. 创建event loop

   server.el = aeCreateEventLoop(server.maxclients+CONFIG_FDSET_INCR);
   if (server.el == NULL) {
       serverLog(LL_WARNING,
           "Failed creating the event loop. Error message: '%s'",
           strerror(errno));
       exit(1);
   }

2. 启动event loop

   // 每个event执行的前置函数
   aeSetBeforeSleepProc(server.el,beforeSleep);
   // 每个event执行完成后的后置函数
   aeSetAfterSleepProc(server.el,afterSleep);
   // 会真正启动event loop的处理, 一旦调用当前线程将block直至系统退出
   aeMain(server.el);
   // 系统退出前的关闭event loop
   aeDeleteEventLoop(server.el);
   return 0;

Event Loop是Redis Server调度的核心

EventLoop很简单, 不多说上代码

EventLoop很简单, 核心主要是event selector 和 event processor

1. event loop中保留了全局待处理events链表(io event 和 time event)

typedef struct aeEventLoop {
   int maxfd;   /* highest file descriptor currently registered */
   int setsize; /* max number of file descriptors tracked */
   long long timeEventNextId;
   time_t lastTime;     /* Used to detect system clock skew */
   aeFileEvent *events; /* Registered events */
   aeFiredEvent *fired; /* Fired events */
   aeTimeEvent *timeEventHead;
   int stop;
   void *apidata; /* This is used for polling API specific data */
   aeBeforeSleepProc *beforesleep;
   aeBeforeSleepProc *aftersleep;
} aeEventLoop;

2. event selector 其实就是简单的while true系循环, 执行顺序如下
   2.1 执行event loop前置的钩子函数 beforesleep
   2.2 调用event processor函数: aeProcessEvents执行所有队列中的io event 和 time event

void aeMain(aeEventLoop *eventLoop) {
   eventLoop->stop = 0;
   while (!eventLoop->stop) {
       if (eventLoop->beforesleep != NULL)
           eventLoop->beforesleep(eventLoop);
       aeProcessEvents(eventLoop, AE_ALL_EVENTS|AE_CALL_AFTER_SLEEP);
   }
}

3. event processor 执行顺序如下
   3.1 从链表中获取待执行的io event
   3.2 在循环中顺序执行io event
   3.3 check是否有待执行的time event, 如果有会执行time event

int aeProcessEvents(aeEventLoop *eventLoop, int flags)
{
        int processed = 0, numevents;
        ... 代码省略
       // poll所有带执行event
       /* Call the multiplexing API, will return only on timeout or when
       * some event fires. */
       numevents = aeApiPoll(eventLoop, tvp);

       /* After sleep callback. */
       if (eventLoop->aftersleep != NULL && flags & AE_CALL_AFTER_SLEEP)
           eventLoop->aftersleep(eventLoop);

       // 执行event... 此处代码很多直接忽略
       for (j = 0; j < numevents; j++) {       
          ... 代码省略
          processed++;
      }
      ... 代码省略
      // 在所有io event执行完后, 会check是否有需要执行的time event
      /* Check time events */
      if (flags & AE_TIME_EVENTS)
            processed += processTimeEvents(eventLoop);
}

EventLoop中主要是两个类型的event

1. 网络io请求event, 即接收到的redis command处理
2. 定时任务event, 只会在server的配置初始化阶段创建

service.c 中的main函数会注册一个1s执行一次的time event:

   /* Create the timer callback, this is our way to process many background
    * operations incrementally, like clients timeout, eviction of unaccessed
    * expired keys and so forth. */
   if (aeCreateTimeEvent(server.el, 1, serverCron, NULL, NULL) == AE_ERR) {
       serverPanic("Can't create event loop timers.");
       exit(1);
   }

IO Event的注册及处理逻辑

1. 服务初始化阶段#创建TCP端口侦听
   1.1 为io请求注册acceptTcpHandler(acceptUnixHandler)
   1.2 acceptTcpHandler(acceptUnixHandler) 将所有请求委派给acceptCommonHandler
2. 服务运行时, acceptCommonHandler处理所有input io, 并将command交给函数processCommand处理
   重要的事情说三遍, 所有的在线命令处理入口就是processCommand函数
   重要的事情说三遍, 所有的在线命令处理入口就是processCommand函数
   重要的事情说三遍, 所有的在线命令处理入口就是processCommand函数

IO Event的在线处理逻辑分析

其中特别要注意的是对于key侦听的逻辑处理, 这里对性能的影响会在后面的文章中分析

/* If this function gets called we already read a whole
* command, arguments are in the client argv/argc fields.
* processCommand() execute the command or prepare the
* server for a bulk read from the client.
*
* If C_OK is returned the client is still alive and valid and
* other operations can be performed by the caller. Otherwise
* if C_ERR is returned the client was destroyed (i.e. after QUIT). */
int processCommand(client *c) {
   // ... 省略一些前置检查的代码,包括:认证状态;集群状态;内存使用...
   // command的处理的核心逻辑
   /* Exec the command */
   if (c->flags & CLIENT_MULTI &&
       c->cmd->proc != execCommand && c->cmd->proc != discardCommand &&
       c->cmd->proc != multiCommand && c->cmd->proc != watchCommand)
   {
       // pipeline command的处理
       queueMultiCommand(c);
       addReply(c,shared.queued);
   } else {
       // 执行command
       call(c,CMD_CALL_FULL);
       c->woff = server.master_repl_offset;

       // 会检查全局的key侦听链表, 并尝试通知到在这些key上侦听的client
       if (listLength(server.ready_keys))
           // 这里对性能的影响会在后面的文章中分析
           handleClientsBlockedOnLists();
   }
   return C_OK;
}

Time Event 的处理逻辑分析

1. 从注释很清晰的知道, 在定时任务中处理了大量的对于内存和数据定期过期&持久化的工作
   1.1 Active expired keys collection (it is also performed in a lazy way >on
   1.2 lookup).
   1.3 Software watchdog.
   1.4 Update some statistic.
   1.5 Incremental rehashing of the DBs hash tables.
   1.6 Triggering BGSAVE / AOF rewrite, and handling of terminated children.
   1.7 Clients timeout of different kinds.
   1.8 Replication reconnection.
   1.9 Many more...
2. 需要注意的是, time event的处理是和io event一样在event loop中串行被处理的, 这些批处理任务的逻辑会block在线请求的处理

/* This is our timer interrupt, called server.hz times per second.
* Here is where we do a number of things that need to be done asynchronously.
* For instance:
*
* - Active expired keys collection (it is also performed in a lazy way >on
*   lookup).
* - Software watchdog.
* - Update some statistic.
* - Incremental rehashing of the DBs hash tables.
* - Triggering BGSAVE / AOF rewrite, and handling of terminated children.
* - Clients timeout of different kinds.
* - Replication reconnection.
* - Many more...
*
* Everything directly called here will be called server.hz times per second,
* so in order to throttle execution of things we want to do less frequently
* a macro is used: run_with_period(milliseconds) { .... }
*/

int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
    // .... 代码省略
}

---

对于Event Loop的分析告一段落, 后面的文章会在Event Loop分析的基础上, 针对性的对Redis在线服务的性能场景进行进一步的讲解.