Linux下使用hiredis库与libevent实现异步接口的I/O复用
1 前言
之前的一篇文章《Linux下使用hiredis库实现优先级队列》，用的同步的接口实践；
后来遇到一个场景，同时需要处理Redis订阅的消息，又需要处理其他网络socket操作、定时器操作，当然多线程是一个思路，本文尝试从Reactive模式上解决这个问题，即用redis的异步接口，与libevent进行对接。
其实最终目的就是就是Redis接入到这篇文章编程实例里面：《Linux下使用libevent库实现服务器端编程》
2.接口
2.1 async.h
/usr/include/hiredis/aync.h里面，定义了异步接口:
连接与释放的接口：
/* Functions that proxy to hiredis */
redisAsyncContext *redisAsyncConnect(const char *ip, int port);
void redisAsyncDisconnect(redisAsyncContext *ac);
可以注册连接、断连接的回调函数：
（上述异步connect是直接返回成功的，需要在callback里面判断是否连接redis成功）
int redisAsyncSetConnectCallback(redisAsyncContext *ac, redisConnectCallback *fn);
int redisAsyncSetDisconnectCallback(redisAsyncContext *ac, redisDisconnectCallback *fn);
读写处理，具体怎么用先不纠结：
/* Handle read/write events */
void redisAsyncHandleRead(redisAsyncContext *ac);
void redisAsyncHandleWrite(redisAsyncContext *ac);
与同步接口用法类似redisCommand，异步命令用如下接口，但是通过redisCallbackFn获取操作的结果：
/* Command functions for an async context. Write the command to the
* output buffer and register the provided callback. */
int redisAsyncCommand(redisAsyncContext *ac, redisCallbackFn *fn, void *privdata, const char *format, ...);
异步实例的结构体定义：
/* Context for an async connection to Redis */
typedef struct redisAsyncContext {
/* Hold the regular context, so it can be realloc'ed. */
redisContext c;
/* Setup error flags so they can be used directly. */
int err;
char *errstr;
/* Not used by hiredis */
void *data;
/* Event library data and hooks */
struct {
void *data;
/* Hooks that are called when the library expects to start
* reading/writing. These functions should be idempotent. */
void (*addRead)(void *privdata);
void (*delRead)(void *privdata);
void (*addWrite)(void *privdata);
void (*delWrite)(void *privdata);
void (*cleanup)(void *privdata);
} ev;
/* Called when either the connection is terminated due to an error or per
* user request. The status is set accordingly (REDIS_OK, REDIS_ERR). */
redisDisconnectCallback *onDisconnect;
/* Called when the first write event was received. */
redisConnectCallback *onConnect;
/* Regular command callbacks */
redisCallbackList replies;
/* Subscription callbacks */
struct {
redisCallbackList invalid;
struct dict *channels;
struct dict *patterns;
} sub;
} redisAsyncContext;
2.2 adapters接口
接口然后下一步看看怎么跟libevent结合，能在结构体看到里面有个ev结构，这个地方就是事件触发的核心。
这块hiredis已经帮我们考虑到了，直接用async.h比较麻烦，所以他都给适配了大部分事件库的接口：
/usr/include/hiredis/adapters
├── ae.h
├── glib.h
├── ivykis.h
├── libevent.h
├── libev.h
├── libuv.h
├── macosx.h
└── qt.h
与libevent关联，我们需要额外调用一个接口redisLibeventAttach：
static int redisLibeventAttach(redisAsyncContext *ac, struct event_base *base) {
redisContext *c = &(ac->c);
redisLibeventEvents *e;
/* Nothing should be attached when something is already attached */
if (ac->ev.data != NULL)
return REDIS_ERR;
/* Create container for context and r/w events */
e = (redisLibeventEvents*)malloc(sizeof(*e));
e->context = ac;
/* Register functions to start/stop listening for events */
ac->ev.addRead = redisLibeventAddRead;
ac->ev.delRead = redisLibeventDelRead;
ac->ev.addWrite = redisLibeventAddWrite;
ac->ev.delWrite = redisLibeventDelWrite;
ac->ev.cleanup = redisLibeventCleanup;
ac->ev.data = e;
/* Initialize and install read/write events */
e->rev = event_new(base, c->fd, EV_READ, redisLibeventReadEvent, e);
e->wev = event_new(base, c->fd, EV_WRITE, redisLibeventWriteEvent, e);
event_add(e->rev, NULL);
event_add(e->wev, NULL);
return REDIS_OK;
}
3 编程实例
以下编程实例准备实现：
- Redis异步连接，断线重连机制（定时器）；
- Redis-LIST-POP功能，外部有事件或超时了，进行触发；
- Libevent定时器，模拟其他业务功能（定时器）；
- Libevent信号处理；
3.1 类定义
定义类class mod_redisev如下：
#include 轻松愉快的main入口：
int main(int argc, char *argv[])
{
class mod_redisev redisev;
redisev.redis_connect();
redisev.dispatch();
redisev.redis_close();
exit(EXIT_SUCCESS);
}
3.2 Redis连接
异步连接功能，用redisLibeventAttach与libevent绑定：
设置两个回调函数：on_redis_connect、on_redis_close
int mod_redisev::redis_connect()
{
int res = -1;
_rc = redisAsyncConnect(_addr.c_str(), _port);
if (!_rc || _rc->err) {
printf("redisAsyncConnect: %s\n", _rc ? _rc->errstr : "error");
return -1;
}
_rc->data = this; // attch to class
res = redisLibeventAttach(_rc, _base);
if (0 != res) {
printf("redisLibeventAttach\n");
return -1;
}
(void)redisAsyncSetConnectCallback(_rc, on_redis_connect);
(void)redisAsyncSetDisconnectCallback(_rc, on_redis_close);
return 0;
}
连接的流程为：触发连接成功了，设置REDIS_OK标识:
void mod_redisev::__on_connect(int status)
{
_status = status;
switch (_status) {
case REDIS_OK:
printf("Redis connected...\n");
(void)redisAsyncCommand(_rc, on_redis_auth, this, "AUTH 123456");
break;
case REDIS_ERR:
printf("Redis reconnecting...\n");
break;
default:
break;
}
}
Redis的连接保活借助了定时器对 _status的检查，如果连接不正常，再次进行connect操作:
void mod_redisev::__do_keepalive()
{
if (_status != REDIS_OK) {
printf("Reconect...\n");
redis_connect();
}
}
3.2 认证
上述__on_connnect函数中，如果连接成功了，则发起认证指令AUTH
AUTH并不是可选的，由于我redis-server配置了密码：
/etc/redis.conf
requirepass 123456
认证结果的处理，如果成功，那么进入下一个逻辑，BRPOP获取数据
void mod_redisev::__on_auth(struct redisReply *reply)
{
if (!reply || reply->type == REDIS_REPLY_ERROR) {
printf("Reply: %s\n", reply ? reply->str : "error");
_status = REDIS_ERR;
return;
}
else if (reply->type != REDIS_REPLY_STATUS) {
printf("Reply unknown: %d\n", reply->type);
_status = REDIS_ERR;
return;
}
printf("AUTH success...\n");
(void)redisAsyncCommand(_rc, on_redis_pop, this, "BRPOP queue1 queue2 queue3 10");
}
注意常见的认证错误有下面这种：
Reply: NOAUTH Authentication required. // 没有配置密码，无须认证
Reply: ERR invalid password // 错误的密码
3.3 拉取数据
认证成功后，进入拉取数据阶段，这里用了循环拉取的逻辑：
以阻塞10秒的形式，分别看 queue1 queue2 queue3 是否有数据过来：
BRPOP queue1 queue2 queue3 10
void mod_redisev::__on_pop(struct redisReply *reply)
{
if (!reply || reply->type == REDIS_REPLY_ERROR) {
printf("Reply: %s\n", reply ? reply->str : "error");
_status = REDIS_ERR;
return;
}
if (reply->type == REDIS_REPLY_NIL) {
printf("BRPOP: empty...\n");
}
else if (reply->type == REDIS_REPLY_ARRAY) {
if (reply->elements > 0) {
struct redisReply *rs = reply->element[1];
printf("BRPOP: %s\n", rs->str);
}
}
redisAsyncCommand(_rc, on_redis_pop, this, "BRPOP queue1 queue2 queue3 10");
}
这里就体现出异步事件的优势出来了，如果使用同步的请求接口，那么就得阻塞10秒，期间什么也不能干；
每秒1次的，假装干活的接口：
class mod_redisev {
...
static void on_do_something(int fd, short events, void *args)
{
printf("Do something...\n");
}
...
};
3.4 信号处理
class mod_redisev {
...
static void on_signal_quit(int fd, short events, void *args)
{
((class mod_redisev *)args)->__on_quit();
}
void __on_quit()
{
printf("quit...\n");
event_base_loopbreak(_base);
}
...
};
最后剩下的两个处理，dispatch与连接关闭：
int mod_redisev::dispatch()
{
struct timeval tv = {1};
event_add(_ev_timer, &tv);
event_add(_ev_kalive, &tv);
event_add(_ev_quit, NULL);
return event_base_dispatch(_base);
}
void mod_redisev::redis_close()
{
if (_rc && _status == REDIS_OK) {
printf("Redis disconnect...\n");
redisAsyncDisconnect(_rc); // __redisAsyncFree() called
_rc = NULL;
}
}
3.5 执行效果
编译方法记得把 -lhiredis、-levent 加进来
g++ -o redis_async -std=c++11 -Wall -g2 -O3 -Os redis_async.cc -levent -lhiredis
Redis connected... // redis连接成功
AUTH success... // auth成功，发起BRPOP请求
Do something... // 每秒1次的 do-something
Do something...
Do something...
Do something...
Do something...
Do something...
Do something...
Do something...
Do something...
Do something...
BRPOP: empty... // BRPOP 10秒超时到，没有数据
Do something... // 每秒1次的、执着的 do-something
Do something...
Do something...
Do something...
Do something...
Do something...
BRPOP: a // 外部我执行了命令，LPUSH queue1 a b c d e
BRPOP: b // BRPOP函数连续触发出来，获取 a b c d e
BRPOP: c
BRPOP: d
BRPOP: e
Do something...
Do something...
Do something...
Do something...
quit... // 外部 ctrl + c
Redis disconnect... // 回收资源，disconnect
4. 总结
本文通过libhiredis库提供的async异步接口，利用了adapters/libevent.h实现快速与libevent衔接，主要适合的场景，还是单线程下的事件触发模型；
主要是几个心得：
- 需要习惯的地方，异步回调的编程思路，每个命令都需要一个回调函数处理；
- 如果中间还想穿插直接用同步接口，如redisCommand，似乎_rc->c是不能直接用；
- hiredis容易用错出现double-free，需要外部_status进行综合判断多多检查；
- valgrind检查内存，中间过程倒是不泄露，但最后redis-libevent释放有不干净的地方；
参考文章:
[1] https://github.com/redis/hiredis/blob/master/README.md