小探mod_memcache_block

[背景]

Twitter 开源了他们自己用的一个 Apache 模块 mod_memcache_block（a distributed IP blocking system），这个模块根据 HTTP 代码请求限制访问频率。 上周我装了下mod_memcache_block，出了很多错误，今天把它记录下来，供网上需要的朋友学习。

[过程]

下面的installation，是说明文档中的，不够详细，不过，放到前面供参考.

INSTALLATION

   1. Install libmemcached-0.25 or better.

   2. Install memcached-1.2.6

   3. Edit the Makefile to indicate the location of libmemcached

   4. Type "make", then "make install"

   5. Update your apache configuration

   6. Restart the server with apachectl restart

 

下载

wget http://download.tangent.org/libmemcached-0.25.tar.gz

wget http://memcached.googlecode.com/files/memcached-1.2.6.tar.gz

wget http://github.com/netik/mod_memcache_block/tarball/master

 

安装

(1)Apache 已经安装，并且支持 / 加载了 mod_so 模块

查看

[root@study ~]# httpd -l

Compiled in modules:

  core.c

  prefork.c

  http_core.c

  mod_so.c

 

(2) 安装 libmemcached

./configure --prefix=/usr/local/libmemcached

make && make install

(3) 安装 memcached

./configure --prefix=/usr/local/memcached

make && make install

(4) 安装 mod_memcache_block

tar zxvf netik-mod_memcache_block-7b1fcec4d3ecdd7dbec9523a69338bbd1a6889be.tar.gz

cd netik-mod_memcache_block-7b1fcec4d3ecdd7dbec9523a69338bbd1a6889be

 

[root@study netik-mod_memcache_block-7b1fcec4d3ecdd7dbec9523a69338bbd1a6889be]# /usr/local/apache2/bin/apxs  -I /usr/local/libmemcached/include/libmemcached -L /usr/local/libmemcached/lib -lmemcached  -c -i -a mod_memcache_block.c

/usr/local/apache2/bin/httpd: error while loading shared libraries: libiconv.so.2: cannot open shared object file: No such file or directory

apxs:Error: Sorry, no shared object support for Apache.

apxs:Error: available under your platform. Make sure.

apxs:Error: the Apache module mod_so is compiled into.

apxs:Error: your server binary `/usr/local/apache2/bin/httpd'..

 

解决： vi /etc/ld.so.conf

/usr/local/lib

/usr/local/libmemcached/lib

ldconfig

 

[root@study netik-mod_memcache_block-7b1fcec4d3ecdd7dbec9523a69338bbd1a6889be]# /usr/local/apache2/bin/apxs  -I /usr/local/libmemcached/include/libmemcached -L /usr/local/libmemcached/lib -lmemcached  -c -i -a mod_memcache_block.c

/usr/local/apache2/build/libtool --silent --mode=compile gcc -prefer-pic   -DLINUX=2 -D_REENTRANT -D_GNU_SOURCE -D_LARGEFILE64_SOURCE -g -O2 -pthread -I/usr/local/apache2/include  -I/usr/local/apache2/include   -I/usr/local/apache2/include  -I/usr/local/libmemcached/include/libmemcached  -c -o mod_memcache_block.lo mod_memcache_block.c && touch mod_memcache_block.slo

In file included from mod_memcache_block.c:62:

/usr/local/libmemcached/include/libmemcached/memcached.h:21:46: error: libmemcached/memcached_constants.h: No such file or directory

/usr/local/libmemcached/include/libmemcached/memcached.h:22:42: error: libmemcached/memcached_types.h: No such file or directory

/usr/local/libmemcached/include/libmemcached/memcached.h:23:47: error: libmemcached/memcached_watchpoint.h: No such file or directory

/usr/local/libmemcached/include/libmemcached/memcached.h:24:40: error: libmemcached/memcached_get.h: No such file or directory

/usr/local/libmemcached/include/libmemcached/memcached.h:25:43: error: libmemcached/memcached_server.h: No such file or directory

/usr/local/libmemcached/include/libmemcached/memcached.h:26:43: error: libmemcached/memcached_string.h: No such file or directory

/usr/local/libmemcached/include/libmemcached/memcached.h:27:43: error: libmemcached/memcached_result.h: No such file or directory

In file included from mod_memcache_block.c:62:

/usr/local/libmemcached/include/libmemcached/memcached.h:73: error: expected specifier-qualifier-list before ��m emcached_allocated � ?

 

解决 :vi /usr/local/libmemcached/include/libmemcached/memcached.h

修改

#include <libmemcached/libmemcached_config.h>

#endif

#include <libmemcached/memcached_constants.h>

#include <libmemcached/memcached_types.h>

#include <libmemcached/memcached_watchpoint.h>

#include <libmemcached/memcached_get.h>

#include <libmemcached/memcached_server.h>

#include <libmemcached/memcached_string.h>

#include <libmemcached/memcached_result.h>

 

为：

#include <libmemcached_config.h>

#endif

#include <memcached_constants.h>

#include <memcached_types.h>

#include <memcached_watchpoint.h>

#include </memcached_get.h>

#include </memcached_server.h>

#include </memcached_string.h>

#include </memcached_result.h>

 

再次执行

[root@study netik-mod_memcache_block-7b1fcec4d3ecdd7dbec9523a69338bbd1a6889be]# /usr/local/apache2/bin/apxs  -I /usr/local/libmemcached/include/libmemcached -L /usr/local/libmemcached/lib -lmemcached  -c -i -a mod_memcache_block.c

/usr/local/apache2/build/libtool --silent --mode=compile gcc -prefer-pic   -DLINUX=2 -D_REENTRANT -D_GNU_SOURCE -D_LARGEFILE64_SOURCE -g -O2 -pthread -I/usr/local/apache2/include  -I/usr/local/apache2/include   -I/usr/local/apache2/include  -I/usr/local/libmemcached/include/libmemcached  -c -o mod_memcache_block.lo mod_memcache_block.c && touch mod_memcache_block.slo

/usr/local/apache2/build/libtool --silent --mode=link gcc -o mod_memcache_block.la  -L/usr/local/libmemcached/lib -lmemcached -rpath /usr/local/apache2/modules -module -avoid-version    mod_memcache_block.lo

/usr/local/apache2/build/instdso.sh SH_LIBTOOL='/usr/local/apache2/build/libtool' mod_memcache_block.la /usr/local/apache2/modules

/usr/local/apache2/build/libtool --mode=install cp mod_memcache_block.la /usr/local/apache2/modules/

cp .libs/mod_memcache_block.so /usr/local/apache2/modules/mod_memcache_block.so

cp .libs/mod_memcache_block.lai /usr/local/apache2/modules/mod_memcache_block.la

cp .libs/mod_memcache_block.a /usr/local/apache2/modules/mod_memcache_block.a

chmod 644 /usr/local/apache2/modules/mod_memcache_block.a

ranlib /usr/local/apache2/modules/mod_memcache_block.a

PATH="$PATH:/sbin" ldconfig -n /usr/local/apache2/modules

----------------------------------------------------------------------

Libraries have been installed in:

   /usr/local/apache2/modules

 

If you ever happen to want to link against installed libraries

in a given directory, LIBDIR, you must either use libtool, and

specify the full pathname of the library, or use the `-LLIBDIR'

flag during linking and do at least one of the following:

   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable

     during execution

   - add LIBDIR to the `LD_RUN_PATH' environment variable

     during linking

   - use the `-Wl,--rpath -Wl,LIBDIR' linker flag

   - have your system administrator add LIBDIR to `/etc/ld.so.conf'

 

See any operating system documentation about shared libraries for

more information, such as the ld(1) and ld.so(8) manual pages.

----------------------------------------------------------------------

chmod 755 /usr/local/apache2/modules/mod_memcache_block.so

[activating module `memcache_block' in /usr/local/apache2/conf/httpd.conf]

 

 

OK, 已经安装，并且加载到 httpd.conf 里了。让我们检查下

[root@study netik-mod_memcache_block-7b1fcec4d3ecdd7dbec9523a69338bbd1a6889be]# more /usr/local/apache2/conf/httpd.conf|grep mod_memcache_block

LoadModule memcache_block_module modules/mod_memcache_block.so

 

OK ，重启下 apache, 下面让我们来玩玩配置吧 !~

 

(5) 配置，测试

/usr/local/memcached/bin/memcached  -d -m 64 -l 192.168.1.4 -p 11211 -u www

在 httpd.conf 的主配置区域添加

<IfModule memcache_block_module>

MBEnable On

MBTimeout 2

MBServers 192.168.1.4:11211

MBPrefix block

MBExpiration 3600

MBMaxBlocks 10

MBResponseLimit 304 50 3600     <=304 是我测试用的，大家根据自己的生产环境自己选择 CODE

</IfModule>

 

参

测试结果

(1) 当我在 1 个小时内，出现 304 50 次下，访问正常

(2) 当我在 1 个小时内，出现 304 50 次上，访问受到拒绝

 

注：

我对 MBResponseLimit 理解 :

在指定时间（ 3600 ）秒，一个固定的 IP 访问的 CODE （ 304 ）出现的次数低于 Count(50) ，那么就正常，否则，将其 IP 记录到 memcached 中 ( 过期时间为： MBExpiration) ，并加以拒绝访问。

 

再看 memcached

STAT cmd_get 55

STAT cmd_set 5

memcached 读取与设置已经开始工作了，如果再配置上对 memcached 的监控，那看得更清楚了。

 

参考： http://github.com/netik/mod_memcache_block/tarball/master