通过UDFS实现Memcached与MySQL的自动更新

本文出自：http://www.mpyun.com/

UDFs是User Defined Functions的缩写，表示MYSQL的用户定义函数，应用程序可以利用这些函数从MYSQL5.0以上版本的数据库中访问Memcached写入或者获取的数据。此外，MYSQL从5.1版本开始支持触发器，从而可以在触发器中使用UDFs直接更新Memcached的内容，这种方式降低了应用程序设计和编写的复杂性。

安装UDFs需要在数据库上安装两个包，分别是libmemcached和memcached_functions_mysql，并且保证Mysql为5.1及以上版本，可通过mysql --version命令查看当前 Mysql版本。

libmemcached下载可以访问其官方站：
http://download.tangent.org/

memcached_functions_mysql下载地址：

http://patg.net/downloads/

第一次安装用的是libmemcached-0.53.tar.gz，编译memcached_functions_mysql-0.9时遇到了下面的问题：

servers.c: In function 'memc_servers_set':

servers.c:122: error: 'memcached_st' has no member named 'hosts'

servers.c:123: error: 'memcached_st' has no member named 'hosts'

servers.c:124: error: 'memcached_st' has no member named 'hosts'

最后用libmemcached-0.34.tar.gz，就没问题了，可能是软件的兼容性问题吧。

[root@bogon home]# tar -zxvf libmemcached-0.34.tar.gz

[root@bogon home]# cd libmemcached-0.34

[root@localhost libmemcached-0.34]#  ./configure --prefix=/usr/local/libmemcached34 --with-memcached=/usr/local/bin/memcached

[root@localhost libmemcached-0.34]# make && make install

[root@bogon home]# tar zxvf memcached_functions_mysql-0.9.tar.gz

[root@localhost memcached_functions_mysql-0.9]# ./configure --prefix=/usr/local/memcache_mysql --with-mysql=/usr/local/mysql/bin/mysql_config --with-libmemcached=/usr/local/libmemcached34 #/usr/local/mysql为MySQL安装目录

加上--with-libmemcached=/usr/local/libmemcached53不然会报以下错误

checking for libmemcached >= 0.17... configure: error: libmemcached not found

[root@localhost memcached_functions_mysql-0.9]# make && make install

安装完毕。

首次需要使用CREATE FUNCTION来初始化用户定义函数。有两种方法可以初始化所有Mysql提供的用户定义函数：

1),在Mysql的SQL命令行中执行memcached_functions_mysql源码目录下的sql/install_functions.sql。

2), 运行memcached_functions_mysql源码目录下的utils/install.pl这个Perl脚本，把memcache function作为UDFs加入MySQL。

这里采用第一种办法：

mysql> use mysql;

mysql> source /home/memcached_functions_mysql-1.1/sql/install_functions.sql

出现如下错误：

ERROR 1126 (HY000): Can't open shared library 'libmemcached_functions_mysql.so' (errno: 22 /usr/local/mysql/lib/mysql/plugin/libmemcached_functions_mysql.so: cannot open shared object file: No such file or directory)

解决办法手动拷贝libmemcached_functions_mysql.so库到 Mysql安装目录：

[root@localhost lib]# cp /usr/local/memcache_mysql/lib/libmemcached_functions_mysql.so* /usr/local/mysql/lib/mysql/plugin/

2.查看安装是否成功：

mysql> select name,dl from mysql.func;

+------------------------------+---------------------------------+

| name                         | dl                              |

+------------------------------+---------------------------------+

| memc_add                     | libmemcached_functions_mysql.so |

| memc_add_by_key              | libmemcached_functions_mysql.so |

| memc_servers_set             | libmemcached_functions_mysql.so |

| memc_server_count            | libmemcached_functions_mysql.so |

| memc_set                     | libmemcached_functions_mysql.so |

| memc_set_by_key              | libmemcached_functions_mysql.so |

| memc_cas                     | libmemcached_functions_mysql.so |

| memc_cas_by_key              | libmemcached_functions_mysql.so |

| memc_get                     | libmemcached_functions_mysql.so |

| memc_get_by_key              | libmemcached_functions_mysql.so |

| memc_delete                  | libmemcached_functions_mysql.so |

| memc_delete_by_key           | libmemcached_functions_mysql.so |

| memc_append                  | libmemcached_functions_mysql.so |

| memc_append_by_key           | libmemcached_functions_mysql.so |

| memc_prepend                 | libmemcached_functions_mysql.so |

| memc_prepend_by_key          | libmemcached_functions_mysql.so |

| memc_increment               | libmemcached_functions_mysql.so |

| memc_decrement               | libmemcached_functions_mysql.so |

| memc_replace                 | libmemcached_functions_mysql.so |

| memc_replace_by_key          | libmemcached_functions_mysql.so |

| memc_servers_behavior_set    | libmemcached_functions_mysql.so |

| memc_servers_behavior_get    | libmemcached_functions_mysql.so |

| memc_behavior_set            | libmemcached_functions_mysql.so |

| memc_behavior_get            | libmemcached_functions_mysql.so |

| memc_list_behaviors          | libmemcached_functions_mysql.so |

| memc_list_hash_types         | libmemcached_functions_mysql.so |

| memc_list_distribution_types | libmemcached_functions_mysql.so |

| memc_udf_version             | libmemcached_functions_mysql.so |

| memc_libmemcached_version    | libmemcached_functions_mysql.so |

| memc_stats                   | libmemcached_functions_mysql.so |

| memc_stat_get_keys           | libmemcached_functions_mysql.so |

| memc_stat_get_value          | libmemcached_functions_mysql.so |

+------------------------------+---------------------------------+

32 rows in set (0.08 sec)

3.memcached_functions_mysql应用实例：

1).新建两张表：urls和results,更新urls表中的内容，使系统自动更新Memcached的内容。results用来就更新Memcached失败的记录。

SQL代码：

use test;

drop table if exists urls;

CREATE TABLE urls(

id int(10) NOT NULL,

url varchar(255) NOT NULL DEFAULT '',

PRIMARY KEY (id)

);

drop table if exists results;

CREATE TABLE results(

id int(10) NOT NULL,

results varchar(255) NOT NULL DEFAULT 'error',

time timestamp NULL DEFAULT CURRENT_TIMESTAMP,

PRIMARY KEY (id)

);

2).建立3个trigger

当向urls表中插入数据时，对Memcached执行set操作。trigger的代码如下：

insert操作：

DELIMITER //

DROP TRIGGER IF EXISTS url_mem_insert;

CREATE TRIGGER url_mem_insert

BEFORE INSERT ON urls

FOR EACH ROW BEGIN

set @mm = memc_set(NEW.id,NEW.url);

if @mm <> 0 then

insert into results(id) values(NEW.id);

end if;

END //

DELIMITER;

更新时对Memcached执行replace操作：

DELIMITER //

DROP TRIGGER IF EXISTS url_mem_update;

CREATE TRIGGER url_mem_update

BEFORE UPDATE ON urls

FOR EACH ROW BEGIN

set @mm = memc_replace(OLD.id,NEW.url);

if @mm <> 0 then

insert into results(id) values(OLD.id);

end if;

END //

DELIMITER;

删除对Memcached执行delete操作：

DELIMITER //

DROP TRIGGER IF EXISTS url_mem_delete;

CREATE TRIGGER url_mem_delete

BEFORE DELETE ON urls

FOR EACH ROW BEGIN

set @mm = memc_delete(OLD.id);

if @mm <> 0 then

insert into results(id) values(OLD.id);

end if;

END //

DELIMITER;

3).设置Memcached相关参数

设置UDFs操作Memcached服务器的IP地址和端口:

mysql> select memc_servers_set('192.168.56.2:11211');

+----------------------------------------+

| memc_servers_set('192.168.56.2:11211') |

+----------------------------------------+

|                                      0 |

+----------------------------------------+

1 row in set (0.01 sec)

mysql> select memc_server_count();

+---------------------+

| memc_server_count() |

+---------------------+

|                   1 |

+---------------------+

1 row in set (0.01 sec)

在MySQL命令行中列出可以修改Memcache参数的行为，如下：

mysql> select memc_list_behaviors()\G

*************************** 1. row ***************************

memc_list_behaviors():

MEMCACHED SERVER BEHAVIORS

MEMCACHED_BEHAVIOR_SUPPORT_CAS

MEMCACHED_BEHAVIOR_NO_BLOCK

MEMCACHED_BEHAVIOR_TCP_NODELAY

MEMCACHED_BEHAVIOR_HASH

MEMCACHED_BEHAVIOR_CACHE_LOOKUPS

MEMCACHED_BEHAVIOR_SOCKET_SEND_SIZE

MEMCACHED_BEHAVIOR_SOCKET_RECV_SIZE

MEMCACHED_BEHAVIOR_BUFFER_REQUESTS

MEMCACHED_BEHAVIOR_KETAMA

MEMCACHED_BEHAVIOR_POLL_TIMEOUT

MEMCACHED_BEHAVIOR_RETRY_TIMEOUT

MEMCACHED_BEHAVIOR_DISTRIBUTION

MEMCACHED_BEHAVIOR_BUFFER_REQUESTS

MEMCACHED_BEHAVIOR_USER_DATA

MEMCACHED_BEHAVIOR_SORT_HOSTS

MEMCACHED_BEHAVIOR_VERIFY_KEY

MEMCACHED_BEHAVIOR_CONNECT_TIMEOUT

MEMCACHED_BEHAVIOR_KETAMA_WEIGHTED

MEMCACHED_BEHAVIOR_KETAMA_HASH

MEMCACHED_BEHAVIOR_BINARY_PROTOCOL

MEMCACHED_BEHAVIOR_SND_TIMEOUT

MEMCACHED_BEHAVIOR_RCV_TIMEOUT

MEMCACHED_BEHAVIOR_SERVER_FAILURE_LIMIT

MEMCACHED_BEHAVIOR_IO_MSG_WATERMARK

MEMCACHED_BEHAVIOR_IO_BYTES_WATERMARK

1 row in set (0.00 sec)

设置MEMCACHED_BEHAVIOR_NO_BLOCK为打开状态，这样在Memcached出现问题（不能连接）时，数据可以继续插入MySQL中，但有报错提示：如果不设置此值，那么Memcached失败时，数据要等到Memcached失败超时后才能插入到表中。

通过下面的设置，可以避免这种情况的发生。

mysql> select memc_servers_behavior_set('MEMCACHED_BEHAVIOR_NO_BLOCK','1');

+--------------------------------------------------------------+

| memc_servers_behavior_set('MEMCACHED_BEHAVIOR_NO_BLOCK','1') |

+--------------------------------------------------------------+

|                                                            0 |

+--------------------------------------------------------------+

1 row in set (0.01 sec)

mysql> select memc_servers_behavior_set('MEMCACHED_BEHAVIOR_TCP_NODELAY','1');

+-----------------------------------------------------------------+

| memc_servers_behavior_set('MEMCACHED_BEHAVIOR_TCP_NODELAY','1') |

+-----------------------------------------------------------------+

|                                                               0 |

+-----------------------------------------------------------------+

1 row in set (0.01 sec)

4).对memcached_functions_mysql的功能进行测试

向urls表中插入数据，然后查看Memcached是否对数据执行set操作:

mysql> use test;

Reading table information for completion of table and column names

You can turn off this feature to get a quicker startup with -A

Database changed

mysql> insert into urls(id,url) values (1,'http://www.test.com.cn');

Query OK, 1 row affected (0.09 sec)

mysql> select memc_get('1');

+------------------------+

| memc_get('1')          |

+------------------------+

| http://www.test.com.cn |

+------------------------+

1 row in set (0.01 sec)

[root@localhost ~]# telnet 192.168.56.2 11211

Trying 192.168.56.2...

Connected to 192.168.56.2 (192.168.56.2).

Escape character is '^]'.

get 1

VALUE 1 0 22

http://www.test.com.cn

END

更新测试：

mysql> update test.urls set url='http://blog.test.com.cn' where id=1;

Query OK, 1 row affected (0.05 sec)

Rows matched: 1  Changed: 1  Warnings: 0

mysql> select memc_get('1');

+-------------------------+

| memc_get('1')           |

+-------------------------+

| http://blog.test.com.cn |

+-------------------------+

1 row in set (0.01 sec)

[root@localhost ~]#  telnet 192.168.56.2 11211

Trying 192.168.56.2...

Connected to 192.168.56.2 (192.168.56.2).

Escape character is '^]'.

get 1

VALUE 1 0 23

http://blog.test.com.cn

END

删除测试：

mysql> delete from test.urls where id=1;

Query OK, 1 row affected (0.00 sec)

mysql> select memc_get('1');

+---------------+

| memc_get('1') |

+---------------+

| NULL          |

+---------------+

1 row in set (0.01 sec)

[root@localhost ~]#  telnet 192.168.56.2 11211

Trying 192.168.56.2...

Connected to 192.168.56.2 (192.168.56.2).

Escape character is '^]'.

get 1

END