Gearman实现Mysql到Redis的数据同步

redis与数据库结合，作为数据库数据的缓存,提高响应速度

一.实现数据库，php，http和redis的架构

server1提供http服务，使用php语言；
server2提供redis缓存服务；
server3提供后端数据库服务…

大概流程 ：客户端通过nginx和php访问后端数据库时，先在redis这个数据库缓存中查找，看是否含有想要的数据，如果没有就去后端数据库查找，将查找到数据返回给客户端一份，另外在redis中缓存一份….下次如果访问相同的数据就直接去redis，缩短时间同时也减轻数据库的查找压力

1.server1安装http，php，php-mysql

yum install -y httpd php php-mysql

2.获取测试页面test.php，修改后放到httpd发布目录

vim test.php 
mv test.php /var/www/html/
systemctl start httpd

test.php测试页面内容

connect('172.25.40.2',6379) or die ("could net connect redis server");
         #redis主机IP
  #      $query = "select * from test limit 9";
        $query = "select * from test";
        for ($key = 1; $key < 10; $key++)
        {
                if (!$redis->get($key))
                {
                        $connect = mysql_connect('172.25.40.3','redis','redhat'); # 连接数据库，ip，用户，密码
                        mysql_select_db(test);
                        $result = mysql_query($query);
                        //如果没有找到$key,就将该查询sql的结果缓存到redis
                        while ($row = mysql_fetch_assoc($result))
                        {
                                $redis->set($row['id'],$row['name']);
                        }
                        $myserver = 'mysql';
                        break;
                }
                else
                {
                        $myserver = "redis";
                        $data[$key] = $redis->get($key);
                }
        }
 
        echo $myserver;
        echo "
";
        for ($key = 1; $key < 10; $key++)
        {
                echo "number is $key";
 
                echo "
";
 
                echo "name is $data[$key]";
 
                echo "
";
        }
?>

3.php添加redis模块

php -m | grep mysql
php -m | grep redis

源码编译php与redis的连接软件—phpredis-master.zip

unzip phpredis-master.zip 
cd phpredis-master/
phpize  # 创建预编译环境

报错：

Can't find PHP headers in /usr/include/php
The php-devel package is required for use of this command.

网上下载安装同版本php-devel

yum install -y php-devel-5.4.16-42.el7.x86_64.rpm 
cd phpredis-master/
phpize   # 创建预编译环境
./configure --help
 ./configure --enable-redis
make && make install   # 源码编译安装

进入目录/usr/lib64/php/modules/下，查看编译后存在的模块文件

cd /etc/php.d/
cp mysql.ini redis.ini
vim redis.ini 
extension=redis.so

4.在php的配置文件中修改时区

vim /etc/php.ini 
date.timezone = Asia/Shanghai

重启httpd查看已经加载的模块

systemctl restart httpd
php -m | grep redis

5.server2安装redis

tar zxf redis-5.0.3.tar.gz 
cd redis-5.0.3
yum install gcc -y  
make 
make install
cd utils/
./install_server.sh 

   vim /etc/redis/6379.conf 
   bind 0.0.0.0
   systemctl start redis_6379
   ps ax

6.server3安装mariadb

systemctl stop redis_6379
chkconfig redis_6379 off
yum install -y mariadb-server
systemctl start mariadb
mysql #不做安全初始化，直接登陆数据库
show databases; #查看test数据库是否存在

导入test数据库测试
cat test.sql

use test; #指定使用test数据库
CREATE TABLE `test` (`id` int(7) NOT NULL AUTO_INCREMENT, `name` char(8) DEFAULT NULL, PRIMARY KEY (`id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8; #建表
INSERT INTO `test` VALUES (1,'test1'),(2,'test2'),(3,'test3'),(4,'test4'),(5,'test5'),(6,'test6'),(7,'test7'),(8,'test8'),(9,'test9'); #插入数据

导入数据

 mysql < test.sql 

登陆到数据库创建用户做授权查看导入数据是否成功

mysql 
MariaDB [(none)]> select * from test.test;
MariaDB [(none)]> grant all on test.* to redis@'%' identified by 'redhat';

浏览器访问

更新test数据库

MariaDB [(none)]>  update test set name='redhat' where id=1;

更新后redis无法自动同步，需要手动删除，redis才会重新获取更新

redis-cli del 1

访问

客户端访问过后，redis会将最新数据缓存到内存。

Gearman实现Mysql到Redis的数据同步

[root@server3 ~]# vim test.sql

use test;
#CREATE TABLE `test` (`id` int(7) NOT NULL AUTO_INCREMENT, `name` char(8) DEFAULT NULL, PRIMARY KEY (`id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8;
#INSERT INTO `test` VALUES (1,'test1'),(2,'test2'),(3,'test3'),(4,'test4'),(5,'test5'),(6,'test6'),(7,'test7'),(8,'test8'),(9,'test9');

DELIMITER $$
CREATE TRIGGER datatoredis AFTER UPDATE ON test FOR EACH ROW BEGIN
    SET @RECV=gman_do_background('syncToRedis', json_object(NEW.id as `id`, NEW.name as `name`)); 
  END$$
DELIMITER ;

重新导入时出现错误

[root@server3 ~]# mysql < test.sql
ERROR 1584 (42000) at line 6: Incorrect parameters in the call to stored function 'json_object'

解决：

[root@server3 ~]# unzip lib_mysqludf_json-master.zip 
[root@server3 ~]# cd lib_mysqludf_json-master/
[root@server3 lib_mysqludf_json-master]# ls
lib_mysqludf_json.c     lib_mysqludf_json.so   README.md
lib_mysqludf_json.html  lib_mysqludf_json.sql
[root@server3 lib_mysqludf_json-master]# yum install -y gcc
[root@server3 lib_mysqludf_json-master]# yum install -y mysql-devel
[root@server3 lib_mysqludf_json-master]# gcc $(mysql_config --cflags) -shared -fPIC -o lib_mysqludf_json.so lib_mysqludf_json.c

[root@server3 mysql]# cd /usr/lib64/mysql/plugin/ #查看插件

MariaDB [(none)]> show global variables like 'plugin_dir'; #对应数据库中
+---------------+--------------------------+
| Variable_name | Value                    |
+---------------+--------------------------+
| plugin_dir    | /usr/lib64/mysql/plugin/ |
+---------------+--------------------------+
1 row in set (0.00 sec)

将插件拷贝到mysql插件目录
[root@server3 plugin]# cp ~/lib_mysqludf_json-master/lib_mysqludf_json.so .

MariaDB [(none)]> CREATE FUNCTION json_object RETURNS STRING SONAME 'lib_mysqludf_json.so';

MariaDB [(none)]> select * from mysql.func;
+-------------+-----+----------------------+----------+
| name        | ret | dl                   | type     |
+-------------+-----+----------------------+----------+
| json_object |   0 | lib_mysqludf_json.so | function |
+-------------+-----+----------------------+----------+
1 row in set (0.00 sec)

server1安装gearman

[root@server1 ~]# yum install -y gearmand-1.1.12-18.el7.x86_64.rpm libgearman-devel-1.1.12-18.el7.x86_64.rpm libevent-devel-2.0.21-4.el7.x86_64.rpm libgearman-1.1.12-18.el7.x86_64.rpm 
[root@server1 ~]# systemctl start gearmand.service 
[root@server1 ~]# netstat -ntlp | grep 4730
tcp        0      0 0.0.0.0:4730            0.0.0.0:*               LISTEN      4096/gearmand       
tcp6       0      0 :::4730                 :::*                    LISTEN      4096/gearmand  



[root@server3 ~]# yum install libgearman-devel-1.1.12-18.el7.x86_64.rpm libgearman-1.1.12-18.el7.x86_64.rpm libevent-devel-2.0.21-4.el7.x86_64.rpm -y
cd gearman-mysql-udf-0.6/
[root@server3 gearman-mysql-udf-0.6]# ./configure --with-mysql --libdir=/usr/lib64/mysql/plugin
make && make install

导入成功
mysql < test.sql
mysql

CREATE FUNCTION gman_do_background RETURNS STRING SONAME 'libgearman_mysql_udf.so';
CREATE FUNCTION gman_servers_set RETURNS STRING SONAME 'libgearman_mysql_udf.so';
MariaDB [(none)]> select * from mysql.func;
+--------------------+-----+-------------------------+----------+
| name               | ret | dl                      | type     |
+--------------------+-----+-------------------------+----------+
| json_object        |   0 | lib_mysqludf_json.so    | function |
| gman_do_background |   0 | libgearman_mysql_udf.so | function |
| gman_servers_set   |   0 | libgearman_mysql_udf.so | function |
+--------------------+-----+-------------------------+----------+
MariaDB [(none)]> SELECT gman_servers_set('172.25.25.1:4370');
+--------------------------------------+
| gman_servers_set('172.25.25.1:4370') |
+--------------------------------------+
| 172.25.25.1:4370                     |
+--------------------------------------+
[root@server3 ~]# mysql < test.sql
MariaDB [(none)]> SHOW TRIGGERS FROM test;
 SET @RECV=gman_do_background('syncToRedis', json_object(NEW.id as `id`, NEW.name as `name`)); 

php添加gearman模块
[root@server1 ~]# cd gearman-1.1.2/

[root@server1 gearman-1.1.2]# phpize # 创建预编译环境，产生configure脚本

[root@server1 gearman-1.1.2]# ./configure --with-gearman
make && make install
查看是否增加了gearman.so插件
[root@server1 gearman-1.1.2]# cd /usr/lib64/php/modules/

[root@server1 gearman-1.1.2]# cd /etc/php.d/
[root@server1 php.d]# cp redis.ini gearman.ini
[root@server1 php.d]# vim gearman.ini 
extension=gearman.so
[root@server1 php.d]# systemctl restart httpd
[root@server1 php.d]# php -m | grep gearman

[root@server1 ~]# mv worker.php /usr/local/
[root@server1 ~]# nohup php /usr/local/worker.php 

测试：
更新test数据库，浏览器访问，可以看到以自动同步数据

MariaDB [(none)]> update test set name='xixi' where id=2;
[root@server2 ~]# redis-cli get 2