mysql-maxwell binlog订阅
原文地址：
http://maxwells-daemon.io/quickstart/
测试经过和结果：
1、mysql安装略过
2、配置mysql,并开启bin_log
# vi /etc/my.cnf
[mysqld]
server-id=1
log-bin=master
binlog_format=row
**需要重启mysql
声明：通过
mysql --help | grep my.cnf查看，可知 mysql配置文件的加载顺序：
/etc/my.cnf /etc/mysql/my.cnf /usr/etc/my.cnf ~/.my.cnf
3、mysql 权限配置，用户名：
maxwell，密码：123456
mysql> set global binlog_row_image=FULL;
mysql> GRANT ALL on maxwell.* to 'maxwell'@'%' identified by '123456';mysql> GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE on *.* to 'maxwell'@'%';
mysql> GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE on *.* to 'maxwell'@'localhost' identified by '123456';
mysql> GRANT ALL on maxwell.* to 'maxwell'@'localhost';
注意：开启bin_log 监控权限
4、测试 maxwell;
#
bin/maxwell --user='maxwell' --password='123456' --host='127.0.0.1' --producer=stdout
错误：
提示显示为 bin_log没打开，重启mysql即可
创建表：
mysql> use test; // 切换数据库
mysql> create table t1(id varchar(10),name varchar(20));
maxwell控制台：
15:19:14,042 INFO AbstractSchemaStore - storing schema @BinlogPosition[master.000001:21162] after applying "create table t1(id varchar(10),name varchar(20))"
mysql> insert into t1(id,name) values('1','haha');
maxwell控制台：
{"database":"test","table":"t1","type":"insert","ts":1469776891,"xid":697,"commit":true,"data":{"id":"1","name":"haha"}}
格式化：
{
"database": "test",
"table": "t1",
"type": "insert",
"ts": 1469776891,
"xid": 697,
"commit": true,
"data": {
"id": "1",
"name": "haha"
}
}
批量修改mysql的数据：
update t1 set name = 'fdafdsa'; //总有有三条数据
得到的结果：
{
"database": "test",
"table": "t1",
"type": "update",
"ts": 1469779091,
"xid": 4861,
"data": {
"id": "1",
"name": "fdafdsa"
},
"old": {
"name": "haha"
}
}
删除数据：
mysql> delete from t1; //共有三条数据
maxwell 收到的数据：
{"database":"test","table":"t1","type":"delete","ts":1469782623,"xid":11929,"data":{"id":"1","name":"fdafdsafdsafdsa"}}
{"database":"test","table":"t1","type":"delete","ts":1469782623,"xid":11929,"data":{"id":"2","name":"fdafdsafdsafdsa"}}
{"database":"test","table":"t1","type":"delete","ts":1469782623,"xid":11929,"commit":true,"data":{"id":"3","name":"fdafdsafdsafdsa"}}
5、maxwell to kafka
a) 创建 topic ：
bin/kafka-topics.sh --create --zookeeper 10.32.30.225:2181,10.32.30.226:2181,10.32.30.227:2181 --replication-factor 2 --partitions 1 --topic
maxwell
b） 控制台消费者：
bin/kafka-console-consumer.sh --zookeeper 10.32.30.225:2181,10.32.30.226:2181,10.32.30.227:2181 --topic
maxwell
c) bin/maxwell --user='maxwell' --password='123456' --host='127.0.0.1' --producer=kafka --kafka.bootstrap.servers=
10.32.30.225:9092,10.32.30.226:9092,10.32.30.227:9092,10.32.30.228:9092
默认的topic 为
maxwell
对表进行CRUD....
6、mysql主从设置
主库： 10.32.30.226
从库： 10.32.30.227
6.1、从库同步主库（略）
6.2、从库bin_log 配置
# cat /etc/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=227
binlog_format=row
测试：启动maxwell，监控slave的bin_log
主库：执行insert ，从库同步到数据，但是maxwell未收到数据
原因分析：bin_log的产生默认情况下，不是链式的
解决方案：
处理方法：修改/etc/my.cnf，增加一行log_slave_updates=1，重启数据库；
主库执行instert ，maxwell 收到数据！