canal快速搭建(我已为你近乎避掉所有坑)

按照我的步骤来:
首先我们不使用Linux自带的Mysql5.1,需要自己去安装5.7版本(要先卸载5.1),这个版本不需要自己去设置UTF-8了自带(所以不会出现乱码问题)!但5.7版本的密码设置需要自己去谷歌修改方法以及怎样设置简单的密码(也可去看我的其他博文)。
我选择的是canal.deployer-1.1.1.tar.gz(之前的版本配置文件似乎有的地方不一样)
jdk1.8
centOS6.9

1.开启mysql的binlog功能:
vim /etc/my.cnf
尾部添加

#binlog数据中包含server_id,标识该数据是由那个server同步过来的
server-id = 1
#mysql向文件名前缀添加数字后缀来按顺序创建二进制日志文件如mysql-binlog.000006 
log-bin = /var/lib/mysql/mysql-bin
#选择基于行的日志记录方式
binlog_format = ROW

用show variables like ‘bin_log%’;
log_bin为ON即可

2.MySQL 里配置Canal Server权限:
#一定要加SHOW VIEW这个东西,不然后面报错

CREATE USER 'canal'@'%' IDENTIFIED BY 'canal';
GRANT SHOW VIEW, SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%';
FLUSH PRIVILEGES;

3.下载canal的压缩包并解压:
官网https://github.com/alibaba/canal/releases
下载 canal.deployer-1.1.1.tar.gz
mkdir /usr/local/canal
下载好的压缩包在上面的目录中解压
tar –zxvf canal.deployer-1.1.1.tar.gz

4.配置修改
在解压好的目录下
vim conf/example/instance.properties

修改这三个地方即可

# mysql serverId,随便设置
canal.instance.mysql.slaveId = 1234

#position info,需要改成自己的数据库信息
canal.instance.master.address = 127.0.0.1:3306

#一定要注释掉下面这个参数,这样就会扫描全库
#canal.instance.defaultDatabaseName = test

5.启动:
sh bin/startup.sh
查看日志:
vim logs/canal/canal.log
有下面的这三句话即表示成功

[main] INFO  com.alibaba.otter.canal.deployer.CanalLauncher - ## start the canal server.
[main] INFO  com.alibaba.otter.canal.deployer.CanalController - ## start the canal server[10.1.29.120:11111]
[main] INFO  com.alibaba.otter.canal.deployer.CanalLauncher - ## the canal server is running now ......

可能第一次打开的时候日志里会出现com.alibaba.druid.pool.DruidDataSource - testWhileIdle is true, validationQuery not set这个错误,不用管,千万别去改其他配置,第二次看就会没有了,一定能启动成功。

6.建一个java程序验证是否搭建好了:
首先第一次我们要:vim conf/canal.properties
找到canal.instance.parser.parallelThreadSize = 16
把这一行的注释符号去掉(java才能成功运行)
然后重启canal:
sh bin/restart.sh

Java maven项目:
pom.xml:


    com.alibaba.otter
    canal.client
    1.1.1

SimpleClient.java:

import java.net.InetSocketAddress;
import java.util.List;
import java.util.stream.Collectors;

import com.alibaba.otter.canal.client.CanalConnector;
import com.alibaba.otter.canal.client.CanalConnectors;
import com.alibaba.otter.canal.protocol.CanalEntry.Column;
import com.alibaba.otter.canal.protocol.CanalEntry.Entry;
import com.alibaba.otter.canal.protocol.CanalEntry.EntryType;
import com.alibaba.otter.canal.protocol.CanalEntry.RowChange;
import com.alibaba.otter.canal.protocol.CanalEntry.RowData;
import com.alibaba.otter.canal.protocol.Message;

public class SimpleClient {

    public static void main(String[] args) throws Exception {
        CanalConnector connector = CanalConnectors.newSingleConnector(
                new InetSocketAddress("192.168.56.230", 11111), "example", "", "");

        connector.connect();
        connector.subscribe(".*\\..*");
        connector.rollback();

        while (true) {
            Message message = connector.getWithoutAck(100);
            long batchId = message.getId();
            if (batchId == -1 || message.getEntries().isEmpty()) {
                System.out.println("sleep");
                Thread.sleep(1000);
                continue;
            }
            printEntries(message.getEntries());
            connector.ack(batchId);
        }
    }

    private static void printEntries(List entries) throws Exception {
        for (Entry entry : entries) {
            if (entry.getEntryType() != EntryType.ROWDATA) {
                continue;
            }

            RowChange rowChange = RowChange.parseFrom(entry.getStoreValue());
            for (RowData rowData : rowChange.getRowDatasList()) {
                switch (rowChange.getEventType()) {
                case INSERT:
                case UPDATE:
                    System.out.print("UPSERT ");
                    printColumns(rowData.getAfterColumnsList());

                    if ("retl_buffer".equals(entry.getHeader().getTableName())) {
                        String tableName = rowData.getAfterColumns(1).getValue();
                        String pkValue = rowData.getAfterColumns(2).getValue();
                        System.out.println("SELECT * FROM " + tableName + " WHERE id = " + pkValue);
                    }
                    break;

                case DELETE:
                    System.out.print("DELETE ");
                    printColumns(rowData.getBeforeColumnsList());
                    break;

                default:
                    break;
                }
            }
        }
    }

    private static void printColumns(List columns) {
        String line = columns.stream()
                .map(column -> column.getName() + "=" + column.getValue())
                .collect(Collectors.joining(","));
        System.out.println(line);
    }

}

当在mysql里面进行delete操作时如下,根据程序没操作时输出sleep,当在mysql里执行一个delete指令,则输出:

在这里插入图片描述

你可能感兴趣的:(canal)