Debezium:mysql connector安装

---

前言：

debezium提供了多种基于kafka的连接器，方便对RDB做数据流处理，包括:MongoDB,Oracle,Mysql,SqlServer,Postgresql,可扩展性强，代码可控，本篇介绍基于mysql的安装使用

插件版本：

Kafka:CDK3.10 (相当于Kafka1.1版本)，这里需要kafka 0.10以上版本才能支持

Debezium:0.83

Mysql:5.5 （mysql5.6版本前后会有一些差异，下面会提到）

---

安装步骤：

1、mysql (这里我们安装5.5版本)

apt-get install mariadb-server-5.5

2、mysql配置

修改/etc/mysql/my.cnf,添加如下内容:

server-id = 223344 （这个id对于debezium来说一定是要唯一的）

log_bin = mysql-bin

binlog_format = ROW

#binlog_row_image = FULL (这一行如果在mysql5.6之前的版本可以不用加，5.6之后的版本要加上，主要控制binlog记录完整性，full为全写)

expire_logs_days = 10

3、重启数据库，配置用户权限

CREATE USER debezium IDENTIFIED BY 'debezium'; 

GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'debezium' IDENTIFIED BY 'debezium';

FLUSH PRIVILEGES;

关于权限的介绍：

SELECT - enables the connector to select rows from tables in databases; used only when performing a snapshot

RELOAD - enables the connector of the FLUSH statement to clear or reload various internal caches, flush tables, or acquire locks; used only when performing a snapshot

SHOW DATABASES - enables the connector to see database names by issuing the SHOW DATABASE statement; used only when performing a snapshot

REPLICATION SLAVE - enables the connector to connect to and read the binlog of its MySQL server; always required for the connector

REPLICATION CLIENT - enables the use of SHOW MASTER STATUS, SHOW SLAVE STATUS, and SHOW BINARY LOGS; always required for the connector

5、安装java环境

安装包准备：jdk-8u144-linux-x64.tar.gz

解压java安装包到指定安装文件：tar -zxvf jdk-8u144-linux-x64.tar.gz -C /usr/lib/jvm/

配置环境变量，修改/etc/profile文件

export JAVA_HOME=/usr/lib/jvm/jdk1.8.0_144

export PATH=$PATH:$JAVA_HOME/bin

export CLASSPATH=.:$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/li

使配置环境变量生效： source ／etc/profile

测试是否安装成功

java –version

6、Debezium安装

下载：weget https://repo1.maven.org/maven2/io/debezium/debezium-connector-mysql/0.8.3.Final/debezium-connector-mysql-0.8.3.Final-plugin.tar.gz

解压压缩包：tar -zxvf debezium-connector-mysql-0.8.3.Final-plugin.tar.gz

把debezium-connector-mysql下所有的jar包复制到kafka的lib下：cp *.jar /opt/cloudera/parcels/KAFKA-3.1.0-1.3.1.0.p0.35/lib/kafka/libs