centos:go-mysql-elasticsearch实现mysql 与elasticsearch实时同步

缺点：删除数据-同步不支持

步骤1：安装go

yum install go

步骤2：安装godep

go get github.com/tools/godep

centos下载不下来，我采取了另一种方法

1. 先从 https://github.com/tools/godep 下载下来
   放到
   /root/go/src/github.com/tools 下面

cd godep
go install ./    #该命令会将 godep 生成到go 文件夹里面的bin目录下

2.设置环境变量
vi /etc/profile

export GOPATH=/root/go
export GOBIN=/root/go/bin
export PATH=$PATH:$GOPATH/bin

source /etc/profile //生效文件

然后运行 godep 发现可以输出一些命令，成功

还有一种就是gomod 现在都是gomod 我也没试过 可以试试

下载地址 https://github.com/siddontang/go-mysql-elasticsearch

或者通过git

git clone https://github.com/siddontang/go-mysql-elasticsearch

cd go-mysql-elasticsearch
 make

make 报错

go: github.com/BurntSushi/[email protected]: reading github.com/BurntSushi/toml/go.mod at revision v0.3.1: unknown revision v0.3.1
make: *** [build-elasticsearch] Error 1

直接执行代理：

go env -w GO111MODULE=on
go env -w GOPROXY=[https://goproxy.io,direct](https://links.jianshu.com/go?to=https%3A%2F%2Fgoproxy.io%2Cdirect)

vi etc/river.toml

# MySQL address, user and password
 # user must have replication privilege in MySQL.
 my_addr = "127.0.0.1:3306"     //需要同步的mysql基本设置
 my_user = "root"
  my_pass = "root"
  
  # Elasticsearch address
  es_addr = "127.0.0.1:9200"     //本地elasticsearch配置
  
 # Path to store data, like master.info, and dump MySQL data 
 data_dir = "./var"             //数据存储的url
  //以下配置保存默认不变
 # Inner Http status address
 stat_addr = "127.0.0.1:12800"
 
 # pseudo server id like a slave 
 server_id = 1001
 
 # mysql or mariadb
 flavor = "mysql"
 # mysqldump execution path
 mysqldump = "mysqldump"
 
 # MySQL data source
 [[source]]
 schema = "hoojjack"             //elasticsearch 与 mysql 同步时对应的数据库名称
 
 # Only below tables will be synced into Elasticsearch.
 # "test_river_[0-9]{4}" is a wildcard table format, you can use it if you have many sub tables, like table_0000 - table_1023
 # I don't think it is necessary to sync all tables in a database.
 tables = ["test_river", "test_river_[0-9]{4}"]    //支持通配符，可以指定只复制hoojjack数据库中指定的表数据
 
 # Below is for special rule mapping
 [[rule]]
 schema = "hoojjack"    //数据库名称
 table = "test_river"   //表名称
 index = "river"        //对应的索引名称
 # title is MySQL test_river field name, es_title is the customized name in Elasticsearch
   [rule.field]
 # This will map column title to elastic search my_title
   title="es_title"     //将可以将mysql的某个属性对应指向elasticsearch的某个field, 如test_river的titile属性对应es_title
 # This will map column tags to elastic search my_tags and use array type
   tags="my_tags,list"
 # This will map column keywords to elastic search keywords and use array type
   keywords=",list"
 
 # wildcard table rule, the wildcard table must be in source tables 
   [[rule]]
   schema = "hoojjack"
   table = "test_river_[0-9]{4}"
   index = "river"
   type = "river"
 
 # title is MySQL test_river field name, es_title is the customized name in Elasticsearch
   [[rule.fields]]
   mysql = "title"
   elastic = "es_title"

执行go-mysql-elasticsearch同步，首先进入 go-mysql-elasticsearch 路径，
cd /root/go/src/github.com/tools/go-mysql-elasticsearch

然后执行  ./bin/go-mysql-elasticsearch -config=./etc/river.toml

后台启用

nohup ./bin/go-mysql-elasticsearch ./etc/river.toml 1>info.log 2>&1 &

报错:canal dump mysql err:
CentOS7开启MySQL binlog日志
vim /etc/my.cnf

#添加
log-bin=mysql-bin
server-id=1

重启mysql

service mysqld restart

验证

mysql -uroot -p
然后输入 password
show variables like '%log_bin%';
![image.png](https://upload-images.jianshu.io/upload_images/19694511-bea6177920758a59.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)