logstash通过Logstash-input-jdbc把mysql数据实时同步到ElasticSearch7.1

1. 安装logstash
2. 安装logstash-input-jdbc

   ./bin/logstash-plugin install logstash-input-jdbc

3. 在  /logstash-core/lib/jars下加入mysql-connector-java-8.0.17.jar

   链接：https://pan.baidu.com/s/1Nr1ooSuFxpb8uoiYIu22rw 
   提取码：115t

    

4. 配置文件(在logstash目录下)

   input {
     jdbc {
       jdbc_driver_library => "/home/soft/logstash/logstash-7.1.0/logstash-core/lib/jars/mysql-connector-java-8.0.17.jar"
       jdbc_driver_class => "com.mysql.jdbc.Driver"
       jdbc_connection_string => "jdbc:mysql://localhost:3306/gy-account"
       jdbc_user => "root"
       jdbc_password => "Timo*123"
       #启用追踪，如果为true，则需要指定tracking_column
       use_column_value => true
       #追踪的字段
       tracking_column => "id"
       #追踪字段的类型，目前只有数字(numeric)和时间类型(timestamp)，默认是数字类型
       tracking_column_type => "numeric"
       #记录最后一次运行的结果
       record_last_run => true
       #上面运行结果的保存位置
       last_run_metadata_path => "/home/soft/logstash/logstash-7.1.0/jdbc-position.txt"
       statement => "SELECT * FROM user where id >:sql_last_value;"
       schedule => " * * * * * *"
     }
   }
   output {
     elasticsearch {
       document_id => "%{id}"
       document_type => "_doc"
       index => "users"
       hosts => ["http://localhost:9200"]
     }
     stdout{
       codec => rubydebug
     }
   }

    

5. 执行改配置文件

   sudo ./logstash -f mysql.conf