logstash kafka output 输出原始数据格式

有个场景需要从kafka topic中读出message,然后转到另一个kakfa集群的topic中,为省事就用了logstash。
配置如下:

input {
  kafka {
    bootstrap_servers => 'client1:9092,client2:9092'
    group_id => 'mygroup'
    topics => ["topic-info"]
    codec => 'plain'
  }
}
output {
  #stdout { codec => rubydebug }
  kafka {
    bootstrap_servers => 'client1:9092,client2:9092'
    topic_id => 'topic-out'
    codec => plain 
  }
}

但消息输出:

2017-11-08T12:14:39.116Z %{host} your_origin_message...

前面加了时间和host,尝试后发现需要添加codec plain的message格式,修改后配置:

input {
  kafka {
    bootstrap_servers => 'client1:9092,client2:9092'
    group_id => 'mygroup'
    topics => ["topic-info"]
    codec => 'plain'
  }
}
output {
  #stdout { codec => rubydebug }
  kafka {
    bootstrap_servers => 'client1:9092,client2:9092'
    topic_id => 'topic-out'
    codec => plain {
      format => "%{message}"
    }
  }
}

修改后结果:
your_origin_message...

你可能感兴趣的:(杂项)