Flume应用案例之两台服务器之间进行实时数据采集

紧接我的上一篇博客　Flume应用案例之监控一个文件实时采集新增的数据输出到控制台
虚拟机啥的我先不搭了，就在一台服务器上演示
技术选型：
exec source + memory channel + avro sink
avro source + memory channel + logger sink

所以需要两个配置文件：
在conf目录下创建exec-memory-avro.conf和avro-memory-logger.conf文件

exec-memory-avro.conf

exec-memory-avro.sources = exec-source
exec-memory-avro.sinks = avro.sink
exec-memory-avro.channels = memory-channel

exec-memory-avro.sources.exec-source.type = exec
exec-memory-avro.sources.exec-source.command = tail -F /home/zq/data/data.log
exec-memory-avro.sources.exec-source.shell = /bin/sh -c

exec-memory-avro.sinks.avro.sink.type = avro
exec-memory-avro.sinks.avro.sink.hostname = zq
exec-memory-avro.sinks.avro.sink.port = 44444

exec-memory-avro.channels.memory-channel.type = memory

exec-memory-avro.sources.exec-source.channels = memory-channel
exec-memory-avro.sinks.avro.sink.channel = memory-channel

avro-memory-logger.conf

avro-memory-logger.sources = avro-source
avro-memory-logger.sinks = logger.sink
avro-memory-logger.channels = memory-channel

avro-memory-logger.sources.avro-source.type = avro
avro-memory-logger.sources.avro-source.bind = zq
avro-memory-logger.sources.avro-source.port = 44444

avro-memory-logger.sinks.logger.sink.type = logger

avro-memory-logger.channels.memory-channel.type = memory

avro-memory-logger.sources.avro-source.channels = memory-channel
avro-memory-logger.sinks.logger-sink.channel = memory-channel

测试:
1．新开一个控制台，进行ssh连接

２．在当前控制台先启动avro-memory-logger.conf，再启动exec-memory-avro.conf

flume-ng agent \
--name avro-memory-logger \
--conf $FLUME_HOME/conf \
--conf-file $FLUME_HOME/conf/avro-memory-logger.conf \
-Dflume.root.logger=INFO,console

flume-ng agent \
--name exec-memory-avro \
--conf $FLUME_HOME/conf \
--conf-file $FLUME_HOME/conf/exec-memory-avro.conf \
-Dflume.root.logger=INFO,console

再开一个终端，前两个中有一个会响应消息