flume第一讲netcat source(仅测试)

flume第一讲netcat source(仅测试)

netcat(网络端口)
利用flume的将网络数据端口号的所接受的数据,采集掉!

在虚拟就新建一个配置文件
[root@doit02 agent]# vi netcat-m-1ogger.conf

[root@doit02 agent]# vi netcat-m-1ogger.conf

# 定义这个agent中各组件的名字
a1.sources = s1
a1.channels = c1
a1.sinks = k1

# 描述和配置source组件：s1
a1.sources.s1.type = netcat
#绑定一个具体的ip或者主句名 0.0.0.0(本机所有的ip)
a1.sources.s1.bind = 0.0.0.0
a1.sources.s1.port = 8848
# source 和 channel关联
a1.sources.s1.channels = c1  

# 描述和配置channel组件，此处使用是内存缓存的方式
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000

# 描述和配置sink组件：k1
a1.sinks.k1.type = logger
# sink也要关联channel
a1.sinks.k1.channel = c1

启动这个配置文件

[root@doit02 flume-1.9.0-bin]# bin/flume-ng agent -n a1 -c conf -f \
agent/netcat-m-1ogger.conf -Dflume.root.logger=INFO,console

在这台机器开启远程连接写入数据

[root@doit02 ~]# telnet doit01 8848
Trying 192.168.133.101...
telnet: connect to address 192.168.133.101: Connection refused
[root@doit02 ~]# telnet doit02 8848
Trying 192.168.133.102...
Connected to doit02.
Escape character is '^]'.
nihaowoshishizijun
OK
woaini
OK
niaowomen
OK

接着就会在这台机器看到数据了

2020-04-21 18:37:27,663 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:95)] Event: { headers:{} body: 6E 69 68 61 6F 77 6F 73 68 69 73 68 69 7A 69 6A nihaowoshishizij }
2020-04-21 18:37:29,751 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:95)] Event: { headers:{} body: 77 6F 61 69 6E 69 0D                            woaini. }
2020-04-21 18:37:32,218 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:95)] Event: { headers:{} body: 6E 69 61 6F 77 6F 6D 65 6E 0D                   niaowomen. }

扩展
如果虚拟机没有telnet这个命令,我们搜索一下这个telnet命令,并下载.

[root@doit04 ~]# yum list | grep telnet
telnet.x86_64                            1:0.17-49.el6_10            updates    
telnet-server.x86_64                     1:0.17-49.el6_10            updates    
[root@doit04 ~]# yum -y install telnet.x86_64
Installed:
  telnet.x86_64 1:0.17-49.el6_10                                                

Complete!