flume 示例一收集tomcat日志

例子场景描述:将tomcat的日志收集到指定的目录,tomcat 安装在/opt/tomcat, 日志存放在var/log/data。

配置tomcat.conf 如下:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# A single-node Flume configuration
 
# Name the components on this agent
 
agent1.sources = source1
agent1.sinks = sink1
agent1.channels = channel1
 
  
 
# Describe/configure source1
 
agent1.sources.source1.type = exec
agent1.sources.source1.command = tail -n +0 -F /opt/tomcat/logs/ catalina.out
agent1.sources.source1.channels = channel1
 
 
# Describe sink1
agent1.sinks.sink1.type = file_roll
agent1.sinks.sink1.sink.directory=/ var /log/data
 
# Use a channel which buffers events in memory
agent1.channels.channel1.type = file
agent1.channels.channel1.checkpointDir=/ var /checkpoint
agent1.channels.channel1.dataDirs=/ var /tmp
agent1.channels.channel1.capacity = 1000
agent1.channels.channel1.transactionCapactiy = 100
  
 
# Bind the source and sink to the channel
agent1.sources.source1.channels = channel1
agent1.sinks.sink1.channel = channel1

运行agent 

?
1
bin/flume-ng agent --conf conf --conf-file tomcat.conf --name agent1 -Dflume.root.logger=INFO,console

agent自动会将日志收集到指定的var/log/data

注意:hadoop-core-1.0.1.jar 放在flume 的lib目录中。否则报ClassNotFoundException
通过以上例子可以推广到收集apache server 、nginx日志

你可能感兴趣的:(flume)