RedHat6.5安装单机flume1.6

版本号：

RedHat6.5   JDK1.8   apache-flume-1.6.0

1.apache-flume-1.6.0-bin.tar.gz 下载

官网下载地址：http://archive.apache.org/dist/flume/1.6.0/apache-flume-1.6.0-bin.tar.gz

百度云盘下载地址：链接：http://pan.baidu.com/s/1pKPah0f 密码：ovbe

2.安装

新建一个/usr/local/flume目录，上传apache-flume-1.6.0-bin.tar.gz，解压：

mkdir /usr/local/flume
tar -xvf  /usr/local/flume/apache-flume-1.6.0-bin.tar.gz

如图：

3.配置 /etc/profile

sudo gedit /etc/profile

1. #set flume environment
2. export FLUME_HOME=/usr/local/flume/apache-flume-1.6.0-bin
3. export FLUME_CONF_DIR=$FLUME_HOME/conf
4. export PATH=$FLUME_HOME/bin:$PATH

source /etc/profile

4.配置flume JAVA_HOME

把flume-env.sh.template重命名为flume-env.sh，往里添加

cd /usr/local/flume/apache-flume-1.6.0-bin/conf

cp flume-env.sh.template flume-env.sh

sudo gedit flume-env.sh

1. export JAVA_HOME=/usr/local/java/jdk1.8

检验

/usr/local/flume/apache-flume-1.6.0-bin/bin/flume-ng version

1.  [root@master conf]# /usr/local/flume/apache-flume-1.6.0-bin/bin/flume-ng version
2. Flume 1.6.0
3. Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git
4. Revision: 2561a23240a71ba20bf288c7c2cda88f443c2080
5. Compiled by hshreedharan on Mon May 11 11:15:44 PDT 2015
6. From source with checksum b29e416802ce9ece3269d34233baf43f
7. [root@master conf]# 

5. 简单测试示例

5.1 添加配置文件

1. touch /usr/local/flume/apache-flume-1.6.0-bin/conf/flume.conf
2. sudo gedit /usr/local/flume/apache-flume-1.6.0-bin/conf/flume.conf

flume.conf输入以下内容：

1. # 指定Agent的组件名称
2. agent1.sources = source1
3. agent1.sinks = sink1
4. agent1.channels = channel1
5.  
6. # 指定Flume source(要监听的路径)
7. agent1.sources.source1.type = spooldir
8. agent1.sources.source1.spoolDir= /usr/local/flume/logtest
9.  
10. # 指定Flume sink
11. agent1.sinks.sink1.type = logger
12.  
13. # 指定Flume channel
14. agent1.channels.channel1.type = memory
15. agent1.channels.channel1.capacity = 1000
16. agent1.channels.channel1.transactionCapacity = 100
17.  
18. # 绑定source和sink到channel上
19. agent1.sources.source1.channels = channel1
20. agent1.sinks.sink1.channel = channel1

在/usr/local/flume目录下新建一个logtest目录：

mkdir /usr/local/flume/logtest

5.2 启动flume agent

1. cd /usr/local/flume/apache-flume-1.6.0-bin
2. bin/flume-ng agent --conf conf --conf-file conf/flume.conf --name agent1 -Dflume.root.logger=INFO,console

参数作用举例

–conf 或 -c 指定配置文件夹，包含flume-env.sh和log4j的配置文件–conf conf

–conf-file 或 -f 配置文件地址–conf-file conf/flume.conf

–name 或 -n agent名称    –name agent1

-z zookeeper连接字符串-z zkhost:2181,zkhost1:2181

-p zookeeper中的存储路径前缀-p /flume 

5.3 写入日志内容

在/usr/local/flume目录下面新建一个testFlume.log日志文件，写入Hello Flume.作为测试内容：

1. touch /usr/local/flume/testFlume.log
2. sudo gedit /usr/local/flume/testFlume.log

 

 

然后拷贝到flume监听路径/usr/local/flume/logtest下：

1. cp /usr/local/flume/testFlume.log /usr/local/flume/logtest

接着就可以在前一个终端看到刚刚采集的内容了,如下：

1. 2017-07-07 21:15:59,063 (pool-3-thread-1) [INFO - org.apache.flume.client.avro.ReliableSpoolingFileEventReader.rollCurrentFile(ReliableSpoolingFileEventReader.java:348)] Preparing to move file /usr/local/flume/logtest/testFlume.log to /usr/local/flume/logtest/testFlume.log.COMPLETED
2. 2017-07-07 21:16:00,841 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 48 65 6C 6C 6F 20 46 6C 75 6D 65 2E             Hello Flume. }

 

Flume成功安装，666！