Flume-NG 使用首记

第一步，配置Cloudera-CDH4软件源，见Cloudera官网文档。略……

第二步，安装Flume客户端

yum -y install flume-ng

第三步，配置Flume

    1、Flume使用经典的alternatives进行多版本管理，查看/添加配置路径：

# alternatives --display flume-ng-conf
flume-ng-conf - status is auto.
link currently points to /etc/flume-ng/conf.empty
/opt/cloudera/parcels/CDH-4.2.0-1.cdh4.2.0.p0.10/etc/flume-ng/conf.empty - priority 10
/etc/flume-ng/conf.empty - priority 30
Current `best' version is /etc/flume-ng/conf.empty.

    2、使用默认配置

cp –fv flume-conf.properties.template flume-conf.properties

    3、更新客户端配置文件 /etc/flume-ng/conf/flume.conf ：

# 定义Agent1对应的属性名称
agent1.sources = ngrinder
agent1.sinks = hdfs4log
agent1.channels = memory4log
# 定义日志的来源，下例为Ngrinder的启动日志
agent1.sources.ngrinder.type = exec
agent1.sources.ngrinder.command = tail -F /data/log/ngrinder/startup.log
agent1.sources.ngrinder.channels = momery01
# 指定日志存储的目标地址，下例为直接写到HDFS中
agent1.sinks.hdfs4log.type = hdfs
agent1.sinks.hdfs4log.hdfs.path = hdfs://hdfs.kisops.org:8020/flume/ngrinder
agent1.sinks.hdfs4log.channel = momery4log
# 定义日志缓冲区，方便在网络阻塞时日志可以延时输出
agent1.channels.memory4log.type = memory
agent1.channels.memory4log.capacity = 1000
agent1.channels.memory4log.transactionCapacity = 100

    4、启动Flume客户端：

su -m -c 'flume-ng agent -n agent1 -f /etc/flume-ng/conf/flume.conf' hdfs

    5、验证：重启Ngrinder；查看HDFS对应目录是否有数据生成。

    6、测试环境

# flume-ng version
Flume 1.3.0-cdh4.2.1
Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git
Revision: 68867f41ba0906cc4050efade51afca563f0e65b
Compiled by jenkins on Mon Apr 22 13:07:14 PDT 2013
From source with checksum a341b20667cda79e8561f388a94a5168

# uname -a

Linux CentOS6.4 2.6.32-358.2.2.el6.x86_64