flume 1.9 源码阅读(二)

Flume NG 启动脚本分析

启动 flume-ng-agent

/root/flume/bin/flume-ng agent --name agent1 -c conf -f flume-conf.conf

配置文件如下：

agent1.sources = source1
agent1.channels = channel1
agent1.sinks = sink1

agent1.sources.source1.type = exec
agent1.sources.source1.channels = channel1
agent1.sources.source1.command = tail -F /root/flume/test.log

agent1.channels.channel1.type = memory
agent1.channels.channel1.capacity = 1000
agent1.channels.channel1.transactionCapacity = 100

agent1.sinks.sink1.type = logger

agent1.sinks.sink1.channel = channel1

[root@localhost conf]# /root/flume/bin/flume-ng agent --name agent1 -c conf -f flume-conf.conf^C
[root@localhost conf]# cat flume-conf.conf 

agent1.sources = source1
agent1.channels = channel1
agent1.sinks = sink1

agent1.sources.source1.type = exec
agent1.sources.source1.channels = channel1
agent1.sources.source1.command = tail -F /root/flume/test.log

agent1.channels.channel1.type = memory
agent1.channels.channel1.capacity = 1000
agent1.channels.channel1.transactionCapacity = 100

agent1.sinks.sink1.type = logger

agent1.sinks.sink1.channel = channel1

这里要注意的是agent的名字要和启动脚本中的agent --name参数一致，我们这里是 agent1, 这个配置文件的意思是采集 /root/flume/test.log 这个文件，并将其打印到控制台，注意需要修改一下conf目录下 log4j.properties 将其重定向到 console。

首先看一下 flume-ng.sh 文件, 其中下面这个函数就是 flume 执行的

run_flume() {
  local FLUME_APPLICATION_CLASS

  if [ "$#" -gt 0 ]; then
    FLUME_APPLICATION_CLASS=$1
    shift
  else
    error "Must specify flume application class" 1
  fi

  if [ ${CLEAN_FLAG} -ne 0 ]; then
    set -x
  fi
  $EXEC $JAVA_HOME/bin/java $JAVA_OPTS $FLUME_JAVA_OPTS "${arr_java_props[@]}" -cp "$FLUME_CLASSPATH" \
      -Djava.library.path=$FLUME_JAVA_LIBRARY_PATH "$FLUME_APPLICATION_CLASS" $*
}

简单的来讲就是解析各种参数最终执行了 java -cp org.apache.flume.node.Application $* 这个命令。

可以看到 org.apache.flume.node.Application 这个就是整个flume-ng-agent的入口类。