java日志产生器开发并结合log4j完成日志输出到控制台以及flume中

首先搭建一个maven项目，pom文件夹导入以下依赖

　　　　 <dependency>
            <groupId>org.apache.flume.flume-ng-clientsgroupId>
            <artifactId>flume-ng-log4jappenderartifactId>
            <version>1.6.0version>
        dependency>

在项目的test测试文件夹下面新建一个java文件夹和resources文件夹，并标记化

然后在java文件夹下面新建LoggerGenerator.java文件

import java.util.logging.Logger;

/**
 * 模拟日志产生
 */
public class LoggerGenerator {

    private static Logger logger = Logger.getLogger(LoggerGenerator.class.getName());

    public static void main(String[] args) throws Exception{
        int index = 0;
        while(true) {
            Thread.sleep(1000);
            logger.info("current value is : " + index++);
        }
    }
}

在resources文件夹下面新建一个log4j.properties配置文件

log4j.rootLogger=INFO,stdout,flume

log4j.appender.stdout = org.apache.log4j.ConsoleAppender
log4j.appender.stdout.target = System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss,SSS} [%t] [%c] [%p] - %m%n


log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender
log4j.appender.flume.Hostname = zq
log4j.appender.flume.Port = 41414
log4j.appender.flume.UnsafeMode = true

效果图：

修改flume的conf文件夹下面的配置文件streaming.conf

agent1.sources = avro-source
agent1.sinks = log-sink
agent1.channels = logger-channel

agent1.sources.avro-source.type = avro
agent1.sources.avro-source.bind = zq
agent1.sources.avro-source.port = 41414

agent1.channels.logger-channel.type = memory
agent1.sinks.log-sink.type = logger

agent1.sources.avro-source.channels=logger-channel
agent1.sinks.log-sink.channel=logger-channel

在这里我先在linux下运行flume程序:

flume-ng agent \
--conf $FLUME_HOME/conf \
--conf-file $FLUME_HOME/conf/streaming.conf \
--name agent1 \
-Dflume.root.logger=INFO,console

成功启动flume

这时候再在idea里面运行LoggerGenerator.java,会发现flume输出了相应信息
如下：