flume实时抽取mysql到hdfs
最近有个项目可能需要异库同步，于是就上网搜了下方法，发现通过flume到hdfs可以做到准实时，照网上的方法试了发现不行，自己又去github看了下源码，这才行
附上源码地址
我用的是Flume 1.6.0-cdh5.16.1
以下是我的flume agent配置，配置信息以上链接中也有详细说明
# The configuration file needs to define the sources,
# the channels and the sinks.
# Sources, channels and sinks are defined per agent,
# in this case called 'agent'
test.sources = sqls
test.channels = c1
test.sinks = hdfssink
# For each one of the sources, the type is defined
test.sources.sqls.type = org.keedio.flume.source.SQLSource
test.sources.sqls.hibernate.connection.url = jdbc:mysql://localhost:3306/testflume
test.sources.sqls.hibernate.connection.user = root
test.sources.sqls.hibernate.connection.password = ~!@Mysql@2018
test.sources.sqls.hibernate.connection.autocommit = true
test.sources.sqls.hibernate.connection.provider_class = org.hibernate.connection.C3P0ConnectionProvider
test.sources.sqls.hibernate.connection.driver_class = com.mysql.jdbc.Driver
test.sources.sqls.hibernate.c3p0.min_size=1
test.sources.sqls.hibernate.c3p0.max_size=10
test.sources.sqls.hibernate.dialect = org.hibernate.dialect.MySQL5Dialect
test.sources.sqls.max.rows = 1000000
test.sources.sqls.table = test
test.sources.sqls.columns.to.select = *
test.sources.sqls.column.name = id
test.sources.sqls.incremental.value = 0
test.sources.sqls.delimiter.entry = ,
test.sources.sqls.enclose.by.quotes = false
test.sources.sqls.run.query.delay=5000
test.sources.sqls.status.file.path = /data/flume
test.sources.sqls.status.file.name = sqls.status
# The channel can be defined as follows.
test.sources.sqls.channels = c1
# Each sink's type must be defined
test.sinks.hdfssink.channel = c1
test.sinks.hdfssink.type = hdfs
test.sinks.hdfssink.hdfs.path = hdfs://nameservice1:8020/flume/mysqls
test.sinks.hdfssink.hdfs.fileType = DataStream
test.sinks.hdfssink.hdfs.writeFormat = Text
test.sinks.hdfssink.hdfs.rollSize = 268435456
test.sinks.hdfssink.hdfs.rollInterval = 7200
test.sinks.hdfssink.hdfs.rollCount = 0
# Each channel's type is defined.
test.channels.c1.type = memory
以下是对一些属性的说明
| 属性 | 说明 |
|---|---|
| .hibernate.connection.url | 数据库连接地址 |
| hibernate.connection.user | 用户名 |
| hibernate.connection.password | 密码 |
| hibernate.connection.autocommit | 是否自动提交 |
| table | 想要获取的表名 |
| columns.to.select | 想要获取的列，默认是* |
| column.name | 索引字段 |
| delimiter.entry | 分隔符 |
| query.delay | 查询间隔 |
| enclose.by.quotes | 是否加引号 |
| status.file.path | 状态文件路径 |
| status.file.name | 状态文件名称 |
| custom.query | 自定义查询语句，使用需谨慎，详情请查看源码地址 |
先去上面给的github地址中下载源码，使用IDEA打开
然后将项目打包，将所有用到的jar包放到flume的lib目录下面，我的目录是/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/lib/，切记要所有jar包，不然会有各种错误！
对了有个问题要说明下，cdh安装的flume只要开启了就是默认agent一直开启的，所有我们把它关掉
不使用配置中的properties，我们自己创一个，我在/data下面创了个agent的配置信息文件
除此之外我们还需要创建个状态文件目录，存放标识上次抽取数据到什么位置了的状态文件，
我的是在/data/flume/下面，然后chmod 777 /data/flume
sink我用的是hdfs sink，所以在hdfs上建一个目录存放接收的文件，我建的是/flume/mysqls，一样hdfs dfs -chmod 777 /flume/mysqls
然后看看source端，在testflume库下面有张test表
我们开启flume agent
[root@mf04 ~]# flume-ng agent -c /opt/cloudera/parcels/CDH/lib/flume-ng/lib/ -f /data/test.properties -n test
Info: Including Hadoop libraries found via (/bin/hadoop) for HDFS access
Info: Including HBASE libraries found via (/bin/hbase) for HBASE access
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
Info: Including Hive libraries found via () for Hive access
+ exec /usr/local/java/jdk1.8.0_191/bin/java -Xmx20m -cp '/opt/cloudera/parcels/CDH/lib/flume-ng/lib:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/lib/*:/etc/hadoop/conf:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/libexec/../../hadoop/lib/*:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/libexec/../../hadoop/.//*:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/libexec/../../hadoop-hdfs/./:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/libexec/../../hadoop-hdfs/lib/*:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/libexec/../../hadoop-hdfs/.//*:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/libexec/../../hadoop-yarn/lib/*:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/libexec/../../hadoop-yarn/.//*:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/*:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//*:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../conf:/usr/local/java/jdk1.8.0_191/lib/tools.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/..:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/activation-1.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/apacheds-i18n-2.0.0-M15.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/api-asn1-api-1.0.0-M20.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/api-util-1.0.0-M20.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/avro.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/aws-java-sdk-bundle-1.11.134.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-beanutils-1.9.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-beanutils-core-1.8.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-codec-1.9.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-collections-3.2.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-configuration-1.6.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-daemon-1.0.13.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-digester-1.8.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-el-1.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-httpclient-3.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-logging-1.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-math-2.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-math3-3.1.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/commons-net-3.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/core-3.1.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/curator-client-2.7.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/curator-framework-2.7.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/curator-recipes-2.7.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/disruptor-3.3.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/findbugs-annotations-1.3.9-1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/gson-2.2.4.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/guava-12.0.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hamcrest-core-1.3.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-annotations-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-annotations-1.2.0-cdh5.16.1-tests.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-client-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-common-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-common-1.2.0-cdh5.16.1-tests.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-examples-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-external-blockcache-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-hadoop2-compat-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-hadoop2-compat-1.2.0-cdh5.16.1-tests.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-hadoop-compat-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-hadoop-compat-1.2.0-cdh5.16.1-tests.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-it-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-it-1.2.0-cdh5.16.1-tests.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-prefix-tree-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-procedure-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-protocol-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-resource-bundle-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-rest-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-rsgroup-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-rsgroup-1.2.0-cdh5.16.1-tests.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-server-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-server-1.2.0-cdh5.16.1-tests.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-shell-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-spark-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hbase-thrift-1.2.0-cdh5.16.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/high-scale-lib-1.1.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/hsqldb-1.8.0.10.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/htrace-core-3.2.0-incubating.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/htrace-core4-4.0.1-incubating.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/htrace-core.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jackson-annotations-2.2.3.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jackson-core-2.2.3.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jackson-core-asl-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jackson-databind-2.2.3-cloudera.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jackson-jaxrs-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jackson-mapper-asl-1.8.10-cloudera.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jackson-xc-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jamon-runtime-2.4.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jasper-compiler-5.5.23.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jasper-runtime-5.5.23.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/java-xmlbuilder-0.4.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jaxb-api-2.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jcodings-1.0.8.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jersey-client-1.9.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jersey-json-1.9.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jets3t-0.9.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jettison-1.3.3.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jetty-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jetty-sslengine-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jetty-util-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/joni-2.1.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jruby-cloudera-1.0.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jsch-0.1.42.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jsp-2.1-6.1.14.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jsp-api-2.1-6.1.14.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/jsp-api-2.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/junit-4.12.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/leveldbjni-all-1.8.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/libthrift-0.9.3.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/metrics-core-2.2.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/netty-all-4.0.50.Final.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/paranamer-2.3.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/servlet-api-2.5-6.1.14.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/slf4j-api-1.7.5.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/slf4j-log4j12.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/snappy-java-1.0.4.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/spymemcached-2.11.6.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/xmlenc-0.52.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/xz-1.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/zookeeper.jar:/etc/hadoop/conf:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/libexec/../../hadoop/lib/*:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/libexec/../../hadoop/.//*:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/libexec/../../hadoop-hdfs/./:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/libexec/../../hadoop-hdfs/lib/*:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/libexec/../../hadoop-hdfs/.//*:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/libexec/../../hadoop-yarn/lib/*:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/libexec/../../hadoop-yarn/.//*:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/*:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//*:/etc/hadoop/conf/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/bin/../lib/hadoop/*:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/bin/../lib/hadoop/lib/*:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/bin/../lib/zookeeper/*:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/bin/../lib/zookeeper/lib/*::/conf:/lib/*:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/antlr-2.7.7.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/antlr-runtime-3.4.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/apache-mime4j-core-0.7.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/apache-mime4j-dom-0.7.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/api-asn1-api-1.0.0-M20.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/api-util-1.0.0-M20.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/argparse4j-0.4.3.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/asm-4.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/asm-commons-4.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/asm-debug-all-4.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/aspectjrt-1.6.5.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/avro.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/bcmail-jdk15-1.45.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/bcprov-jdk15-1.45.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/boilerpipe-1.1.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/commons-codec-1.4.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/commons-collections-3.2.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/commons-configuration-1.6.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/commons-el-1.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/commons-fileupload-1.3.3.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/commons-math3-3.1.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/commons-net-3.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/concurrentlinkedhashmap-lru-1.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/config-1.0.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/curator-client-2.7.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/curator-framework-2.7.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/curator-recipes-2.7.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/dom4j-1.6.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/fastutil-6.3.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/fontbox-1.8.4.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/gson-2.2.4.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/hadoop-annotations.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/hadoop-auth.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/hadoop-common.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/hadoop-hdfs.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/hppc-0.5.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/htrace-core-3.2.0-incubating.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/htrace-core4-4.0.1-incubating.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/httpclient-4.2.5.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/httpcore-4.2.5.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/httpmime-4.2.5.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/isoparser-1.0-RC-1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jackson-annotations-2.3.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jackson-core-2.3.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jackson-core-asl-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jackson-databind-2.3.1-cloudera.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jackson-mapper-asl-1.8.10-cloudera.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/javax.servlet-3.0.0.v201112011016.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jcl-over-slf4j-1.7.5.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jdom-1.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jempbox-1.8.4.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jetty-continuation-8.1.10.v20130312.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jetty-deploy-8.1.10.v20130312.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jetty-http-8.1.10.v20130312.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jetty-io-8.1.10.v20130312.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jetty-jmx-8.1.10.v20130312.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jetty-security-8.1.10.v20130312.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jetty-server-8.1.10.v20130312.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jetty-servlet-8.1.10.v20130312.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jetty-util-8.1.10.v20130312.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jetty-webapp-8.1.10.v20130312.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jetty-xml-8.1.10.v20130312.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jhighlight-1.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/joda-time-1.6.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jsch-0.1.42.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/jsr305-1.3.9.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/juniversalchardet-1.0.3.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-hadoop-compatibility.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-avro.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-core.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-hadoop-core.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-hadoop-parquet-avro.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-hadoop-rcfile.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-hadoop-sequencefile.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-json.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-maxmind.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-metrics-servlets.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-saxon.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-solr-cell.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-solr-core.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-tika-core.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-tika-decompress.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-twitter.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/kite-morphlines-useragent.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/leveldbjni-all-1.8.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/lucene-analyzers-common.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/lucene-analyzers-kuromoji.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/lucene-analyzers-phonetic.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/lucene-codecs.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/lucene-core.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/lucene-expressions.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/lucene-grouping.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/lucene-highlighter.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/lucene-join.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/lucene-memory.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/lucene-misc.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/lucene-queries.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/lucene-queryparser.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/lucene-spatial.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/lucene-suggest.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/maxmind-db-1.0.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/metadata-extractor-2.6.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/metrics-core-3.0.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/metrics-healthchecks-3.0.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/metrics-json-3.0.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/metrics-jvm-3.0.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/metrics-servlets-3.0.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/netcdf-4.2-min.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/netty-3.10.5.Final.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/netty-all-4.0.23.Final.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/noggit-0.5.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/org.restlet-2.1.4.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/org.restlet.ext.servlet-2.1.4.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/paranamer-2.3.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/parquet-avro.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/parquet-column.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/parquet-common.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/parquet-encoding.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/parquet-format.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/parquet-hadoop.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/parquet-jackson.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/pdfbox-1.8.4.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/poi-3.10-beta2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/poi-ooxml-3.10-beta2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/poi-ooxml-schemas-3.10.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/poi-scratchpad-3.10-beta2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/rome-0.9.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/Saxon-HE-9.5.1-5.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/snakeyaml-1.10.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/snappy-java-1.0.4.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/solr-cell.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/solr-core.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/solr-solrj.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/spatial4j-0.4.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/tagsoup-1.2.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/tika-core-1.5.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/tika-parsers-1.5.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/tika-xmp-1.5.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/ua-parser-1.3.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/vorbis-java-core-0.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/vorbis-java-core-0.1-tests.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/vorbis-java-tika-0.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/wstx-asl-3.2.7.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/xercesImpl-2.9.1.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/xml-apis-1.3.04.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/xmlbeans-2.6.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/xmlenc-0.52.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/xmpcore-5.1.2.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/xz-1.0.jar:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/flume-ng/../search/lib/zookeeper.jar' -Djava.library.path=:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/lib/native:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hadoop/lib/native:/opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/hbase/bin/../lib/native/Linux-amd64-64 org.apache.flume.node.Application -f /data/test.properties -n test
19/01/15 11:41:42 INFO node.PollingPropertiesFileConfigurationProvider: Configuration provider starting
19/01/15 11:41:42 INFO node.PollingPropertiesFileConfigurationProvider: Reloading configuration file:/data/test.properties
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:hdfssink
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:hdfssink
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:hdfssink
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:hdfssink
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:hdfssink
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:hdfssink
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:hdfssink
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Added sinks: hdfssink Agent: test
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:c1
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:hdfssink
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Processing:sqls
19/01/15 11:41:42 WARN conf.FlumeConfiguration: Agent configuration for 'test' has no configfilters.
19/01/15 11:41:42 INFO conf.FlumeConfiguration: Post-validation flume configuration contains configuration for agents: [test]
19/01/15 11:41:42 INFO node.AbstractConfigurationProvider: Creating channels
19/01/15 11:41:42 INFO channel.DefaultChannelFactory: Creating instance of channel c1 type memory
19/01/15 11:41:42 INFO node.AbstractConfigurationProvider: Created channel c1
19/01/15 11:41:42 INFO source.DefaultSourceFactory: Creating instance of source sqls, type org.keedio.flume.source.SQLSource
19/01/15 11:41:42 INFO source.SQLSource: Reading and processing configuration values for source sqls
19/01/15 11:41:42 INFO common.Version: HCANN000001: Hibernate Commons Annotations {4.0.5.Final}
19/01/15 11:41:42 INFO hibernate.Version: HHH000412: Hibernate Core {4.3.10.Final}
19/01/15 11:41:42 INFO cfg.Environment: HHH000206: hibernate.properties not found
19/01/15 11:41:42 INFO cfg.Environment: HHH000021: Bytecode provider name : javassist
19/01/15 11:41:42 INFO source.HibernateHelper: Opening hibernate session
19/01/15 11:41:42 INFO internal.ConnectionProviderInitiator: HHH000130: Instantiating explicit connection provider: org.hibernate.connection.C3P0ConnectionProvider
19/01/15 11:41:42 INFO internal.C3P0ConnectionProvider: HHH010002: C3P0 using driver: com.mysql.jdbc.Driver at URL: jdbc:mysql://192.168.199.237:3306/testflume
19/01/15 11:41:42 INFO internal.C3P0ConnectionProvider: HHH000046: Connection properties: {user=root, password=****, autocommit=true}
19/01/15 11:41:42 INFO internal.C3P0ConnectionProvider: HHH000006: Autocommit mode: true
19/01/15 11:41:42 INFO log.MLog: MLog clients using log4j logging.
19/01/15 11:41:42 INFO c3p0.C3P0Registry: Initializing c3p0-0.9.2.1 [built 20-March-2013 10:47:27 +0000; debug? true; trace: 10]
19/01/15 11:41:42 INFO impl.AbstractPoolBackedDataSource: Initializing c3p0 pool... com.mchange.v2.c3p0.PoolBackedDataSource@7cfc77e7 [ connectionPoolDataSource -> com.mchange.v2.c3p0.WrapperConnectionPoolDataSource@1fd7de4b [ acquireIncrement -> 3, acquireRetryAttempts -> 30, acquireRetryDelay -> 1000, autoCommitOnClose -> false, automaticTestTable -> null, breakAfterAcquireFailure -> false, checkoutTimeout -> 0, connectionCustomizerClassName -> null, connectionTesterClassName -> com.mchange.v2.c3p0.impl.DefaultConnectionTester, debugUnreturnedConnectionStackTraces -> false, factoryClassLocation -> null, forceIgnoreUnresolvedTransactions -> false, identityToken -> 1hgf4cha0m33vwywt512i|5e7d509f, idleConnectionTestPeriod -> 0, initialPoolSize -> 1, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 0, maxIdleTime -> 0, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 10, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 1, nestedDataSource -> com.mchange.v2.c3p0.DriverManagerDataSource@d530b185 [ description -> null, driverClass -> null, factoryClassLocation -> null, identityToken -> 1hgf4cha0m33vwywt512i|b07ce20, jdbcUrl -> jdbc:mysql://192.168.199.237:3306/testflume, properties -> {user=******, password=******, autocommit=true} ], preferredTestQuery -> null, propertyCycle -> 0, statementCacheNumDeferredCloseThreads -> 0, testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> 0, usesTraditionalReflectiveProxies -> false; userOverrides: {} ], dataSourceName -> null, factoryClassLocation -> null, identityToken -> 1hgf4cha0m33vwywt512i|74150459, numHelperThreads -> 3 ]
Tue Jan 15 11:41:42 CST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jan 15 11:41:42 CST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
19/01/15 11:41:42 INFO dialect.Dialect: HHH000400: Using dialect: org.hibernate.dialect.MySQL5Dialect
19/01/15 11:41:42 INFO internal.LobCreatorBuilder: HHH000424: Disabling contextual LOB creation as createClob() method threw error : java.lang.reflect.InvocationTargetException
19/01/15 11:41:43 INFO internal.TransactionFactoryInitiator: HHH000399: Using default transaction strategy (direct JDBC transactions)
19/01/15 11:41:43 INFO ast.ASTQueryTranslatorFactory: HHH000397: Using ASTQueryTranslatorFactory
19/01/15 11:41:43 INFO sink.DefaultSinkFactory: Creating instance of sink: hdfssink, type: hdfs
19/01/15 11:41:43 INFO node.AbstractConfigurationProvider: Channel c1 connected to [sqls, hdfssink]
19/01/15 11:41:43 INFO node.Application: Starting new configuration:{ sourceRunners:{sqls=PollableSourceRunner: { source:org.keedio.flume.source.SQLSource{name:sqls,state:IDLE} counterGroup:{ name:null counters:{} } }} sinkRunners:{hdfssink=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@f2cf2a9 counterGroup:{ name:null counters:{} } }} channels:{c1=org.apache.flume.channel.MemoryChannel{name: c1}} }
19/01/15 11:41:43 INFO node.Application: Starting Channel c1
19/01/15 11:41:43 INFO instrumentation.MonitoredCounterGroup: Monitored counter group for type: CHANNEL, name: c1: Successfully registered new MBean.
19/01/15 11:41:43 INFO instrumentation.MonitoredCounterGroup: Component type: CHANNEL, name: c1 started
19/01/15 11:41:43 INFO node.Application: Starting Sink hdfssink
19/01/15 11:41:43 INFO node.Application: Starting Source sqls
19/01/15 11:41:43 INFO source.SQLSource: Starting sql source sqls ...
19/01/15 11:41:43 INFO instrumentation.MonitoredCounterGroup: Monitored counter group for type: SOURCE, name: SOURCESQL.sqls: Successfully registered new MBean.
19/01/15 11:41:43 INFO instrumentation.MonitoredCounterGroup: Component type: SOURCE, name: SOURCESQL.sqls started
19/01/15 11:41:43 INFO instrumentation.MonitoredCounterGroup: Monitored counter group for type: SINK, name: hdfssink: Successfully registered new MBean.
19/01/15 11:41:43 INFO instrumentation.MonitoredCounterGroup: Component type: SINK, name: hdfssink started
19/01/15 11:41:43 INFO hdfs.HDFSDataStream: Serializer = TEXT, UseRawLocalFileSystem = false
19/01/15 11:41:43 INFO hdfs.BucketWriter: Creating hdfs://nameservice1:8020/flume/mysqls/FlumeData.1547523703138.tmp
Tue Jan 15 11:41:45 CST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jan 15 11:41:45 CST 2019 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
没有报错说明没问题，然后查看hdfs下的文件
[root@mf01 ~]# hdfs dfs -tail -f /flume/mysqls/FlumeData.1547523796517.tmp
1;haha;13
2;hah;13
3;haa;13
4;ha;13
5;aha;13
6;haa;13
7;qqq;12
8;qdg;12
9;zxcsdf;12
10;zxcsdf;12
11;zxcsdf;12
12;zxcsdf;12
13;zxcsdf;12
14;zxcsdf;12
15;zxcsdf;12
16;zxcsdf;12
17;zxcsdf;12
18;zxydf;12
19;zxydf;12
20;zxydf;12
21;zxydf;12
22;zxydf;12
23;qqqq;12
没有问题，接着我们向数据库插入一条数据
[root@mf01 ~]# hdfs dfs -tail -f /flume/mysqls/FlumeData.1547523796517.tmp
1;haha;13
2;hah;13
3;haa;13
4;ha;13
5;aha;13
6;haa;13
7;qqq;12
8;qdg;12
9;zxcsdf;12
10;zxcsdf;12
11;zxcsdf;12
12;zxcsdf;12
13;zxcsdf;12
14;zxcsdf;12
15;zxcsdf;12
16;zxcsdf;12
17;zxcsdf;12
18;zxydf;12
19;zxydf;12
20;zxydf;12
21;zxydf;12
22;zxydf;12
23;qqqq;12
24;aaa;12
24;aaa;12
24;aaa;12
然后就开始出问题了，会重复读最后的操作，我也不知道是什么原因，查看状态文件也正常
[root@mf04 ~]# vim /data/flume/sqls.status
{"SourceName":"sqls","URL":"jdbc:mysql:\/\/192.168.199.237:3306\/testflume","LastIndex":"24","ColumnsToSelect":"*","Table":"test"}
~
~
哪位大神能帮忙分析什么原因吗？