org.apache.flume.conf.ConfigurationException: brokerList must contain at least one Kafka brokerList

楼主在使用flume采集数据,sink到kafka中是,启动flume后报错,楼主在网上搜了一圈,说的原因是对的,就是flume1.8版本之后,要对应kafka0.9版本以上,另外配置文件的配置项也不同,楼主这里不再说明,更改完成后,依然,报错:

WARN kafka.KafkaSink: The Property 'topic' is not set. Using the default topic name: default-flume-topic
19/04/20 13:11:44 INFO kafka.KafkaSinkUtil: context={ parameters:{kafka.flumeBatchSize=20, kafka.bootstrap.servers=hadoop001:9092 hadoop002:9092 hadoop003:9092, kafka.producer.acks=1, channel=c1, kafka.topic=calllog, type=org.apache.flume.sink.kafka.KafkaSink} }
19/04/20 13:11:44 ERROR node.AbstractConfigurationProvider: Sink k1 has been removed due to an error during configuration
org.apache.flume.conf.ConfigurationException: brokerList must contain at least one Kafka broker
	at org.apache.flume.sink.kafka.KafkaSinkUtil.addDocumentedKafkaProps(KafkaSinkUtil.java:55)
	at org.apache.flume.sink.kafka.KafkaSinkUtil.getKafkaProperties(KafkaSinkUtil.java:37)
	at org.apache.flume.sink.kafka.KafkaSink.configure(KafkaSink.java:211)
	at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
	at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:413)
	at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:98)
	at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)

折腾了一上午,楼主发现,环境变量未生效,source /etc/profile .之后再启动,就好了。首先需要配置环境变量,楼主之前,已经配好的,但是用的是虚拟机,重启之后,一些命令的执行,都需要刷新一下环境变量,比如hadoop fs  命令等

你可能感兴趣的:(大数据)