Kafka使用ambari安装遇到的问题

错误1 : kafka.common.InconsistentBrokerIdException: Configured brokerId 1 doesn’t match stored brokerId 0 in meta.properties

[2016-06-29 15:05:29,127] FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.common.InconsistentBrokerIdException: Configured brokerId 1 doesn't match stored brokerId 0 in meta.properties
            at kafka.server.KafkaServer.getBrokerId(KafkaServer.scala:630)
            at kafka.server.KafkaServer.startup(KafkaServer.scala:175)
            at io.confluent.support.metrics.SupportedServerStartable.startup(SupportedServerStartable.java:99)
            at io.confluent.support.metrics.SupportedKafka.main(SupportedKafka.java:45)

解决方案:
$KAFKA_HOME/conf/server.properties文件里的属性值log.dirs=/storage/kafka-logs,进入/storage/kafka-logs(具体看log.dirs后面的值)目录,修改meta.properties文件

源文件

version=0
broker.id=0

修改后(修改broker.id为当前节点的broker.id)

version=0
broker.id=1

错误2 : ambari安装Kafka后其他节点使用broker的IP无法访问

ambari安装Kafka后,管理页面看到的配置是
这里写图片描述

在broker节点上看到的配置是【管理页面配置的localhost会自动映射为当前节点的主机名】
这里写图片描述

然后用JAVA代码生产、消费Kafka,报错

package cn.gcks.kafka_test;

import java.util.Properties;

import kafka.javaapi.producer.Producer;
import kafka.producer.KeyedMessage;
import kafka.producer.ProducerConfig;
import kafka.serializer.StringEncoder;

public class Test123 {
    public static void main(String[] args) {
        Properties properties = new Properties();
        properties.put("zookeeper.connect", "111.111.111.111:2181,111.111.111.112:2181,111.111.111.113:2181");
        properties.put("serializer.class", StringEncoder.class.getName());
        properties.put("metadata.broker.list", "111.111.111.111:6667,111.111.111.112:6667,111.111.111.113:6667");

        Producer producer = new Producer(new ProducerConfig(properties));
        for (int i = 0; i < 100; i++) {
            producer.send(new KeyedMessage("test123", "dddddd: " + i++));
        }
    }
}

具体原因:
Kafka使用ambari安装遇到的问题_第1张图片

解决方案:分别修改每个broker的$KAFKA_HOME/conf/server.properties文件,属性这里写图片描述,主机名改成IP

你可能感兴趣的:(大数据,Kafka,Ambari)