Kafka集群安装笔记

Kafka集群安装文档

---

安装scala 环境

依赖scala环境，需要先安装scala，略

安装zookeeper集群

依赖zookeeper，需要先安装zookeeper集群

kafka安装配置

下载 kafka_2.11-0.10.1.0.tgz
解压到/cluster/kafka/kafka2.11

修改config/server.properties

broker.id=0  #当前机器在集群中的唯一标识，和zookeeper的myid性质一样
log.dirs=/cluster/kafka/kafkalogs/
zookeeper.connect=bi-server-01:2181,bi-server-02:2181,bi-server-03:2181 #设置zookeeper的连接端口

注意：broker.id在集群的每台机器中设置为唯一标示

启动

kafka2.11/bin/kafka-server-start.sh kafka2.11/config/server.properties 

后台启动-daemon

kafka2.11/bin/kafka-server-start.sh -daemon  kafka2.11/config/server.properties 

检查服务是否启动jps

16960 Jps
16568 Kafka

创建Topic来验证是否创建成功

kafka2.11/bin/kafka-topics.sh  --create --zookeeper bi-server-01:2181,bi-server-02:2181,bi-server-03:2181 --replication-factor 2 --partitions 3 --topic testTopic

查看topic

kafka2.11/bin/kafka-topics.sh  --zookeeper bi-server-01:2181,bi-server-02:2181,bi-server-03:2181 -list testTopic

创建producer：

kafka2.11/bin/kafka-console-producer.sh -broker-list bi-server-01:9092,bi-server-02:9092,bi-server-03:9092 -topic testTopic

创建consumer：

kafka2.11/bin/kafka-console-consumer.sh -zookeeper bi-server-01:2181,bi-server-02:2181,bi-server-03:2181 - from-begining -topic testTopic

在producer中设置消息，在consumer中会实时接收消息。