kafka的集群配置

环境资源准备

-JDK1.8.0
-centos7
-kafka0.11.0.1

安装JDK

1、解压JDK文件
tar -zxvf  jdk-*.tar

2、修改/etc/profile
JAVA_HOME=/usr/local/bin/java/1.8.0_152
PATH=$JAVA_HOME/bin:$PATH
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export JAVA_HOME
export PATH
export CLASSPATH

3、生效profile文件
source /etc/profile

4、查看安装是否完成
java -version

kafka安装

1、解压kafka
tar -zxvf  kafka_2.12-0.11.0.1.tgz

2、重命名解压的文件
mv kafka_2.12-0.11.0.1 kafka-0.11.0.1

3、启动zookeeper
./zookeeper-server-start.sh -daemon ../config/zookeeper.properties

4、查看zookeeper是否启动成功
jps -l

5、启动kafka
./kafka-server-start.sh ../config/server.properties &

6、查看kafka是否启动成功
jps 

单机版本的kafka启动完成

测试

1、创建一个topic
sh kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic topic1

2、查看topic是否创建成功
sh kafka-topics.sh --list --zookeeper localhost:2181 

3、发送消息
sh kafka-console-producer.sh --broker-list localhost:9092 --topic topic1
abcdefg

4、客户端接收消息
sh kafka-console-consumer.sh --zookeeper localhost:2181 --topic topic1 --from-beginning

接单机版配置

环境准备

服务器名称	IP	myid	brokerid
centos_1	192.168.4.36	1	1
centos_2	192.168.4.10	2	2
centos_3	192.168.4.102	3	3

1、修改/etc/hosts文件，在文件末尾添加
192.168.4.36 centos_1
192.168.4.102 centos_2
192.168.4.10 centos_3
2、将修改的文件复制到centos_2,centos_3上
 scp /etc/hosts root@centos_2:/etc/hosts
 scp /etc/hosts root@centos_3:/etc/hosts
3、关闭防火墙
systemctl stop firewalld.service #停止firewall
systemctl disable firewalld.service #禁止firewall开机启动

zookeeper配置（已经存在zookeeper环境的话可以跳过此步骤）

1、修改zookeeper.properties
##myid所在的目录
dataDir=/usr/local/bin/kafka/kafka_2.12-0.11.0.1/zookeeper
initLimit=5
syncLimit=2
##IP和server.myid中的myid必须对应
server.1=centos_1:2888:3888
server.2=centos_2:2888:3888
server.3=centos_3:2888:3888
2、将修改的文件复制到centos_2,centos_3上
scp  /usr/local/bin/kafka/kafka_2.12-0.11.0.1/config/zookeeper.properties root@centos_2:/usr/local/bin/kafka/kafka_2.12-0.11.0.1/config/zookeeper.properties
scp  /usr/local/bin/kafka/kafka_2.12-0.11.0.1/config/zookeeper.properties root@centos_3:/usr/local/bin/kafka/kafka_2.12-0.11.0.1/config/zookeeper.properties
3、分别在dataDir目录下分别创建myid

4、启动zookeeper
cd 工作目录
./zookeeper-server-start.sh -daemon ../config/zookeeper.properties
ssh root@centos_2
cd 工作目录
./zookeeper-server-start.sh -daemon ../config/zookeeper.properties
ssh root@centos_3
cd 工作目录
./zookeeper-server-start.sh -daemon ../config/zookeeper.properties

修改service.properties

centos_1:
broker.id =1
listeners=PLAINTEXT://centos_1:9092
zookeeper.connect=centos_1:2181,centos_2:2181,centos_3:2181

centos_2: 
broker.id =2
listeners=PLAINTEXT://centos_2:9092
zookeeper.connect=centos_1:2181,centos_2:2181,centos_3:2181

centos_3:
broker.id =3
listeners=PLAINTEXT://centos_3:9092
zookeeper.connect=centos_1:2181,centos_2:2181,centos_3:2181

启动kafka
./kafka-server-start.sh ../config/server.properties &

集群配置完成