Zookeeper和kafka集群搭建步骤(超详细,易理解)

前提要求:三个节点、centos源

目录

前提要求:三个节点、centos源

Zookeeper集群搭建:

【任务一】修改主机名、配置主机映射

【任务二】上传软件包到第一节点的/root用户下

【任务三】解压软件包

【任务四】安装jdk环境(三个节点)

【任务五】对zookeeper文件操作

1.修改文件名

2.修改zoo.cfg配置文件内容

3.进入/tmp目录

【任务六】将zookeeper目录传到第二、三节点

【任务七】在第二、三节点对/tmp目录进行相同操作

【任务八】用脚本启动zookeeper服务

【任务九】安装net-tools工具

【任务十】查看端口(出现2181端口)

【任务十一】查看状态

Kafka集群搭建:

【任务一】修改server.properties配置文件

【任务二】将kafka_2.11-1.1.1配置文件传到第二、三节点

【任务三】修改第二、三节点server.properties配置文件

【任务四】用脚本运行文件

【任务五】查看端口(出现9092端口)


Zookeeper集群搭建:

【任务一】修改主机名、配置主机映射

[root@controller ~]# hostnamectl set-hostname zookeeper1
[root@controller ~]# vi /etc/hosts
vi /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.100.10 zookeeper1
192.168.100.20 zookeeper2
192.168.100.30 zookeeper3
[root@controller ~]# bash
[root@zookeeper1 ~]# 
[root@compute ~]# hostnamectl set-hostname zookeeper2
[root@compute ~]# vi /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.100.10 zookeeper1
192.168.100.20 zookeeper2
192.168.100.30 zookeeper3
[root@compute ~]# bash
[root@zookeeper2 ~]# 
[root@localhost ~]# hostnamectl set-hostname zookeeper3
[root@localhost ~]# vi /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.100.10 zookeeper1
192.168.100.20 zookeeper2
192.168.100.30 zookeeper3
[root@localhost ~]# bash
[root@zookeeper3 ~]#

【任务二】上传软件包到第一节点的/root用户下

[root@controller ~]# ls
anaconda-ks.cfg  chinaskills_cloud_iaas_v2.0.3.iso  kafka_2.11-1.1.1.tgz  zookeeper-3.4.14.tar.gz

【任务三】解压软件包

[root@zookeeper1 ~]# tar -xf zookeeper-3.4.14.tar.gz 
[root@zookeeper1 ~]# tar -xf kafka_2.11-1.1.1.tgz 
[root@zookeeper1 ~]# ls
anaconda-ks.cfg                    kafka_2.11-1.1.1      zookeeper-3.4.14
chinaskills_cloud_iaas_v2.0.3.iso  kafka_2.11-1.1.1.tgz  zookeeper-3.4.14.tar.gz

【任务四】安装jdk环境(三个节点)

[root@zookeeper1 ~]# yum install -y java-openjdk*
[root@zookeeper2 ~]# yum install -y java-openjdk*
[root@zookeeper3 ~]# yum install -y java-openjdk*

【任务五】对zookeeper文件操作

1.修改文件名

[root@zookeeper1 ~]# cd zookeeper-3.4.14
[root@zookeeper1 zookeeper-3.4.14]# ls
bin              ivy.xml      README.md                 zookeeper-3.4.14.jar.md5   zookeeper-it
build.xml        lib          README_packaging.txt      zookeeper-3.4.14.jar.sha1  zookeeper-jute
conf             LICENSE.txt  src                       zookeeper-client           zookeeper-recipes
dist-maven       NOTICE.txt   zookeeper-3.4.14.jar      zookeeper-contrib          zookeeper-server
ivysettings.xml  pom.xml      zookeeper-3.4.14.jar.asc  zookeeper-docs
[root@zookeeper1 zookeeper-3.4.14]# cd conf/
[root@zookeeper1 conf]# ls
configuration.xsl  log4j.properties  zoo_sample.cfg
[root@zookeeper1 conf]# mv zoo_sample.cfg zoo.cfg
[root@zookeeper1 conf]# ls
configuration.xsl  log4j.properties  zoo.cfg

2.修改zoo.cfg配置文件内容

(1)找到#maxClientCnxns=60命令在下面添加

server.1=192.168.100.10:2888:3888
server.2=192.168.100.20:2888:3888
server.3=192.168.100.30:2888:3888

3.进入/tmp目录

(1)创建zookeeper目录

[root@zookeeper1 ~]# cd /tmp
[root@zookeeper1 tmp]# ls
hsperfdata_root  systemd-private-f8f7589af730483481a9bc57b003b952-chronyd.service-tNB0d0  vmware-root_704-2990744159
[root@zookeeper1 tmp]# mkdir zookeeper
[root@zookeeper1 tmp]# ls
hsperfdata_root                                                          vmware-root_704-2990744159
systemd-private-f8f7589af730483481a9bc57b003b952-chronyd.service-tNB0d0  zookeeper

(2)进入zookeeper目录中创建myid文件,添加  '  1  ';

[root@zookeeper1 tmp]# cd zookeeper/
[root@zookeeper1 zookeeper]# ls
myid
[root@zookeeper1 zookeeper]# cat myid 
1

【任务六】将zookeeper目录传到第二、三节点

[root@zookeeper1 ~]# scp -r zookeeper-3.4.14 root@zookeeper2://root
[root@zookeeper1 ~]# scp -r zookeeper-3.4.14 root@zookeeper3://root

【任务七】在第二、三节点对/tmp目录进行相同操作

[root@zookeeper2 ~]# cd /tmp/
[root@zookeeper2 tmp]# ls
hsperfdata_root  systemd-private-fcd7259fd6534958b7a2c4d011b6802b-chronyd.service-W1bB8M  vmware-root_713-4290166671
[root@zookeeper2 tmp]# mkdir zookeeper
[root@zookeeper2 tmp]# cd zookeeper/
[root@zookeeper2 zookeeper]# vi myid
2
[root@zookeeper2 zookeeper]# cat myid
2
[root@zookeeper3 ~]# cd /tmp
[root@zookeeper3 tmp]# ls
hsperfdata_root  systemd-private-8729f6a12051421f8573e6efbd244e2c-chronyd.service-39stLG  vmware-root_692-2696942994
[root@zookeeper3 tmp]# mkdir zookeeper
[root@zookeeper3 tmp]# cd zookeeper/
[root@zookeeper3 zookeeper]# vi myid
3
[root@zookeeper3 zookeeper]# cat myid 
3

【任务八】用脚本启动zookeeper服务

[root@zookeeper1 ~]# sh zookeeper-3.4.14/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /root/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

[root@zookeeper2 ~]# sh zookeeper-3.4.14/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /root/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

[root@zookeeper3 ~]# sh zookeeper-3.4.14/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /root/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

【任务九】安装net-tools工具


[root@zookeeper1 ~]# yum install -y net-tools
[root@zookeeper2 ~]# yum install -y net-tools
[root@zookeeper3 ~]# yum install -y net-tools 

【任务十】查看端口(出现2181端口)

[root@zookeeper1 ~]# netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      991/sshd            
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1197/master         
tcp6       0      0 :::22                   :::*                    LISTEN      991/sshd            
tcp6       0      0 ::1:25                  :::*                    LISTEN      1197/master         
tcp6       0      0 :::36865                :::*                    LISTEN      1848/java           
tcp6       0      0 :::2181                 :::*                    LISTEN      1848/java           
tcp6       0      0 192.168.100.10:3888     :::*                    LISTEN      1848/java           
tcp6       0      0 :::21                   :::*                    LISTEN      1498/vsftpd         
udp        0      0 127.0.0.1:323           0.0.0.0:*                           709/chronyd         
udp6       0      0 ::1:323                 :::*                                709/chronyd 


[root@zookeeper2 ~]# netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      995/sshd            
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1229/master         
tcp6       0      0 :::2181                 :::*                    LISTEN      1763/java           
tcp6       0      0 :::46534                :::*                    LISTEN      1763/java           
tcp6       0      0 192.168.100.20:2888     :::*                    LISTEN      1763/java           
tcp6       0      0 192.168.100.20:3888     :::*                    LISTEN      1763/java           
tcp6       0      0 :::22                   :::*                    LISTEN      995/sshd            
tcp6       0      0 ::1:25                  :::*                    LISTEN      1229/master         
udp        0      0 127.0.0.1:323           0.0.0.0:*                           728/chronyd         
udp6       0      0 ::1:323                 :::*                                728/chronyd     


[root@zookeeper3 ~]# netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      997/sshd            
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1191/master         
tcp6       0      0 :::36402                :::*                    LISTEN      1784/java           
tcp6       0      0 :::22                   :::*                    LISTEN      997/sshd            
tcp6       0      0 ::1:25                  :::*                    LISTEN      1191/master         
tcp6       0      0 :::2181                 :::*                    LISTEN      1784/java           
tcp6       0      0 192.168.100.30:3888     :::*                    LISTEN      1784/java           
udp        0      0 127.0.0.1:323           0.0.0.0:*                           716/chronyd         
udp6       0      0 ::1:323                 :::*                                716/chronyd         

【任务十一】查看状态

[root@zookeeper1 ~]# sh zookeeper-3.4.14/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /root/zookeeper-3.4.14/bin/../conf/zoo.cfg
Mode: follower



[root@zookeeper2 ~]# sh zookeeper-3.4.14/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /root/zookeeper-3.4.14/bin/../conf/zoo.cfg
Mode: leader


[root@zookeeper3 ~]# sh zookeeper-3.4.14/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /root/zookeeper-3.4.14/bin/../conf/zoo.cfg
Mode: follower

Kafka集群搭建:

【任务一】修改server.properties配置文件

[root@zookeeper1 ~]# cd kafka_2.11-1.1.1
[root@zookeeper1 kafka_2.11-1.1.1]# ls
\bin  config  libs  LICENSE  NOTICE  site-docs
[root@zookeeper1 kafka_2.11-1.1.1]# cd config/
[root@zookeeper1 config]# ls
connect-console-sink.properties    connect-file-source.properties  log4j.properties        zookeeper.properties
connect-console-source.properties  connect-log4j.properties        producer.properties
connect-distributed.properties     connect-standalone.properties   server.properties
connect-file-sink.properties       consumer.properties             tools-log4j.properties
[root@zookeeper1 config]# vi server.properties


broker.id=0                      >>>>>>修改为  broker.id=1

#listeners=PLAINTEXT://:9092     >>>>>>修改为  listeners=PLAINTEXT://192.168.100.10:9092 

【任务二】将kafka_2.11-1.1.1配置文件传到第二、三节点

[root@zookeeper1 ~]# scp -r kafka_2.11-1.1.1 root@zookeeper2://root
[root@zookeeper1 ~]# scp -r kafka_2.11-1.1.1 root@zookeeper3://root

【任务三】修改第二、三节点server.properties配置文件

[root@zookeeper2 ~]# cd kafka_2.11-1.1.1
[root@zookeeper2 kafka_2.11-1.1.1]# ls
\bin  config  libs  LICENSE  NOTICE  site-docs
[root@zookeeper2 kafka_2.11-1.1.1]# cd config/
[root@zookeeper2 config]# ls
connect-console-sink.properties    connect-file-source.properties  log4j.properties        zookeeper.properties
connect-console-source.properties  connect-log4j.properties        producer.properties
connect-distributed.properties     connect-standalone.properties   server.properties
connect-file-sink.properties       consumer.properties             tools-log4j.properties
[root@zookeeper2 config]# vi server.properties


broker.id=0                      >>>>>>修改为  broker.id=2

#listeners=PLAINTEXT://:9092     >>>>>>修改为  listeners=PLAINTEXT://192.168.100.20:9092



[root@zookeeper3 ~]# cd kafka_2.11-1.1.1
[root@zookeeper3 kafka_2.11-1.1.1]# ls
\bin  config  libs  LICENSE  NOTICE  site-docs
[root@zookeeper3 kafka_2.11-1.1.1]# cd config/
[root@zookeeper3 config]# ls
connect-console-sink.properties    connect-file-source.properties  log4j.properties        zookeeper.properties
connect-console-source.properties  connect-log4j.properties        producer.properties
connect-distributed.properties     connect-standalone.properties   server.properties
connect-file-sink.properties       consumer.properties             tools-log4j.properties
[root@zookeeper3 config]# vi server.properties


broker.id=0                      >>>>>>修改为  broker.id=3

#listeners=PLAINTEXT://:9092     >>>>>>修改为  listeners=PLAINTEXT://192.168.100.30:9092

【任务四】用脚本运行文件

[root@zookeeper1 ~]# sh kafka_2.11-1.1.1/bin/kafka-server-start.sh -daemon kafka_2.11-1.1.1/config/server.properties
[root@zookeeper2 ~]# sh kafka_2.11-1.1.1/bin/kafka-server-start.sh -daemon kafka_2.11-1.1.1/config/server.properties
[root@zookeeper3 ~]# sh kafka_2.11-1.1.1/bin/kafka-server-start.sh -daemon kafka_2.11-1.1.1/config/server.properties

【任务五】查看端口(出现9092端口)

[root@zookeeper1 ~]# netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      991/sshd            
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1197/master         
tcp6       0      0 :::22                   :::*                    LISTEN      991/sshd            
tcp6       0      0 :::38328                :::*                    LISTEN      10406/java          
tcp6       0      0 ::1:25                  :::*                    LISTEN      1197/master         
tcp6       0      0 :::36865                :::*                    LISTEN      1848/java           
tcp6       0      0 192.168.100.10:9092     :::*                    LISTEN      10406/java          
tcp6       0      0 :::2181                 :::*                    LISTEN      1848/java           
tcp6       0      0 192.168.100.10:3888     :::*                    LISTEN      1848/java           
tcp6       0      0 :::21                   :::*                    LISTEN      1498/vsftpd         
udp        0      0 127.0.0.1:323           0.0.0.0:*                           709/chronyd         
udp6       0      0 ::1:323                 :::*                                709/chronyd         



[root@zookeeper2 ~]# netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      995/sshd            
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1229/master         
tcp6       0      0 192.168.100.20:9092     :::*                    LISTEN      2104/java           
tcp6       0      0 :::2181                 :::*                    LISTEN      1763/java           
tcp6       0      0 :::46534                :::*                    LISTEN      1763/java           
tcp6       0      0 192.168.100.20:2888     :::*                    LISTEN      1763/java           
tcp6       0      0 :::33100                :::*                    LISTEN      2104/java           
tcp6       0      0 192.168.100.20:3888     :::*                    LISTEN      1763/java           
tcp6       0      0 :::22                   :::*                    LISTEN      995/sshd            
tcp6       0      0 ::1:25                  :::*                    LISTEN      1229/master         
udp        0      0 127.0.0.1:323           0.0.0.0:*                           728/chronyd         
udp6       0      0 ::1:323                 :::*                                728/chronyd   





[root@zookeeper3 ~]# netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      997/sshd            
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1191/master         
tcp6       0      0 :::36402                :::*                    LISTEN      1784/java           
tcp6       0      0 :::22                   :::*                    LISTEN      997/sshd            
tcp6       0      0 :::34648                :::*                    LISTEN      2789/java           
tcp6       0      0 ::1:25                  :::*                    LISTEN      1191/master         
tcp6       0      0 192.168.100.30:9092     :::*                    LISTEN      2789/java           
tcp6       0      0 :::2181                 :::*                    LISTEN      1784/java           
tcp6       0      0 192.168.100.30:3888     :::*                    LISTEN      1784/java           
udp        0      0 127.0.0.1:323           0.0.0.0:*                           716/chronyd         
udp6       0      0 ::1:323                 :::*                                716/chronyd         

你可能感兴趣的:(zookeeper,kafka,linux)