docker 部署 zookeeper & kafka

Zookeeper 集群部署

主机名称	IP地址	mypid
zookeeper_1	10.160.1.5	1
zookeeper_2	10.160.1.10	2
zookeeper_3	10.160.1.7	3
zookeeper_4	10.160.1.9	4

1. 官方下载二进制包

官方地址
https://archive.apache.org/dist/zookeeper/

mkdir /data/package 

cd /data/package 

wget http://archive.apache.org/dist/zookeeper/zookeeper-3.8.1/apache-zookeeper-3.8.1-bin.tar.gz 

2. 初始化

cd /data/package 

# 解压
tar -zxvf apache-zookeeper-3.8.1-bin.tar.gz 
mv apache-zookeeper-3.8.1-bin ../zookeeper-3.8.1

# 添加数据、日志目录
cd ../zookeeper-3.8.1 
mkdir logs data

3. 修改配置文件

cd /data/zookeeper-3.8.1 

# 拷贝默认配置文件
cp conf/zoo_sample.cfg conf/zoo.cfg

# 修改配置文件 
egrep -v "^#|^$" conf/zoo.cfg

tickTime=2000
initLimit=10
syncLimit=5
dataLogDir=/data/zookeeper-3.8.1/logs
dataDir=/data/zookeeper-3.8.1/data
clientPort=2181
server.1=10.160.1.5:2888:3888
server.2=10.160.1.10:2888:3888
server.3=10.160.1.7:2888:3888
server.4=10.160.1.9:2888:3888

# 在对应的节点新增 data/mypid 文件，填写对应的 server.id 
echo 1 > data/mypid

4. 启动

bin/zkServer.sh start

5. 查看集群状态

bin/zkServer.sh status 

# 正常日志
/bin/java
ZooKeeper JMX enabled by default
Using config: /data/zookeeper-3.8.1/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower

Kafka 集群部署

使用 docker 部署集群，CentOS 安装 docker & docker-compose

主机名称	IP地址	brokerID
kafka_1	10.160.1.13	1
kafka_2	10.160.1.12	2
kafka_3	10.160.1.11	3

说明: 以下操作均在 /data/kafka/ 路径下执行

version: "3"
services: 

  kafka1: 
    image: bitnami/kafka:2.8.1
    container_name: kafka1
    hostname: kafka1
    restart: always 
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 10.160.1.5:2181,10.160.1.10:2181:10.160.1.7:2181,10.160.1.9:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://10.160.1.13:9092
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      ALLOW_PLAINTEXT_LISTENER: yes
    volumes: 
      - "./data:/bitnami/kafka"
    ports: 
      - "9092:9092"

说明：

• container_name 和 hostname 保持一样，不要和其他主机重复
• KAFKA_ADVERTISED_LISTENERS：填写对应服务器地址
• KAFKA_BROKER_ID: 填写对应的 ID 即可