CentOS7 部署 Kafka 集群

为什么80%的码农都做不了架构师？>>>   

环境

主机名	IP	操作系统	kafka 版本
kafka224	192.168.1.224	CentOS7.5	2.12
kafka225	192.168.1.225	CentOS7.5	2.12
kafka226	192.168.1.226	CentOS7.5	2.12

• 下载 kafka_2.12-2.1.0.tgz
• zookeeper 连接: 192.168.1.221:2181,192.168.1.222:2181,192.168.1.223:2181

---

各节点初始配置

• 关闭 selinux、防火墙
• 部署 java 运行环境
• 创建数据目录

  mkdir -p /var/lib/kafka
  

• 创建日志目录

  mkdir -p /var/log/kafka
  

---

部署 kafka

• 登陆 kafka224，下载 kafka，解压至 /opt/ 下
• 修改日志目录

  cd /opt/kafka/
  rm -rf logs && ln -s /var/log/kafka logs
  

• 修改 /opt/kafka/config/server.properties

  broker.id=224
  listeners=PLAINTEXT://192.168.1.224:9092
  log.dirs=/var/lib/kafka
  zookeeper.connect=192.168.1.221:2181,192.168.1.222:2181,192.168.1.223:2181
  

• 启动脚本(/opt/kafka/bin/kafka-server-start.sh)太繁琐，我精简了下

  #!/bin/bash
  base_dir=$(dirname $0)
  if [ "x$KAFKA_LOG4J_OPTS" = "x" ]; then
      export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:$base_dir/../config/log4j.properties"
  fi
  if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
      # jvm 堆内存根据实际情况修改
      export KAFKA_HEAP_OPTS="-Xmx2G -Xms2G"
      # 开启 jmx
      export JMX_PORT=1099
  fi
  EXTRA_ARGS=${EXTRA_ARGS-'-name kafkaServer -loggc'}
  EXTRA_ARGS="-daemon "$EXTRA_ARGS
  exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$base_dir/../config/server.properties"
  

• 打包 kafka 目录，复制到 kafka225 和 kafka226 上，并修改 server.properties

  # kafka225
  broker.id=225
  listeners=PLAINTEXT://192.168.1.225:9092
  # kafka226
  broker.id=226
  listeners=PLAINTEXT://192.168.1.226:9092
  

---

启动集群(两种启动方式)

• 直接启动二进制
  • 在每个节点上启动 kafka 服务

    /opt/kafka/bin/kafka-server-start.sh
    

• systemd 启动
  • 创建 /usr/lib/systemd/system/kafka.service

    [Unit]
    Description=Kafka
    Requires=network.service
    After=network.service
    [Service]
    Environment=JAVA_HOME=/opt/jre
    ExecStart=/opt/kafka/bin/kafka-server-start.sh
    ExecStop=/opt/kafka/bin/kafka-server-stop.sh
    Type=forking
    KillMode=none
    [Install]
    WantedBy=multi-user.target
    

  • 启动 kafka 服务

    systemctl daemon-reload
    systemctl start kafka
    

---

转载于:https://my.oschina.net/colben/blog/2994107