linux下安装kafka教程

github地址：https://github.com/douzixiansheng/MQ/blob/master/kafka_linux_install.md

Kafka linux 安装步骤

官方步骤：
http://kafka.apache.org/quickstart

安装步骤

• 安装JDK
  Kafka 使用Zookeeper 来保存相关配置信息，Kafka及Zookeeper 依赖Java运行环境。
  https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

• • 下载 JDK (jdk-8u221-linux-x64.tar.gz)
    
    解压
    

    tar -zxvf jdk-8u65-linux-x64.tar.gz
    mv jdk1.8.0_221 java
    

    配置Java环境变量
    

    vi /etc/profile
    

    在配置文件尾部增加
    

    export JAVA_HOME=/usr/local/software/java
    export PATH=$JAVA_HOME/bin:$PATH
    export CLASSPATH=:#JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
    

• 下载Kafka

  wget http://labfile.oss.aliyuncs.com/courses/859/kafka_2.10-0.10.2.1.tgz
  

  • 解压Kafka

  tar -zxvf kafka_2.10-0.10.2.1.tgz 
  

  • 进入配置文件目录

  cd /usr/local/software/kafka_2.10-0.10.2.1/config
  

  • 修改配置文件server.properties

  log.dirs=/usr/local/logs/kafka
  zookeeper.connect=localhost:2181
  

• 编写启动脚本kafka_start.sh

  #启动zookeeper
  /usr/local/software/kafka_2.10-0.10.2.1/bin/zookeeper-server-start.sh /usr/local/software/kafka_2.10-0.10.2.1/config/zookeeper.properties &
  #等3秒后执行
  sleep 3
  #启动kafka
  /usr/local/software/kafka_2.10-0.10.2.1/bin/kafka-server-start.sh /usr/local/software/kafka_2.10-0.10.2.1/config/server.properties &
  

• 编写停止脚本kafka_stop.sh

  #关闭zookeeper
  /usr/local/software/kafka_2.10-0.10.2.1/bin/zookeeper-server-stop.sh /usr/local/software/kafka_2.10-0.10.2.1/config/zookeeper.properties &
  #等3秒后执行
  sleep 3
  #关闭kafka
  /usr/local/software/kafka_2.10-0.10.2.1/bin/kafka-server-stop.sh /usr/local/software/kafka_2.10-0.10.2.1/config/server.properties &
  

• 启动kafka

  ./kafka_start.sh
  

• 使用jps 查看

  root@FM:/usr/local/software# jps
  4649 Jps
  4377 Kafka
  4139 QuorumPeerMain
  

Kafka 使用

• 创建主题

./kafka-topics.sh --create --topic mytopic2 --zookeeper localhost:2181 --partitions 3 --replication-factor 1

执行结果
Created topic "mytopic2".

• 查看当前主题

 ./kafka-topics.sh --describe --zookeeper localhost:2181
 或只看主题名称
 ./kafka-topics.sh --list --zookeeper localhost:2181

执行结果
Topic:mytopic2  PartitionCount:3        ReplicationFactor:1     Configs:
    Topic: mytopic2 Partition: 0    Leader: 0       Replicas: 0     Isr: 0
    Topic: mytopic2 Partition: 1    Leader: 0       Replicas: 0     Isr: 0
    Topic: mytopic2 Partition: 2    Leader: 0       Replicas: 0     Isr: 0
Topic:mytopic3  PartitionCount:3        ReplicationFactor:1     Configs:
    Topic: mytopic3 Partition: 0    Leader: 0       Replicas: 0     Isr: 0
    Topic: mytopic3 Partition: 1    Leader: 0       Replicas: 0     Isr: 0
    Topic: mytopic3 Partition: 2    Leader: 0       Replicas: 0     Isr: 0

• 查看某个主题是否存在

  ./kafka-topics.sh --list --zookeeper localhost:2181|grep mytopic11
  

• 修改主题

  ./kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic2 --partitions 10
  

• 删除配置参数

  ./kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic2 --delete-config flush.messages
  

• 删除主题

  ./kafka-topics.sh --zookeeper localhost:2181 --delete --topic mytopic2
  结果：
  Topic mytopic2 is marked for deletion.
  Note: This will have no impact if delete.topic.enable is not set to true.
  

• 启动kafka producer

   ./kafka-console-producer.sh --broker-list localhost:9092 --topic mytopic3
  

• 启动kafka consumer

  ./kafka-console-consumer.sh --zookeeper localhost --topic mytopic3
  

• 终端写入kafka主题数据

  ./kafka-console-producer.sh --broker-list localhost:9092 --topic mytopic3
  

• 终端消费kafka主题数据

  ./kafka-console-consumer.sh --zookeeper localhost:2181 --topic mytopic3 --from-beginning