Kafka详细安装

我的环境版本：
CentOS release 6.9
Java版本：jdk1.8
zookeeper版本: zookeeper-3.4.10
Kafka版本：kafka_2.11-0.10.0.0

第一步：安装jdk
1）下载源码包并解压：
mkdir /usr/java
cd /usr/java
tar –zxvf jdk-8u171-linux-x64.tar.gz
2）修改环境变量：
vim /etc/profile

export JAVA_HOME=/usr/java/jdk1.8.0_171
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

source /etc/profile
java –version验证

第二步：安装zookeeper
1）下载源码包并解压：
http://www.apache.org/dyn/closer.cgi/zookeeper
tar -zxvf zookeeper-3.4.10.tar.gz
mv zookeeper-3.4.10 /usr/local/zookeeper

2）修改环境变量：
vim /etc/profile
最后一行加上

# ZooKeeper Env
export ZOOKEEPER_HOME=/usr/local/zookeeper
export PATH=$PATH:$ZOOKEEPER_HOME/bin

source /etc/profile
3）重命名配置文件并修改配置文件：
#初次使用需要修改模板配置文件，我们先copy一份
cp /usr/local/zookeeper/conf/zoo_sample.cfg /usr/local/zookeeper/conf/zoo.cfg
#再创建两个目录
mkdir /usr/local/zookeeper/data /usr/local/zookeeper/logs
vim /usr/local/zookeeper/conf/zoo.cfg

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper/data
dataLogDir=/usr/local/zookeeper/logs
clientPort=2181

如果是多节点配置文件中尾部增加
server.1=从节点IP:2888:3888
server.2=从节点IP:2888:3888
server.3=从节点IP:2888:3888
同时增加
#master
echo “1”>/usr/local/zookeeper/data/myid
#slave1
echo “2”>/usr/local/zookeeper/data/myid
#slave2
echo “3”>/usr/local/zookeeper/data/myid

4）启动ZooKeeper 服务
/usr/local/zookeeper/bin/zkServer.sh start
#停止命令
/usr/local/zookeeper/bin/zkServer.sh stop

第三步：安装kafka
1）下载压缩包并解压：
mkdir -p /usr/local/kafka
一定要在这个目录中解压
tar –zxvf kafka_2.11-0.10.0.0.tgz

2）修改配置文件：
vim /usr/local/kafka/kafka_2.11-0.10.0.0/config/server.properties

#都是本机，IP则写为localhost或者127.0.0.1
zookeeper.connect=zookeeper所在主机IP:2181
advertised.listeners=PLAINTEXT://本机IP:9092

3）启动Kafka：

nohup /usr/local/kafka/kafka_2.11-0.10.0.0/bin/kafka-server-start.sh /usr/local/kafka/kafka_2.11-0.10.0.0/config/server.properties &

Kafka常用shell命令：

//启动zookeeper
/usr/local/zookeeper/bin/zkServer.sh start

//启动kafka二选一
nohup /usr/local/kafka/kafka_2.11-0.10.0.0/bin/kafka-server-start.sh /usr/local/kafka/kafka_2.11-0.10.0.0/config/server.properties &
/usr/local/kafka_2.11-0.10.0.0/bin/kafka-server-start.sh -daemon /usr/local/kafka_2.11-0.10.0.0/config/server.properties

//创建一个topic
/usr/local/kafka/kafka_2.11-0.10.0.0/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

//查看有哪些topic
/usr/local/kafka/kafka_2.11-0.10.0.0/bin/kafka-topics.sh --list --zookeeper localhost:2181

//往topic上发布消息 Ctrl+D结束
//默认监听kafka的端口是9092
/usr/local/kafka/kafka_2.11-0.10.0.0/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

//消费topic上消息 Ctrl+C结束
//Zookeeper默认运行在本地的2181端口上，里面存有元数据
/usr/local/kafka/kafka_2.11-0.10.0.0/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic example --from-beginning

//查看topic状态信息
/usr/local/kafka_2.11-0.10.0.0/bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic test
不加后面的--topic以及后面的参数则为查看所有的topic