Kafka 入门教程之一: 安装
Kafka 入门教程之一: 安装
1. 官网下载软件
http://kafka.apache.org/
2. 安装单节点kafka
a) 创建用户kafka
[root@tjtestrac1 ~]# useradd kafka
[root@tjtestrac1 ~]# passwd kafka
Changing password for user kafka.
New password:
BAD PASSWORD: The password is shorter than 8 characters
Retype new password:
passwd: all authentication tokens updated successfully.
b) 解压软件包
[kafka@tjtestrac1 ~]$ ls
kafka_2.12-2.1.0.tgz
[kafka@tjtestrac1 ~]$ tar -xvf kafka_2.12-2.1.0.tgz
c) 配置环境变量 kafka_home 和 java_home
[kafka@tjtestrac1 ~]$ vi .bash_profile
export KAFKA_HOME=/home/kafka/kafka_2.12-2.1.0
export JAVA_HOME=/u02/cassandra/java/jdk1.8.0_162
PATH=$PATH:$HOME/.local/bin:$HOME/bin:$JAVA_HOME/bin:$KAFKA_HOME/bin [kafka@tjtestrac1 ~]$ source .bash_profile
3. 启动zookeeper, kafka 组件
a) 为zookeeper 创建文件夹
[kafka@tjtestrac1 config]$ mkdir -p /home/kafka/zk
b) 修改kafka 内置的zk 配置文件
[kafka@tjtestrac1 config]$ vi zookeeper.properties
dataDir=/home/kafka/zk
# the port at which the clients will connect
clientPort=2181
# disable the per-ip limit on the number of connections since this is a non-production config
maxClientCnxns=0
c) 启动zk
kafka@tjtestrac1 config]$ zookeeper-server-start.sh $KAFKA_HOME/config/zookeeper.properties &
d) 查看后台进程
[kafka@tjtestrac1 ~]$ jps
30323 QuorumPeerMain
30660 Jps
4. 启动kafka
a) 为kafka 创建文件夹
[kafka@tjtestrac1 ~]$ mkdir -p /home/kafka/kf
b) 修改kafka 的配置文件
[kafka@tjtestrac1 config]$ vi server.properties
log.dirs=/home/kafka/kf
c) 启动kafka 进程
[kafka@tjtestrac1 config]$ kafka-server-start.sh $KAFKA_HOME/config/server.properties &
d) 查看后台进程
[kafka@tjtestrac1 config]$ jps
30323 QuorumPeerMain
4739 Kafka
5414 Jps
5. 创建一个主题topic
[kafka@tjtestrac1 config]$ kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
[2018-11-22 17:27:47,517] INFO Accepted socket connection from /127.0.0.1:58228 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2018-11-22 17:27:47,519] INFO Client attempting to establish new session at /127.0.0.1:58228 (org.apache.zookeeper.server.ZooKeeperServer)
[2018-11-22 17:27:47,521] INFO Established session 0x10052ce304e0001 with negotiated timeout 30000 for client /127.0.0.1:58228 (org.apache.zookeeper.server.ZooKeeperServer)
[2018-11-22 17:27:47,802] INFO Got user-level KeeperException when processing sessionid:0x10052ce304e0001 type:setData cxid:0x4 zxid:0x1f txntype:-1 reqpath:n/a Error Path:/config/topics/test Error:KeeperErrorCode = NoNode for /config/topics/test (org.apache.zookeeper.server.PrepRequestProcessor)
Created topic "test".
列出主题:
[kafka@tjtestrac1 config]$ kafka-topics.sh --list --zookeeper localhost:2181
[2018-11-22 17:29:31,251] INFO Accepted socket connection from /0:0:0:0:0:0:0:1:64976 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2018-11-22 17:29:31,253] INFO Client attempting to establish new session at /0:0:0:0:0:0:0:1:64976 (org.apache.zookeeper.server.ZooKeeperServer)
[2018-11-22 17:29:31,255] INFO Established session 0x10052ce304e0002 with negotiated timeout 30000 for client /0:0:0:0:0:0:0:1:64976 (org.apache.zookeeper.server.ZooKeeperServer)
test
[2018-11-22 17:29:31,302] INFO Processed session termination for sessionid: 0x10052ce304e0002 (org.apache.zookeeper.server.PrepRequestProcessor)
[2018-11-22 17:29:31,303] INFO Closed socket connection for client /0:0:0:0:0:0:0:1:64976 which had sessionid 0x10052ce304e0002 (org.apache.zookeeper.server.NIOServerCnxn)
6. 启动生产者来发送数据
[kafka@tjtestrac1 ~]$ kafka-console-producer.sh --broker-list localhost:9092 --topic test
>jason
>comm on
>test message
7. 启动消费者来接收数据
[kafka@tjtestrac1 ~]$ kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
[kafka@tjtestrac1 ~]$ kafka-console-producer.sh --broker-list localhost:9092 --topic test
>jason
>comm on
>test message
>next message