Kafka是使用scala编写的运行与jvm虚拟机上的程序,虽然也可以在windows上使用,但是kafka基本上是运行在linux服务器上,因此我们这里也使用linux来开始今天的实战。
首先确保你的机器上安装了jdk(如果没有安装的话请参考:https://blog.csdn.net/gaitiangai/article/details/103009888 进行安装),kafka需要java运行环境,以前的kafka还需要zookeeper,新版的kafka已经内置了一个zookeeper环境,所以我们可以直接使用
下载地址:
http://kafka.apache.org/downloads.html
我下载的是:kafka_2.12-2.3.0.tgz
注意,kafka是需要Java环境的,我这里下载的这个kafka是需要Java1.8环境的,两者必须对应。如果安装Java1.7是无法启动成功的。
进入你的kafka安装文件所在地址
cd 你的文件下载目录
解压你下载的安装文件
tar -xzf kafka_2.12-2.3.0.tgz
进入解压之后的目录
cd kafka_2.12-2.3.0
在kafka解压目录下有一个config的文件夹,里面放置的是我们的配置文件
broker.id 申明当前kafka服务器在集群中的唯一ID,需配置为integer,并且集群中的每一个kafka服务器的id都应是唯一的,我们这里采用默认配置即可(默认为0)
listeners 申明此kafka服务器需要监听的端口号,必须修改为你的kafka所在机器的IP,否则是无法访问的,eg:listeners=PLAINTEXT://172.17.20.138:9092。并确保服务器的9092端口能够访问
zookeeper.connect 申明kafka所连接的zookeeper的地址 ,需配置为zookeeper的地址,由于本次使用的是kafka高版本中自带zookeeper,使用默认配置即可
zookeeper.connect=localhost:2181
当我们有多个应用,在不同的应用中都使用zookeer,都使用默认的zk端口的话就会2181端口冲突,我们可以设置自己的端口号,在config文件夹下zookeeper.properties文件中修改为
clientPort=2185
也就是zk开放接口为2185.
同时修改kafka的接入端口,server.properties文件中修改为
zookeeper.connect=localhost:2185
这样我们就成功修改了kafka里面的端口号
cd进入kafka解压目录,执行:
bin/zookeeper-server-start.sh config/zookeeper.properties
有类似如图输出,表示启动成功:
[2019-11-11 17:04:00,032] INFO tickTime set to 3000 (org.apache.zookeeper.server.ZooKeeperServer)
[2019-11-11 17:04:00,032] INFO minSessionTimeout set to -1 (org.apache.zookeeper.server.ZooKeeperServer)
[2019-11-11 17:04:00,032] INFO maxSessionTimeout set to -1 (org.apache.zookeeper.server.ZooKeeperServer)
[2019-11-11 17:04:00,058] INFO Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory)
[2019-11-11 17:04:00,068] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)
进入kafka解压目录,执行如下命令:
bin/kafka-server-start.sh config/server.properties
启动成功会有类似如下输出:
[2019-11-11 17:50:20,610] WARN No meta.properties file under dir /tmp/kafka-logs/meta.properties (kafka.server.BrokerMetadataCheckpoint)
[2019-11-11 17:50:20,899] INFO [ExpirationReaper-0-topic]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2019-11-11 17:50:20,936] INFO [ExpirationReaper-0-Heartbeat]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2019-11-11 17:50:20,942] INFO [ExpirationReaper-0-Rebalance]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2019-11-11 17:50:20,944] INFO Successfully created /controller_epoch with initial epoch 0 (kafka.zk.KafkaZkClient)
[2019-11-11 17:50:21,024] INFO [GroupCoordinator 0]: Starting up. (kafka.coordinator.group.GroupCoordinator)
[2019-11-11 17:50:21,026] INFO [GroupCoordinator 0]: Startup complete. (kafka.coordinator.group.GroupCoordinator)
[2019-11-11 17:50:21,088] INFO [GroupMetadataManager brokerId=0] Removed 0 expired offsets in 60 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
[2019-11-11 17:50:21,119] INFO [ProducerId Manager 0]: Acquired new producerId block (brokerId:0,blockStartProducerId:0,blockEndProducerId:999) by writing to Zk with path version 1 (kafka.coordinator.transaction.ProducerIdManager)
[2019-11-11 17:50:21,200] INFO [TransactionCoordinator id=0] Starting up. (kafka.coordinator.transaction.TransactionCoordinator)
[2019-11-11 17:50:21,214] INFO [TransactionCoordinator id=0] Startup complete. (kafka.coordinator.transaction.TransactionCoordinator)
[2019-11-11 17:50:21,216] INFO [Transaction Marker Channel Manager 0]: Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2019-11-11 17:50:21,376] INFO [/config/changes-event-process-thread]: Starting (kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread)
[2019-11-11 17:50:21,391] INFO [SocketServer brokerId=0] Started data-plane processors for 1 acceptors (kafka.network.SocketServer)
[2019-11-11 17:50:21,403] INFO Kafka version: 2.3.0 (org.apache.kafka.common.utils.AppInfoParser)
[2019-11-11 17:50:21,403] INFO Kafka commitId: fc1aaa116b661c8a (org.apache.kafka.common.utils.AppInfoParser)
[2019-11-11 17:50:21,403] INFO Kafka startTimeMs: 1573465821392 (org.apache.kafka.common.utils.AppInfoParser)
[2019-11-11 17:50:21,413] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)
OK,如此kafka就算安装和启动成功了。当前机器就是你的kafka集群中的0号broker,即0号节点机器了。
如果你的kafka启动失败,请重点确认你的kafka版本号和jdk版本号是否匹配。