Kafka彻底删除topic详解

一、前言

严格来讲，kafka删除topic并不是很难，但是步骤比较麻烦，稍有不慎就会失败。本文就kafka删除topic的步骤做出详细解析。

 

二、详细步骤

①停止kafka服务

停止kafka的所有的消费者和生产者，关闭kafka，防止在删除topic之后，程序发现topic不存在而自动创建topic

②修改配置文件

在kafka安装目录的conf目录下，修改server.properties文件，在文件的末尾添加如下配置：

[root@hadoop01 config]# pwd /home/software/kafka/config [root@hadoop01 config]# vim server.properties [root@hadoop01 config]#

auto.create.topics.enable = false delete.topic.enable=true

这一步配置很重要。

auto.create.topics.enable = false。这个配置的默认值为true，此时topic不存在时会自动创建，这样会给删除topic带来意想不到的问题。

delete.topic.enable=true。不设置这个参数，则topic无法删除。

kafka集群中所有的机器都要执行这一步操作。

③删除topic

进入kafka安装目录的bin目录，执行指令：./kafka-topics --delete --zookeeper 【zookeeper server:port】 --topic 【topic name】

[root@hadoop01 bin]# pwd /home/software/kafka/bin [root@hadoop01 bin]# ./kafka-topics.sh --delete --zookeeper hadoop01:2181 --topic park SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/software/hbase/lib/phoenix-4.8.1-HBase-0.98-client.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/software/kafka/libs/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. Topic park is marked for deletion. Note: This will have no impact if delete.topic.enable is not set to true. [root@hadoop01 bin]#

注意，这一步执行完毕，topic可能并没有真正删除，只是被标记为删除：“Topic park is marked for deletion.”

④删除kafka存储目录

topic相关的存储目录为在server.properties文件log.dirs配置的目录（本案例中为：/home/software/kafka/tmp）

[root@hadoop01 tmp]# pwd /home/software/kafka/tmp [root@hadoop01 tmp]# ll total 4 drwxr-xr-x 26 root root 4096 Dec 11 17:08 kafka-logs [root@hadoop01 tmp]# cd kafka-logs/ [root@hadoop01 kafka-logs]# ll total 112 -rw-r--r-- 1 root root    0 Nov 28 08:55 cleaner-offset-checkpoint drwxr-xr-x 2 root root 4096 Dec 11 09:02 __consumer_offsets-5 drwxr-xr-x 2 root root 4096 Dec 11 09:02 __consumer_offsets-8 drwxr-xr-x 2 root root 4096 Dec 11 16:13 park -rw-r--r-- 1 root root    4 Dec 11 17:08 log-start-offset-checkpoint -rw-r--r-- 1 root root   54 Nov 28 08:55 meta.properties -rw-r--r-- 1 root root  548 Dec 11 17:08 recovery-point-offset-checkpoint -rw-r--r-- 1 root root  553 Dec 11 17:09 replication-offset-checkpoint [root@hadoop01 kafka-logs]#

删除跟topic相关的目录

[root@hadoop01 kafka-logs]# rm -rf park

注意：kafka集群中的每台机器都需要执行这一步的操作

⑤删除zookeeper服务器上的topic

在其中一台zookeeper服务器上，进入zookeeper安装目录的bin目录，登录到shell

[root@hadoop01 bin]# pwd /home/software/zookeeper/bin [root@hadoop01 bin]# ./zkCli.sh -server hadoop01:2181 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/software/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/software/hbase/lib/phoenix-4.8.1-HBase-0.98-client.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. Connecting to hadoop01:2181 2018-12-11 17:16:40,319 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.7-1713338, built on 11/09/2015 04:32 GMT 2018-12-11 17:16:40,323 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=hadoop01 2018-12-11 17:16:40,323 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_131 2018-12-11 17:16:40,328 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2018-12-11 17:16:40,330 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/home/software/jdk1.8/jre 2018-12-11 17:16:40,330 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/home/software/zookeeper/bin/../build/classes:/home/software/zookeeper/bin/../build/lib/*.jar:/home/software/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/software/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/home/software/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/home/software/zookeeper/bin/../lib/log4j-1.2.16.jar:/home/software/zookeeper/bin/../lib/jline-0.9.94.jar:/home/software/zookeeper/bin/../zookeeper-3.4.7.jar:/home/software/zookeeper/bin/../src/java/lib/*.jar:/home/software/zookeeper/bin/../conf:.:/home/software/jdk1.8/lib/dt.jar:/home/software/jdk1.8/lib/tools.jar:/home/software/hbase/lib/phoenix-4.8.1-HBase-0.98-client.jar:/home/software/hbase/lib/phoenix-4.8.1-HBase-0.98-server.jar 2018-12-11 17:16:40,330 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2018-12-11 17:16:40,330 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2018-12-11 17:16:40,330 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler= 2018-12-11 17:16:40,330 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux 2018-12-11 17:16:40,330 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64 2018-12-11 17:16:40,331 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=2.6.32-431.el6.x86_64 2018-12-11 17:16:40,332 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=root 2018-12-11 17:16:40,332 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/root 2018-12-11 17:16:40,332 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/home/software/zookeeper/bin 2018-12-11 17:16:40,335 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=hadoop01:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@2cdf8d8a Welcome to ZooKeeper! 2018-12-11 17:16:40,387 [myid:] - INFO  [main-SendThread(hadoop01:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server hadoop01/192.168.205.141:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2018-12-11 17:16:40,537 [myid:] - INFO  [main-SendThread(hadoop01:2181):ClientCnxn$SendThread@876] - Socket connection established to hadoop01/192.168.205.141:2181, initiating session 2018-12-11 17:16:40,556 [myid:] - INFO  [main-SendThread(hadoop01:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server hadoop01/192.168.205.141:2181, sessionid = 0x1679abd61be0004, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: hadoop01:2181(CONNECTED) 0]

查看所有的topic

[zk: hadoop01:2181(CONNECTED) 0] ls /brokers/topics [park] [zk: hadoop01:2181(CONNECTED) 1]

执行删除指令，删除topic

[zk: hadoop01:2181(CONNECTED) 1] rmr /brokers/topics/park

此时，topic被彻底删除。

如果topic还存在，或者仍然被标记为“Topic park is marked for deletion.”，则通过指令ls /admin/delete_topics，找到需要删除的topic，然后执行指令rmr /admin/delete_topics/【topic name】

 

至此，topic被彻底删除！