kafka topic增加replica报错解决

刚开始测试环境中给一个业务方建的topic没有增加replica,导致前天虚拟机down掉,业务方反馈无法写入的问题。于是决定将所有replica为1的topic改为2,于是照着官方文档上的说明试试,但由于编辑json文件出错了,却手贱地执行了 --execute,后面即使改对,也无法成功。总是报错:

bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file topics-to-move.json --execute
Current partition replica assignment

{"version":1,"partitions":[{"topic":"production_process_flow_dev","partition":2,"replicas":[1]},{"topic":"production_process_flow_dev","partition":1,"replicas":[3]},{"topic":"production_process_flow_dev","partition":3,"replicas":[2]},{"topic":"production_process_flow_dev","partition":0,"replicas":[2]}]}

Save this to use as the --reassignment-json-file option during rollback
Partitions reassignment failed due to Partition reassignment currently in progress for Map(). Aborting operation
kafka.common.AdminCommandFailedException: Partition reassignment currently in progress for Map(). Aborting operation
	at kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:216)
	at kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:133)
	at kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:47)
	at kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)

经过一番源代码跟踪,最后确定应该是zk上创建了/admin/reassign_partitions,而这个是由于误操作创建又没有成功结束残留下来的,导致正常的reassigin都没办法进行,于是只能命令行登陆zk,直接删除该节点:

[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper, admin, consumers, config, controller, brokers, controller_epoch]
[zk: localhost:2181(CONNECTED) 1] ls /admin/reassign_partitions
[]
[zk: localhost:2181(CONNECTED) 2] get /admin/reassign_partitions
{"version":1,"partitions":[]}
cZxid = 0xd00008216
ctime = Mon Oct 26 14:47:30 CST 2015
mZxid = 0xd00008216
mtime = Mon Oct 26 14:47:30 CST 2015
pZxid = 0xd00008216
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 29
numChildren = 0
[zk: localhost:2181(CONNECTED) 3] rmr /admin/reassign_partitions
[zk: localhost

之后再次之行:

bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file topics-to-move.json --execute
Current partition replica assignment

{"version":1,"partitions":[{"topic":"production_process_flow_dev","partition":2,"replicas":[1]},{"topic":"production_process_flow_dev","partition":1,"replicas":[3]},{"topic":"production_process_flow_dev","partition":3,"replicas":[2]},{"topic":"production_process_flow_dev","partition":0,"replicas":[2]}]}

Save this to use as the --reassignment-json-file option during rollback
Successfully started reassignment of partitions {"version":1,"partitions":[{"topic":"production_process_flow_dev","partition":0,"replicas":[2,1]},{"topic":"production_process_flow_dev","partition":1,"replicas":[3,2]},{"topic":"production_process_flow_dev","partition":2,"replicas":[1,3]},{"topic":"production_process_flow_dev","partition":3,"replicas":[2,1]}]}bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file topics-to-move.json --verify
Status of partition reassignment:
Reassignment of partition [production_process_flow_dev,0] completed successfully
Reassignment of partition [production_process_flow_dev,1] completed successfully
Reassignment of partition [production_process_flow_dev,2] completed successfully
Reassignment of partition [production_process_flow_dev,3] completed successfully





你可能感兴趣的:(kafka)