kafka伪集群搭建常见错误

一：kafka 集群搭建常见错误
[2017-09-26 08:26:45,015] FATAL [Kafka Server 1], Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.common.KafkaException: Failed to acquire lock on file .lock in /home/wenyuancheng/mykafka/logs. A Kafka instance in another process or thread is using this directory.
        at kafka.log.LogManager$$anonfun$lockLogDirs$1.apply(LogManager.scala:112)
        at kafka.log.LogManager$$anonfun$lockLogDirs$1.apply(LogManager.scala:109)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
        at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
        at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
        at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:35)
        at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
        at scala.collection.AbstractTraversable.map(Traversable.scala:104)
        at kafka.log.LogManager.lockLogDirs(LogManager.scala:109)
        at kafka.log.LogManager.(LogManager.scala:70)
        at kafka.log.LogManager$.apply(LogManager.scala:598)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:215)
        at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
        at kafka.Kafka$.main(Kafka.scala:65)
        at kafka.Kafka.main(Kafka.scala)
[2017-09-26 08:26:45,021] INFO [Kafka Server 1], shutting down (kafka.server.KafkaServer)
[2017-09-26 08:26:45,028] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2017-09-26 08:26:45,037] INFO Session: 0x25ea1e7abb30006 closed (org.apache.zookeeper.ZooKeeper)
[2017-09-26 08:26:45,040] INFO EventThread shut down for session: 0x25ea1e7abb30006 (org.apache.zookeeper.ClientCnxn)
[2017-09-26 08:26:45,063] INFO [Kafka Server 1], shut down completed (kafka.server.KafkaServer)
[2017-09-26 08:26:45,064] FATAL Exiting Kafka. (kafka.server.KafkaServerStartable)
[2017-09-26 08:26:45,069] INFO [Kafka Server 1], shutting down (kafka.server.KafkaServer)

关于kafka 一种具体的解决方案：
1：
 删除 /tmp/kafka-logs；

2：netstat -lnp|grep 9092；并且 kill pid；


3: 最后再重新启动下就ok了
 ./kafka-server-start.sh ../config/server.properties


二：

Java HotSpot(TM) 64-Bit Server VM warning: Cannot open file /home/wenyuancheng/mykafka/k2/bin/../logs/kafkaServer-gc.log due to Permission denied

log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /home/wenyuancheng/mykafka/k2/bin/../logs/server.log (权限不够)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.(FileOutputStream.java:221)
        at java.io.FileOutputStream.(FileOutputStream.java:142)
        at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
        at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
        at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223)
        at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
        at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
        at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
        at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
        at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
        at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
        at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
        at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
        at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
        at org.apache.log4j.LogManager.(LogManager.java:127)
        at org.slf4j.impl.Log4jLoggerFactory.(Log4jLoggerFactory.java:66)
        at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:72)
        at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:45)
        at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
        at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
        at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)
        at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
        at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
        at org.apache.kafka.common.utils.Utils.(Utils.java:67)
        at kafka.Kafka$.getPropsFromArgs(Kafka.scala:41)
        at kafka.Kafka$.main(Kafka.scala:57)
        at kafka.Kafka.main(Kafka.scala)
log4j:ERROR Either File or DatePattern options are not set for appender [kafkaAppender].
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /home/wenyuancheng/mykafka/k2/bin/../logs/kafka-request.log (权限不够)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.(FileOutputStream.java:221)
        at java.io.FileOutputStream.(FileOutputStream.java:142)
        at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
        at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
        at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223)
        at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
        at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
        at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
        at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
        at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
        at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672)
        at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516)
        at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
        at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
        at org.apache.log4j.LogManager.(LogManager.java:127)
        at org.slf4j.impl.Log4jLoggerFactory.(Log4jLoggerFactory.java:66)
        at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:72)
        at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:45)
        at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
        at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
        at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)
        at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
        at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
        at org.apache.kafka.common.utils.Utils.(Utils.java:67)
        at kafka.Kafka$.getPropsFromArgs(Kafka.scala:41)
        at kafka.Kafka$.main(Kafka.scala:57)
        at kafka.Kafka.main(Kafka.scala)
log4j:ERROR Either File or DatePattern options are not set for appender [requestAppender].
log4j:ERROR setFile(null,true) call failed.




权限不够的问题，直接换成root用户去执行就可以了；


四：


[2017-09-26 08:45:40,526] FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.common.InconsistentBrokerIdException: Configured broker.id 2 doesn't match stored broker.id 1 in meta.properties. If you moved your data, make sure your configured broker.id matches. If you intend to create a new broker, you should remove all data in your data directories (log.dirs).
        at kafka.server.KafkaServer.getBrokerId(KafkaServer.scala:674)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:198)
        at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
        at kafka.Kafka$.main(Kafka.scala:65)
        at kafka.Kafka.main(Kafka.scala)
[2017-09-26 08:45:40,529] INFO shutting down (kafka.server.KafkaServer)
[2017-09-26 08:45:40,532] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)


vi /home/wenyuancheng/mykafka/k2/config/server.properties
将对应的log.dir 的目录改成的不同的，集群中的每个logdir 目录改成不同的



host.name=192.168.18.37
port=9093



./kafka-topics.sh --create --zookeeper 192.168.18.37:2181 --replication-factor 2 --partitions 1 --topic wenyctest