KafKa集群配置：
1.下载解压最新的kafka安装文件 ，并分别解压到三台服务器上
tar -xzf kafka_2.10-0.8.2.0.tgz cd kafka_2.10-0.8.2.0
2.修改Server.properties配置文件：
zookeeper.connect=archtest01:2181,archtest02:2181,arch_open:2181
修改SVR71服务器broker.id=0
修改SVR72服务器broker.id=1
修改SVR87服务器broker.id=2
3.启动KafKa服务
mkdir logs
bin/kafka-server-start.sh config/server.properties > logs/kafka.log&
zk:文件存储目录：/data/server/zookeeper/data
KafKa文件存储目录：/tmp
4.测试集群是否安装成功
测试KafKa集群是否安装成功：
创建Topic
bin/kafka-topics.sh --create --zookeeper archtest01:2181 --replication-factor 3 --partitions 1 --topic qeeka-topic
查看Topic
bin/kafka-topics.sh --describe --zookeeper archtest01:2181 --topic qeeka-topic
生产者发送消息
bin/kafka-console-producer.sh --broker-list archtest01:9092 --topic qeeka-topic
消费者接收消息
bin/kafka-console-consumer.sh --zookeeper archtest01:2181 --from-beginning --topic qeeka-topic
5.简单压力测试(服务器配置较低，需更换服务器后进一步测试，简单测试结果如下：)
-------------------------------------------------------------------------- 生产者：4线程发送50万数据 耗时27秒 每秒1.8万 写入速度每秒17M [root@archtest02 /usr/local/kafka/kafka_2.10-0.8.2.1]# bin/kafka-producer-perf-test.sh --messages 500000 --message-size 1000 --batch-size 1000 --topics qeeka-topic --threads 4 --broker-list archtest01:9092,archtest02:9092,arch_open:9092 start.time, end.time, compression, message.size, batch.size, total.data.sent.in.MB, MB.sec, total.data.sent.in.nMsg, nMsg.sec 2015-03-13 16:46:36:804, 2015-03-13 16:47:03:305, 0, 1000, 1000, 476.84, 17.9932, 500000, 18867.2126 -------------------------------------------------------------------------- 消费者：3线程接收50万数据 耗时10秒 每秒1万 读取速度每秒93M [root@archtest02 /usr/local/kafka/kafka_2.10-0.8.2.1]# bin/kafka-consumer-perf-test.sh --zookeeper archtest01,archtest02,arch_open --messages 500000 --topic qeeka-topic --threads 3 start.time, end.time, fetch.size, data.consumed.in.MB, MB.sec, data.consumed.in.nMsg, nMsg.sec 2015-03-13 16:47:49:620, 2015-03-13 16:47:59:591, 1048576, 467.1689, 93.9788, 500000, 100583.3836 -------------------------------------------------------------------------- 生产者：4线程发送500万数据 耗时247秒 每秒2万 写入速度每秒19M [root@archtest02 /usr/local/kafka/kafka_2.10-0.8.2.1]# bin/kafka-producer-perf-test.sh --messages 5000000 --message-size 1000 --batch-size 1000 --topics qeeka-topic --threads 4 --broker-list archtest01:9092,archtest02:9092,arch_open:9092 start.time, end.time, compression, message.size, batch.size, total.data.sent.in.MB, MB.sec, total.data.sent.in.nMsg, nMsg.sec 2015-03-13 16:55:24:082, 2015-03-13 16:59:31:992, 0, 1000, 1000, 4768.37, 19.2343, 5000000, 20168.6096 -------------------------------------------------------------------------- 消费者：3线程接收500万数据 耗时120秒 每秒4.3万 读取速度每秒41M [root@archtest02 /usr/local/kafka/kafka_2.10-0.8.2.1]# bin/kafka-consumer-perf-test.sh --zookeeper archtest01,archtest02,arch_open --messages 5000000 --topic qeeka-topic --threads 3 start.time, end.time, fetch.size, data.consumed.in.MB, MB.sec, data.consumed.in.nMsg, nMsg.sec 2015-03-13 17:01:52:511, 2015-03-13 17:03:51:895, 1048576, 4758.7033, 41.6029, 5000000, 43712.4073