zookeeper+kafka集群
------------------------------------
zookeper：
59.110.173.251（xianshang ssq） /home/appinstall/zookeeper-3.4.9
47.94.133.25 （xianshang 5M host）/home/songlk/appinstall/zookeeper-3.4.9
60.205.136.57 （jsfund） /home/songlk/appinstall/zookeeper-3.4.9
zoo.cfg:
dataDir=/home/centos/zookeeper
server.1=59.110.173.251:2888:3888
server.2=47.94.133.25:2888:3888
server.3=60.205.136.57:2888:3888
写集群文件
echo 1 > /home/centos/zookeeper/myid
echo 2 > /home/centos/zookeeper/myid
echo 3 > /home/centos/zookeeper/myid
--------------
iptables -A INPUT -p tcp --dport 2181 -j ACCEPT
iptables -A INPUT -p tcp --dport 2888 -j ACCEPT
iptables -A INPUT -p tcp --dport 3888 -j ACCEPT
分别启动
59.110.173.251
/home/appinstall/zookeeper-3.4.9/bin/zkServer.sh start
47.94.133.25
/home/songlk/appinstall/zookeeper-3.4.9/bin/zkServer.sh start
60.205.136.57
/home/songlk/appinstall/zookeeper-3.4.9/bin/zkServer.sh start
查看leader与slave状态
/home/appinstall/zookeeper-3.4.9/bin/zkServer.sh status
/home/songlk/appinstall/zookeeper-3.4.9/bin/zkServer.sh status
/home/songlk/appinstall/zookeeper-3.4.9/bin/zkServer.sh status
-----
flume
/home/songlk/appinstall/flume
----
kafka
59.110.173.251（xianshang ssq）
/home/appinstall/kafka
47.94.133.25 （xianshang 5M host）
/home/songlk/appinstall/kafka
60.205.136.57（ jsfund）
/home/songlk/appinstall/kafka
配置server.properties
broker.id=251
listeners=PLAINTEXT://:9092
log.dirs=/home/appinstall/kafka/kafka-logs
zookeeper.connect=59.110.173.251:2181,47.94.133.25:2181,60.205.136.57:2181
broker.id=25
listeners=PLAINTEXT://:9092
log.dirs=/home/songlk/appinstall/kafka/kafka-logs
zookeeper.connect=59.110.173.251:2181,47.94.133.25:2181,60.205.136.57:2181
broker.id=57
listeners=PLAINTEXT://:9092
log.dirs=/home/songlk/appinstall/kafka/kafka-logs
zookeeper.connect=59.110.173.251:2181,47.94.133.25:2181,60.205.136.57:2181
----
producer.properties
59.110.173.251:9092,47.94.133.25:9092,60.205.136.57:9092
启动kafka
/home/appinstall/kafka/bin/kafka-server-start.sh -daemon /home/appinstall/kafka/config/server.properties
/home/songlk/appinstall/kafka/bin/kafka-server-start.sh -daemon /home/songlk/appinstall/kafka/config/server.properties
/home/songlk/appinstall/kafka/bin/kafka-server-start.sh -daemon /home/songlk/appinstall/kafka/config/server.properties
关闭
/home/appinstall/kafka/bin/kafka-server-stop.sh
/home/songlk/appinstall/kafka/bin/kafka-server-stop.sh
/home/songlk/appinstall/kafka/bin/kafka-server-stop.sh
启动异常：
There is insufficient memory for the Java Runtime Environment to continue.
export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
------------------------
./kafka-topics.sh --describe --zookeeper localhost:2181 --topic test
8.创建主题
$>/home/appinstall/kafka/bin/kafka-topics.sh --create --zookeeper 59.110.173.251:2181 --replication-factor 3 --partitions 3 --topic test
[root@iz2zej15counp893693urlz bin]# /home/appinstall/kafka/bin/kafka-topics.sh --create --zookeeper 59.110.173.251:2181 --replication-factor 3 --partitions 3 --topic test
Created topic "test".
9.查看主题列表
$>/home/appinstall/kafka/bin/kafka-topics.sh --list --zookeeper 59.110.173.251:2181
/home/songlk/appinstall/kafka/bin/kafka-topics.sh --list --zookeeper 59.110.173.251:2181
/home/songlk/appinstall/kafka/bin/kafka-topics.sh --list --zookeeper 59.110.173.251:2181
[root@iz2zej15counp893693urlz bin]# /home/appinstall/kafka/bin/kafka-topics.sh --list --zookeeper 59.110.173.251:2181
test
10.启动控制台生产者
/home/songlk/appinstall/kafka/bin/kafka-console-producer.sh --broker-list 60.205.136.57:9092 --topic test
11.启动控制台消费者
/home/songlk/appinstall/kafka/bin/kafka-console-consumer.sh --bootstrap-server 60.205.136.57:9092 --topic test --from-beginning --zookeeper 60.205.136.57:2181
====================
12.在生产者控制台输入hello world
--------------------------------------------------------------------------------------------------------------