kafka集群搭建

系统：UBUNTU12.04和UBUNTU14.04

kafka版本：kafka_2.10-0.8.1.1

服务器数量3台，分别为：

1.OS：ubuntu12.04，IP：192.168.1.1

2.OS：ubuntu12.04，IP：192.168.1.2

3.OS：ubuntu14.04，IP：192.168.1.3

kafka的集群的实现包含2个部分：

1.zookeeper的集群，通过修改zookeeper.properties实现。

2.kafka的集群，通过修改server.properties实现。

1.修改zookeeper.properties文件，3台服务器都按照以下格式配置
#
dataDir=/home/com/zookeeper
# the port at which the clients will connect
clientPort=2181
# disable the per-ip limit on the number of connections since this is a non-production config
maxClientCnxns=0
initLimit=5 
syncLimit=2
#server的数量必须大于等于3个且不能为偶数
server.1=192.168.1.1:2888:3888
server.2=192.168.1.2:2888:3888
server.3=192.168.1.3:2888:3888

新建文件/home/com/zookeeper/myid

文件内写入对应的id的值：1

id的值需要和zookeeper.properties文件中的server.x的值相对应。

2.修改server.properties文件

############################# Server Basics #############################


# The id of the broker. This must be set to a unique integer for each broker.
#每台服务器都配置不一样的值
#192.168.1.1为0，192.168.1.2为1，192.168.1.3为2，
broker.id=0


############################# Socket Server Settings #############################


# The port the socket server listens on
port=9092


# Hostname the broker will bind to. If not set, the server will bind to all interfaces
#host.name=localhost
#填写对应服务器ip即可
host.name=192.168.1.1
# Hostname the broker will advertise to producers and consumers. If not set, it uses the
# value for "host.name" if configured.  Otherwise, it will use the value returned from
# java.net.InetAddress.getCanonicalHostName().
#advertised.host.name=
advertised.host.name=192.168.1.1


############################# Zookeeper #############################


# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
#zookeeper.connect=localhost:2181
zookeeper.connect=192.168.1.1:2181,192.168.1.2:2181,192.168.1.3:2181
# Timeout in ms for connecting to zookeeper
zookeeper.connection.timeout.ms=1000000

集群配置完成后，启动zookeeper服务

bin/zookeeper-server-start.sh config/zookeeper.properties 

启动kafka服务

bin/kafka-server-start.sh config/server.properties