Linux 安装 kafka（单机集群）
前言
因为个人比较穷，所以只用一台服务器作为示例，请大家见谅，只要大家会举一反三，这些都不是问题，写得不好的地方，请大家多多包涵，阿导不胜感激。
准备工作
- centos 7
- zookeeper
- kafka
因为花名阿导，所以我会在根目录创建一个文件夹 dao,然后创建 zookeeper 和 kafka 两个文件夹，具体命令如下所示：
cd /
mkdir dao
cd dao
mkdir zookeeper kafka
zookeeper
好了，话不多说，先来一波 zookeeper 的配置，因为过去搭建过主从结构的 zookeeper 集群，我这边文件夹命名没有去改动，按照道理来说，这三个 zookeeper 节点地位相当，没有主次之分，我这边创建了8个文件夹，首先在 zookeeper 目录下创建 data 和 logs,然后分别在data 和 logs 下面创建 master 、slave_1 、slave_2,命令如下所示：
cd /dao/zookeeper
mkdir data logs
cd data
mkdir master slave_1 slave_2
cd …/logs
mkdir master slave_1 slave_2
这个时候需要下载一个 zookeeper,然后解压，拷贝三份，具体操作如下：
cd /dao/zookeeper
wget http://mirrors.hust.edu.cn/apache/zookeeper/zookeeper-3.4.10/zookeeper-3.4.10.tar.gz
tar -zxvf zookeeper-3.4.10.tar.gz
cp -r zookeeper-3.4.10 zookeeper_master
cp -r zookeeper-3.4.10 zookeeper_slave_1
mv zookeeper-3.4.10 zookeeper_slave_2
目录如下图
嗯，接下来看看 zookeeper 重要的配置了，首先在 data 下面三个文件夹内添加 myid 文件，文件内容分别为 1，2，3，命令如下：
vi /dao/zookeeper/data/master/myid
输入 i ，然后 1，最后保存退出（wq）
vi /dao/zookeeper/data/slave_1/myid
输入 i ，然后 2，最后保存退出（wq）
vi /dao/zookeeper/data/slave_2/myid
输入 i ，然后 3，最后保存退出（wq）
然后我们看看 zoo.cfg 的配置
vi /dao/zookeeper/zookeeper_master/conf/zoo.cfg
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/dao/zookeeper/data/master
dataLogDir=/dao/zookeeper/logs/master
# the port at which the clients will connect
clientPort=2191
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=127.0.0.1:2888:3888
server.2=127.0.0.1:2889:3889
server.3=127.0.0.1:2890:3890
vi /dao/zookeeper/zookeeper_slave_1/conf/zoo.cfg
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/dao/zookeeper/data/slave_1
dataLogDir=/dao/zookeeper/logs/slave_1
# the port at which the clients will connect
clientPort=2192
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=127.0.0.1:2888:3888
server.2=127.0.0.1:2889:3889
server.3=127.0.0.1:2890:3890
vi /dao/zookeeper/zookeeper_slave_2/conf/zoo.cfg
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/dao/zookeeper/data/slave_2
dataLogDir=/dao/zookeeper/logs/slave_2
# the port at which the clients will connect
clientPort=2193
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=127.0.0.1:2888:3888
server.2=127.0.0.1:2889:3889
server.3=127.0.0.1:2890:3890
配置(主要就是 clientPort 配置不同 )已经搞定，是时候来一波启动了，当然你可以写脚本，启动方式都差不多，到 zookeeper 文件夹下，然后启动启动就行了，具体命令如下：
cd /dao/zookeeper/zookeeper_master/bin/
./zkServer.sh start
cd /dao/zookeeper/zookeeper_slave_1/bin/
./zkServer.sh start
cd /dao/zookeeper/zookeeper_slave_2/bin/
./zkServer.sh start
其他就不说了哈，反正我不会（嘿嘿），直接看看 kafka
kafka
一样先创建一个 logs,然后创建三个文件夹 kafka_1 、kafka_2 、kafka_3,具体命令如下：
cd /dao/kafka
mkdir logs
cd logs
mkdir kafka_1 kafka_2 kafka_3
然后下载 kafka ,解压并拷贝三份
wget http://mirrors.hust.edu.cn/apache/kafka/2.0.1/kafka_2.11-2.0.1.tgz
tar -zxvf kafka_2.11-2.0.1.tgz
cp -r kafka_2.11-2.0.1 kafka_1
cp -r kafka_2.11-2.0.1 kafka_2
mv kafka_2.11-2.0.1 kafka_3
又到了看看主要配置的时候了，来来跟踪阿导的脚本，如下操作
vi /dao/kafka/kafka_1/config/server.properties
broker.id=0
port=9092
listeners=PLAINTEXT://{内网 IP}:9092
advertised.listeners=PLAINTEXT://{外网 IP 或者 域名}:9092
message.max.byte=5242880
default.replication.factor=2
replica.fetch.max.bytes=5242880
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/dao/kafka/logs/kafka_1/
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=localhost:2191,localhost:2192,localhost:2193
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
vi /dao/kafka/kafka_1/config/consumer.properties
bootstrap.servers={外网 IP}:9092
group.id=logGroup
zookeeper.connect=localhost:2191,localhost:2192,localhost:2193
vi /dao/kafka/kafka_2/config/server.properties
broker.id=1
port=9093
listeners=PLAINTEXT://{内网 IP}:9093
advertised.listeners=PLAINTEXT://{外网 IP 或者 域名}:9093
message.max.byte=5242880
default.replication.factor=2
replica.fetch.max.bytes=5242880
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/dao/kafka/logs/kafka_2/
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=localhost:2191,localhost:2192,localhost:2193
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
vi /dao/kafka/kafka_2/config/consumer.properties
bootstrap.servers={外网 IP}:9093
group.id=logGroup
zookeeper.connect=localhost:2191,localhost:2192,localhost:2193
vi /dao/kafka/kafka_3/config/server.properties
broker.id=2
port=9094
listeners=PLAINTEXT://{内网 IP}:9094
advertised.listeners=PLAINTEXT://{外网 IP 或者 域名}:9094
message.max.byte=5242880
default.replication.factor=2
replica.fetch.max.bytes=5242880
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/dao/kafka/logs/kafka_2/
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=localhost:2191,localhost:2192,localhost:2193
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
vi /dao/kafka/kafka_3/config/consumer.properties
bootstrap.servers={外网 IP}:9094
group.id=logGroup
zookeeper.connect=localhost:2191,localhost:2192,localhost:2193
总结一下，主要以下配置不同
- server.properties
broker.id
port
listeners
advertised.listeners
log.dirs
- consumer.properties
bootstrap.servers={外网 IP}:9094
启动试试
sh /dao/kafka/kafka_1/bin/kafka-server-start.sh /dao/kafka/kafka_1/config/server.properties &
sh /dao/kafka/kafka_2/bin/kafka-server-start.sh /dao/kafka/kafka_2/config/server.properties &
sh /dao/kafka/kafka_3/bin/kafka-server-start.sh /dao/kafka/kafka_3/config/server.properties &
启动成功展示如下,一般启动出错都是你的 IP 没写对，我也是摸滚了好久。。。
需要云服务器的不要错过优惠
阿里云低价购买云服务，值得一看