zookeeper-3.4.10.tar.gz下载地址
链接:https://pan.baidu.com/s/1PS_fZ3AFPKzJDBT7PDsu3w
提取码:v2ul
kafka_2.11-1.0.0.tgz下载地址
链接:https://pan.baidu.com/s/1H8uvKNYmFnDbBoEb0BPkEw
提取码:g2j9
Zookeeper配置
解压zookeeper-3.4.10.tar.gz
1、复制配置文件
cp zoo_sample.cfg zoo.cfg
2、修改配置文件(三台机器配置文件相同)
vi zoo.cfg
dataDir=/home/wucan/zookeeper/zookeeper-3.4.10/data(事先创建好该目录)
clientPort=2181
server.1=192.168.43.209:2888:3888
server.2=192.168.43.210:2888:3888
server.3=192.168.43.211:2888:3888
3、在数据文件夹下新建myid文件
vi myid
192.168.43.209机器输入1,保存退出;
192.168.43.210机器输入2,保存退出;
192.168.43.211机器输入3,保存退出;
4、配置环境变量
vi /etc/profile (在最后添加如下内容)
export ZOOKEEPER_HOME=/home/wucan/zookeeper/zookeeper-3.4.10
export PATH=$PATH:$ZOOKEEPER_HOME/bin:$ZOOKEEPER_HOME/conf
保存后输入source /etc/profile命令使修改生效。
5、基本操作命令
zkServer.sh start (启动)
zkServer.sh restart (重启)
zkServer.sh status (查看状态)
zkServer.sh stop (关闭)
nohup bin/zkServer.sh start conf/zoo.cfg &(开启服务)
Kafka配置
一、修改server.properties配置文件(三台机器配置文件除id外,都相同)
broker.id=1
host.name=192.168.43.209
port=9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=../logs(事先创建好该目录)
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
message.max.byte=5242880
default.replication.factor=2
replica.fetch.max.bytes=5242880
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=192.168.43.209:2181, 192.168.43.210:2181, 192.168.43.211:2181
zookeeper.connection.timeout.ms=6000
delete.topic.enable=true(设置topic可以被删除)
二、开启服务
nohup bin/kafka-server-start.sh config/server.properties &
问题及解决方案:
1.org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for itheima-0: 30040 ms has passed since batch creation plus linger time
解决方案:
在客户端服务器hosts中加入ip hostname映射
2.2019-03-20 16:26:36.789 INFO 18445 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-2, groupId=sce] Group coordinator 10.1.0.41:9092 (id: 2147483646 rack: null) is unavailable or invalid, will attempt rediscovery
解决方案:
在部署Kafka的服务器hosts中加入ip hostname映射