关于在云服务器部署docker后,部署kafka,外网无法访问解决方式
这里主要测试的华为云服务器,但是我想所有的云服务器应该是大同小异的,其次是外网访问阶段,kafka部署后无法访问,以及在容器中更改了server.properties,重启容器还是不正确,通过查找发现服务器可以做以下修改之后就能成功使用kafka tools和代码访问kafka了
这里使用的kafka和zookeeper
docker pull wurstmeister/zookeeper
docker pull wurstmeister/kafka
1.进入hosts文件
vi /etc/hosts
//添加公网ip
公网ip
2.删除已经部署好的kafka,也可以不做删除,但重新给端口,命名就可以了
docker run -d --name kafka -p 9093:9093 -e KAFKA_BROKER_ID=0 -e KAFKA_ZOOKEEPER_CONNECT=(一定要是公网ip,否则远程是无法操作kafka的,kaka tools也是无法使用的):2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://(一定要是公网ip,否则远程是无法操作kafka的,kaka tools也是无法使用的):9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -t wurstmeister/kafka
这里着重说明以下,我通过查询以下的网站,均试过了,发现只有这样改才能做到kafka的外部访问,如果访问不了大家也可以查看以下的博客试一试
https://blog.csdn.net/xiaoyu_bd/article/details/82496257
https://blog.csdn.net/chenyulancn/article/details/79499401
这个时候其实已经可以访问kafka了,但是有些人还是无法访问,究其原因是因为在云服务器的出入站规则中没有增加2181端口和个人给的kafka端口,所以无法访问的,以下是我的出入站规则
3.现在就能通过代码测试是否可以访问了,我这里创建了一个test的topic;然后还可以通过kafka tools查看是否有kafka了.
package com.project.spark.util
/**
* @Author ：
* @Date ： Create in 2019-07-17 14:40
* @Description ：<描述>
*/
object Producer extends App {
import java.util.Properties
import org.apache.kafka.clients.producer.{KafkaProducer, ProducerRecord}
val topic = "par"
val brokers = "公网ip:9092"
val props = new Properties()
props.put("bootstrap.servers", brokers)
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer")
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer")
val producer = new KafkaProducer[String, String](props)
val t = System.currentTimeMillis()
var msg = "启动成功"
val record = new ProducerRecord[String, String](topic, "key", msg)
producer.send(record)
producer.close()
}
关于有些小伙伴在这些都做成功之后,在控制台begin topic的时候发现有以下错误
[2019-07-18 05:45:26,746] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-07-18 05:45:26,849] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-07-18 05:45:26,950] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-07-18 05:45:27,152] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-07-18 05:45:27,654] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-07-18 05:45:28,558] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
这个原因是因为你开启的时候ip没有写公网ip,所以会造成无法连接,下面是我测试的几条命令
kafka-console-consumer.sh --bootstrap-server 公网ip:给的端口 --topic par --from-beginning
kafka-console-consumer.sh --bootstrap-server 公网ip:给的端口 --topic par --from-beginning