kafka命令行消费kafka-console-consumer.sh 报错 whose size is larger than the fetch size 1048576

kafka命令行消费kafka-console-consumer.sh 报错 whose size is larger than the fetch size 1048576

问题

[appuser@node01 kafka-0.10.0]$ bin/kafka-console-consumer.sh --bootstrap-server node01:9092,node02:9092,node03:9092 --topic topic_test1 --new-consumer
[2020-04-02 14:17:30,184] ERROR Error processing message, terminating consumer process:  (kafka.tools.ConsoleConsumer$)
org.apache.kafka.common.errors.RecordTooLargeException: There are some messages at [Partition=Offset]: {topic_test1-0=74691861} whose size is larger than the fetch size 1048576 and hence cannot be ever returned. Increase the fetch size, or decrease the maximum message size the broker will allow.
Processed a total of 0 messages

分析

kafka 单条消息过大，消费者读取数据大小和消费端配置max.partition.fetch.bytes（一次fetch请求，从一个partiti