Netty中粘包拆包问题解决探讨
⭐️ 前言
开发的小伙伴们对于Netty并不陌生，本文就Netty粘包拆包问题及其解决方案做一个介绍，希望能对大家有所帮助。
⭐️ 什么是粘包拆包问题
我们知道，传统的IO是面向流的，而Netty（它的底层是Java NIO）是面向Buffer的。所以当发送很多体量比较小的消息时，消息会堆积在Buffer（例如send buffer）中，触发Flush后才会真正在网路上传输数据。这种情况下，接收端会接收到很多连在一起的体量较小的消息，这就产生了粘包；另外，当需要发送的消息体量较大，大到超出Buffer的最大容量时，只能先发送消息的一部分，剩下的部分会在稍晚一些发送，这样，一个大体量的消息被拆开了，于是就产生了拆包问题。
下文会演示粘包现象，并探讨粘包拆包的解决方案。
⭐️ netty handler 执行顺序
在演示之前，我们先来看看netty handler 的执行顺序，handler可以通过ChannelPipeline的addLast方法按顺序添加。总体来说，接收消息会沿着handler链路从前往后寻找InboundHandler依次处理；发送消息时，会沿着handler链路从后往前寻找OutboundHandler依次处理，若在handler链路中间的某个InboundHandler发送数据，则分两种情况：
1、调用ctx.writeAndFlush，从当前handler沿链路向前寻找OutboundHandler依次处理
2、调用ctx.channel().writeAndFlush，从handler链路的tail向前寻找OutboundHandler依次处理
⭐️ 日志设置
日志很重要，在java项目中，日志需要两个组件，日志门面和日志实现，日志门面这里采用slf4j，具体日志实现选择log4j，依赖如下，这里顺便给出netty的依赖。
<dependencies>
<dependency>
<groupId>io.nettygroupId>
<artifactId>netty-allartifactId>
<version>4.1.59.Finalversion>
dependency>
<dependency>
<groupId>log4jgroupId>
<artifactId>log4jartifactId>
<version>1.2.17version>
dependency>
<dependency>
<groupId>org.slf4jgroupId>
<artifactId>slf4j-apiartifactId>
<version>1.7.25version>
dependency>
dependencies>
log4j.properties对log进行相关设置
log4j.rootLogger=DEBUG,stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy/MM/dd HH:mm:ss,SSS} [%p]%C{1}-%m%n
⭐️ 粘包演示
这里客户端发送了三条消息，在最后一条消息发送时进行flush操作。
server端代码
public class Server {
public static void main(String[] args) {
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childOption(ChannelOption.SO_KEEPALIVE, true)
.childHandler(new ChannelInitializer<SocketChannel>() {
protected void initChannel(SocketChannel socketChannel) throws Exception {
ChannelPipeline pipeline = socketChannel.pipeline();
pipeline.addLast(new LoggingHandler())
.addLast(new StringDecoder())
.addLast(new ServerTestHandler());
}
});
System.out.println("server ready");
ChannelFuture sync = bootstrap.bind(8888).sync();
sync.channel().closeFuture().sync();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}
}
客户端代码
public class Client {
public static void main(String[] args) {
EventLoopGroup workGroup = new NioEventLoopGroup();
Bootstrap bootstrap = new Bootstrap();
try {
bootstrap.group(workGroup)
.channel(NioSocketChannel.class)
.handler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel socketChannel) throws Exception {
ChannelPipeline pipeline = socketChannel.pipeline();
pipeline.addLast(new LoggingHandler())
.addLast(new StringEncoder(CharsetUtil.UTF_8));
}
});
System.out.println("client ok");
ChannelFuture localhost = bootstrap.connect("localhost", 8888).sync();
// 发送消息
String msg1 = "hello world";
localhost.channel().write(msg1);
String msg2 = "my name is eryx";
localhost.channel().write(msg2);
String msg3 = "i am robot";
localhost.channel().writeAndFlush(msg3);
localhost.channel().closeFuture().sync();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
workGroup.shutdownGracefully();
}
}
}
这里定义一个ServerTestHandler，用以显示客户端发给服务端的数据
public class ServerTestHandler extends ChannelInboundHandlerAdapter {
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
System.out.println("来自client的消息：" + String.valueOf(msg));
}
}
客户端日志
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 68 65 6c 6c 6f 20 77 6f 72 6c 64 |hello world |
+--------+-------------------------------------------------+----------------+
2023/11/17 12:04:15,780 [DEBUG]AbstractInternalLogger-[id: 0x07a76a5a, L:/127.0.0.1:54667 - R:localhost/127.0.0.1:8888] WRITE: 15B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 6d 79 20 6e 61 6d 65 20 69 73 20 65 72 79 78 |my name is eryx |
+--------+-------------------------------------------------+----------------+
2023/11/17 12:04:15,780 [DEBUG]AbstractInternalLogger-[id: 0x07a76a5a, L:/127.0.0.1:54667 - R:localhost/127.0.0.1:8888] WRITE: 10B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 69 20 61 6d 20 72 6f 62 6f 74 |i am robot |
+--------+-------------------------------------------------+----------------+
2023/11/17 12:04:15,780 [DEBUG]AbstractInternalLogger-[id: 0x07a76a5a, L:/127.0.0.1:54667 - R:localhost/127.0.0.1:8888] FLUSH
服务端日志
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 68 65 6c 6c 6f 20 77 6f 72 6c 64 6d 79 20 6e 61 |hello worldmy na|
|00000010| 6d 65 20 69 73 20 65 72 79 78 69 20 61 6d 20 72 |me is eryxi am r|
|00000020| 6f 62 6f 74 |obot |
+--------+-------------------------------------------------+----------------+
来自client的消息：hello worldmy name is eryxi am robot
可以看到，在客户端分3次写入的消息，服务端接收到时，三条消息连接在了一起，发生了粘包问题。
⭐️ 如何解决粘包拆包
为解决粘拆包问题，netty提供了一些解码器，这里介绍两个：
1、FixedLengthFrameDecoder 固定长度帧解码器
2、LengthFieldBasedFrameDecoder 以长度字段为基础的解码器
FixedLengthFrameDecoder
FixedLengthFrameDecoder对于固定长度的消息很方便，我们对server做一些调整，这里客户端发送的虽然不是固定长度消息，但可以更清洗的理解FixedLengthFrameDecoder的效果。
server端代码
public class Server {
public static void main(String[] args) {
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childOption(ChannelOption.SO_KEEPALIVE, true)
.childHandler(new ChannelInitializer<SocketChannel>() {
protected void initChannel(SocketChannel socketChannel) throws Exception {
ChannelPipeline pipeline = socketChannel.pipeline();
pipeline.addLast(new LoggingHandler())
// 以5为帧的固定长度
.addLast(new FixedLengthFrameDecoder(5))
.addLast(new StringDecoder())
.addLast(new ServerTestHandler());
}
});
System.out.println("server ready");
ChannelFuture sync = bootstrap.bind(8888).sync();
sync.channel().closeFuture().sync();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}
}
客户端的代码和日志均与上一小节相同，而server端的日志输出如下：
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 68 65 6c 6c 6f 20 77 6f 72 6c 64 6d 79 20 6e 61 |hello worldmy na|
|00000010| 6d 65 20 69 73 20 65 72 79 78 69 20 61 6d 20 72 |me is eryxi am r|
|00000020| 6f 62 6f 74 |obot |
+--------+-------------------------------------------------+----------------+
来自client的消息：hello
来自client的消息： worl
来自client的消息：dmy n
来自client的消息：ame i
来自client的消息：s ery
来自client的消息：xi am
来自client的消息： robo
可见，server端解析的消息，每一条都有5个字符。
LengthFieldBasedFrameDecoder
LengthFieldBasedFrameDecoder包含一些参数，说明如下：
（1） maxFrameLength：发送的数据包最大长度
（2） lengthFieldOffset：长度域偏移量，指的是长度域位于整个数据包字节数组中的下标
（3） lengthFieldLength：长度域的字节数长度
（4） lengthAdjustment：长度域的偏移量矫正
（5） initialBytesToStrip：丢弃的起始字节数
在只关注消息本身和其长度的情况下，LengthFieldBasedFrameDecoder可以和LengthFieldPrepender配合使用，LengthFieldPrepender可以指定一个参数，即长度域的字节数长度。
server端代码
public class Server {
public static void main(String[] args) {
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childOption(ChannelOption.SO_KEEPALIVE, true)
.childHandler(new ChannelInitializer<SocketChannel>() {
protected void initChannel(SocketChannel socketChannel) throws Exception {
ChannelPipeline pipeline = socketChannel.pipeline();
pipeline.addLast(new LoggingHandler())
.addLast(new LengthFieldBasedFrameDecoder(1024,0, 4, 0, 4))
.addLast(new StringDecoder())
.addLast(new ServerTestHandler());
}
});
System.out.println("server ready");
ChannelFuture sync = bootstrap.bind(8888).sync();
sync.channel().closeFuture().sync();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}
}
client端代码
public class Client {
public static void main(String[] args) {
EventLoopGroup workGroup = new NioEventLoopGroup();
Bootstrap bootstrap = new Bootstrap();
try {
bootstrap.group(workGroup)
.channel(NioSocketChannel.class)
.handler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel socketChannel) throws Exception {
ChannelPipeline pipeline = socketChannel.pipeline();
pipeline.addLast(new LoggingHandler())
.addLast(new LengthFieldPrepender(4))
.addLast(new StringEncoder(CharsetUtil.UTF_8));
}
});
System.out.println("client ok");
ChannelFuture localhost = bootstrap.connect("localhost", 8888).sync();
// 发送消息
String msg1 = "hello world";
localhost.channel().write(msg1);
String msg2 = "my name is eryx";
localhost.channel().write(msg2);
String msg3 = "i am robot";
localhost.channel().writeAndFlush(msg3);
localhost.channel().closeFuture().sync();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
workGroup.shutdownGracefully();
}
}
}
客户端日志
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 0b |.... |
+--------+-------------------------------------------------+----------------+
2023/11/17 15:16:19,338 [DEBUG]AbstractInternalLogger-[id: 0x5598fbcb, L:/127.0.0.1:50147 - R:localhost/127.0.0.1:8888] WRITE: 11B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 68 65 6c 6c 6f 20 77 6f 72 6c 64 |hello world |
+--------+-------------------------------------------------+----------------+
2023/11/17 15:16:19,341 [DEBUG]AbstractInternalLogger-[id: 0x5598fbcb, L:/127.0.0.1:50147 - R:localhost/127.0.0.1:8888] WRITE: 4B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 0f |.... |
+--------+-------------------------------------------------+----------------+
2023/11/17 15:16:19,342 [DEBUG]AbstractInternalLogger-[id: 0x5598fbcb, L:/127.0.0.1:50147 - R:localhost/127.0.0.1:8888] WRITE: 15B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 6d 79 20 6e 61 6d 65 20 69 73 20 65 72 79 78 |my name is eryx |
+--------+-------------------------------------------------+----------------+
2023/11/17 15:16:19,342 [DEBUG]AbstractInternalLogger-[id: 0x5598fbcb, L:/127.0.0.1:50147 - R:localhost/127.0.0.1:8888] WRITE: 4B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 0a |.... |
+--------+-------------------------------------------------+----------------+
2023/11/17 15:16:19,342 [DEBUG]AbstractInternalLogger-[id: 0x5598fbcb, L:/127.0.0.1:50147 - R:localhost/127.0.0.1:8888] WRITE: 10B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 69 20 61 6d 20 72 6f 62 6f 74 |i am robot |
+--------+-------------------------------------------------+----------------+
2023/11/17 15:16:19,342 [DEBUG]AbstractInternalLogger-[id: 0x5598fbcb, L:/127.0.0.1:50147 - R:localhost/127.0.0.1:8888] FLUSH
服务端日志
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 0b 68 65 6c 6c 6f 20 77 6f 72 6c 64 00 |....hello world.|
|00000010| 00 00 0f 6d 79 20 6e 61 6d 65 20 69 73 20 65 72 |...my name is er|
|00000020| 79 78 00 00 00 0a 69 20 61 6d 20 72 6f 62 6f 74 |yx....i am robot|
+--------+-------------------------------------------------+----------------+
来自client的消息：hello world
来自client的消息：my name is eryx
来自client的消息：i am robot
2023/11/17 15:16:19,383 [DEBUG]AbstractInternalLogger-[id: 0xb4d43375, L:/127.0.0.1:8888 - R:/127.0.0.1:50147] READ COMPLETE
可见，每一天消息的都被正确的解析了，消息的界限明确了，粘包问题解决了！！！
笔者水平有限，若有不对的地方欢迎评论指正！