本文主要分析 ByteToMessageDecoder
,它是用来解决粘包半包处理主要处理类。
什么是粘包半包?
tcp是基于报文的协议,当使用netty传输数据时,可能一次读数据不是一个完整的数据,可能读取的是一半或者多余具体应用层数据的数据。
如何解决这样的问题呢?
可以用一个全局缓存池,每次解码器进行解码时,只能解码一个完整数据,不够或者多余则放到下次解码。
ByteToMessageDecoder 就是这样解决这样的问题的。
在netty读取数据时,会尝试一次性最多进行16次IO读取,每次读取都会调用 pipeline.fireChannelRead(byteBuf)
方法:
AbstractNioByteChannel$NioByteUnsafe
的read
方法:
do {
byteBuf = allocHandle.allocate(allocator);
allocHandle.lastBytesRead(doReadBytes(byteBuf));
if (allocHandle.lastBytesRead() <= 0) {
// nothing was read. release the buffer.
byteBuf.release();
byteBuf = null;
close = allocHandle.lastBytesRead() < 0;
if (close) {
// There is nothing left to read as we received an EOF.
readPending = false;
}
break;
}
allocHandle.incMessagesRead(1);
readPending = false;
pipeline.fireChannelRead(byteBuf);
byteBuf = null;
} while (allocHandle.continueReading());
这样,经过pipeline传递,如果有配置 ByteToMessageDecoder
的子类作为handler处理器,则会进入到ByteToMessageDecoder
的 channelRead
:方法:
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
if (msg instanceof ByteBuf) {
CodecOutputList out = CodecOutputList.newInstance();
try {
first = cumulation == null;
cumulation = cumulator.cumulate(ctx.alloc(),
first ? Unpooled.EMPTY_BUFFER : cumulation, (ByteBuf) msg);
callDecode(ctx, cumulation, out);
} catch (DecoderException e) {
throw e;
} catch (Exception e) {
throw new DecoderException(e);
} finally {
if (cumulation != null && !cumulation.isReadable()) {
numReads = 0;
cumulation.release();
cumulation = null;
} else if (++ numReads >= discardAfterReads) {
// We did enough reads already try to discard some bytes so we not risk to see a OOME.
// See https://github.com/netty/netty/issues/4275
numReads = 0;
discardSomeReadBytes();
}
int size = out.size();
firedChannelRead |= out.insertSinceRecycled();
fireChannelRead(ctx, out, size);
out.recycle();
}
} else {
ctx.fireChannelRead(msg);
}
}
cumulation = cumulator.cumulate(ctx.alloc(),
first ? Unpooled.EMPTY_BUFFER : cumulation, (ByteBuf) msg);
当 是第一次解析,则使用空ByteBuf和msg进行合并,否则就使用上一次没有用完的cumulation进行合并。
4. 调用 callDecode 对cumulation进行解码。
5. 接完码后,会在finally块中,调用fireChannelRead
进行数据传递。
ByteToMessageDecoder
的 callDecode
方法:
protected void callDecode(ChannelHandlerContext ctx, ByteBuf in, List
fireChannelRead(ctx, out, outSize);
往下一个handler传递。清空out数组。decodeRemovalReentryProtection
进行具体解码操作。ByteToMessageDecoder
的decodeRemovalReentryProtection
方法:
final void decodeRemovalReentryProtection(ChannelHandlerContext ctx, ByteBuf in, List
decode
方法进行具体解码操作。STATE_HANDLER_REMOVED_PENDING
状态,并调用对应时间总体的 ByteToMessageDecoder
解码器就是到这里了,它并没有给出具体的解码方法,只是收集读取到的字节数据,并调用子类具体decode方法进行解码。
接下来以几个netty自带解码器框架来进行分析。
基于行的解码器,即以 \n
作为分隔符,即一行就是一次解码后的内容。
LineBasedFrameDecoder
的 decode
方法:
@Override
protected final void decode(ChannelHandlerContext ctx, ByteBuf in, List
关键代码:
if (stripDelimiter) {
frame = buffer.readRetainedSlice(length);
buffer.skipBytes(delimLength);
} else {
frame = buffer.readRetainedSlice(length + delimLength);
}
return frame;
基于分隔符的解码器,和 LineBasedFrameDecoder 类似,只是LineBasedFrameDecoder可以配置多个分隔符,大师decode解码时,每次回以长度帧最小的分隔符进行解码。
固定长度解码器,即配置长度,每次只读取这么多数据。
FixedLengthFrameDecoder
的 decode
方法:
@Override
protected final void decode(ChannelHandlerContext ctx, ByteBuf in, List
可以用单独的header指定body大小的decoder,这个decoder最强大。
LengthFieldBasedFrameDecoder
构造方法为:
public LengthFieldBasedFrameDecoder(
ByteOrder byteOrder, int maxFrameLength, int lengthFieldOffset, int lengthFieldLength,
int lengthAdjustment, int initialBytesToStrip, boolean failFast) {
this.byteOrder = checkNotNull(byteOrder, "byteOrder");
checkPositive(maxFrameLength, "maxFrameLength");
checkPositiveOrZero(lengthFieldOffset, "lengthFieldOffset");
checkPositiveOrZero(initialBytesToStrip, "initialBytesToStrip");
if (lengthFieldOffset > maxFrameLength - lengthFieldLength) {
throw new IllegalArgumentException(
"maxFrameLength (" + maxFrameLength + ") " +
"must be equal to or greater than " +
"lengthFieldOffset (" + lengthFieldOffset + ") + " +
"lengthFieldLength (" + lengthFieldLength + ").");
}
this.maxFrameLength = maxFrameLength;
this.lengthFieldOffset = lengthFieldOffset;
this.lengthFieldLength = lengthFieldLength;
this.lengthAdjustment = lengthAdjustment;
this.lengthFieldEndOffset = lengthFieldOffset + lengthFieldLength;
this.initialBytesToStrip = initialBytesToStrip;
this.failFast = failFast;
}
前面几个为检查参数正确性,后面为基本赋值:
frameLength += lengthAdjustment + lengthFieldEndOffset;
actualFrameLength = frameLengthInt - initialBytesToStrip;
从Java doc注释几个例子来理解下整个decoder流程:
lengthFieldOffset=0 //
lengthFieldLength=2 //
lengthAdjustment=0 //
initialBytesToStrip=0 //
这样如果解析前的ByteBuf为左边,经过decoder解析出来完整的包为右边
BEFORE DECODE (14 bytes) AFTER DECODE (14 bytes)
+--------+----------------+ +--------+----------------+
| Length | Actual Content |----->| Length | Actual Content |
| 0x000C | "HELLO, WORLD" | | 0x000C | "HELLO, WORLD" |
+--------+----------------+ +--------+----------------+
lengthFieldOffset=0 //
lengthFieldLength=2 //
lengthAdjustment=0 //
initialBytesToStrip=2 //
最后的初始字节进位,则说明是跳过部分字节开始:
BEFORE DECODE (14 bytes) AFTER DECODE (12 bytes)
+--------+----------------+ +----------------+
| Length | Actual Content |----->| Actual Content |
| 0x000C | "HELLO, WORLD" | | "HELLO, WORLD" |
+--------+----------------+ +----------------+
lengthFieldOffset=0 //
lengthFieldLength=2 //
lengthAdjustment=-2 //
initialBytesToStrip=0 //
此时 lengthAdjustment,解密后为:
+--------+----------------+ +--------+----------------+
| Length | Actual Content |----->| Length | Actual Content |
| 0x000E | "HELLO, WORLD" | | 0x000E | "HELLO, WORLD" |
+--------+----------------+ +--------+----------------+
LengthFieldBasedFrameDecoder
的 decode
方法:
@Override
protected final void decode(ChannelHandlerContext ctx, ByteBuf in, List
仍然是decode方法:
LengthFieldBasedFrameDecoder
的 decode
方法:
protected Object decode(ChannelHandlerContext ctx, ByteBuf in) throws Exception {
if (discardingTooLongFrame) {
// 如果上一次处理,超出了帧大小,则将多余的字节丢弃。
discardingTooLongFrame(in);
}
if (in.readableBytes() < lengthFieldEndOffset) {
// 如果读取字节数,比长度字段偏移量都少,说明是个半包,直接退出
return null;
}
// actualLengthFieldOffset 表示长度字段偏移量
int actualLengthFieldOffset = in.readerIndex() + lengthFieldOffset;
// 获取帧长度
long frameLength = getUnadjustedFrameLength(in, actualLengthFieldOffset, lengthFieldLength, byteOrder);
// 如果长度小于0,则抛出异常
if (frameLength < 0) {
failOnNegativeLengthField(in, frameLength, lengthFieldEndOffset);
}
// 此时 frameLength 为 偏移量+调整量
frameLength += lengthAdjustment + lengthFieldEndOffset;
if (frameLength < lengthFieldEndOffset) {
// 长度调整完后,比偏移量还少,肯定有问题
failOnFrameLengthLessThanLengthFieldEndOffset(in, frameLength, lengthFieldEndOffset);
}
if (frameLength > maxFrameLength) {
// 大于最大长度
exceededFrameLength(in, frameLength);
return null;
}
// never overflows because it's less than maxFrameLength
int frameLengthInt = (int) frameLength;
if (in.readableBytes() < frameLengthInt) {
// 解析完之后,发现不够一个帧长度,说明是半包
return null;
}
if (initialBytesToStrip > frameLengthInt) {
// 跳过的字节数大于帧长度,直接报错
failOnFrameLengthLessThanInitialBytesToStrip(in, frameLength, initialBytesToStrip);
}
// 跳过帧长度
in.skipBytes(initialBytesToStrip);
// extract frame
int readerIndex = in.readerIndex();
// 实际长度
int actualFrameLength = frameLengthInt - initialBytesToStrip;
// 解析出帧
ByteBuf frame = extractFrame(ctx, in, readerIndex, actualFrameLength);
// 重置reader index
in.readerIndex(readerIndex + actualFrameLength);
return frame;
}
总体流程由上面注释。
本文从源码上,对 Netty 粘包半包原理,从总体思路,到具体 ByteToMessageDecoder
子类进行处理过程中decode方法分析。
关注博主公众号: 六点A君。
哈哈哈,一起研究Netty: