netty http2 server侧的核心逻辑个人认为,主要在编解码处理器和Stream Transform Channel这块,分别处理Http2 消息帧的编解码,以及连接的多流处理机制。对应用的处理类分别:
ChannelHandler | Desc |
---|---|
io.netty.handler.codec.http2.Http2FrameCodec | 负责http2帧和消息的编解码 |
io.netty.handler.codec.http2.Http2MultiplexHandler | 负责流的连接通道复用,将Stream转为Http2MultiplexHandlerStreamChannel |
对于netty http2 server的搭建代码,可以见前面的文章:HTTP2: netty http2 server demo
Http2FrameCodec会对Header和Data帧进行解码,先看看调用栈:
对帧的解码第一个核心逻辑是在io.netty.handler.codec.http2.DefaultHttp2FrameReader#readFrame(ChannelHandlerContext, ByteBuf, Http2FrameListener),此时ChannelHandlerContext对应的Channel是NioSocketChannel,还没有到StreamChannel。看看该方法的代码逻辑:
@Override
public void readFrame(ChannelHandlerContext ctx, ByteBuf input, Http2FrameListener listener)
throws Http2Exception {
if (readError) {
input.skipBytes(input.readableBytes());
return;
}
try {
do {
if (readingHeaders) {
processHeaderState(input);
if (readingHeaders) {
// Wait until the entire header has arrived.
return;
}
}
// The header is complete, fall into the next case to process the payload.
// This is to ensure the proper handling of zero-length payloads. In this
// case, we don't want to loop around because there may be no more data
// available, causing us to exit the loop. Instead, we just want to perform
// the first pass at payload processing now.
processPayloadState(ctx, input, listener);
if (!readingHeaders) {
// Wait until the entire payload has arrived.
return;
}
} while (input.isReadable());
} catch (Http2Exception e) {
readError = !Http2Exception.isStreamError(e);
throw e;
} catch (RuntimeException e) {
readError = true;
throw e;
} catch (Throwable cause) {
readError = true;
PlatformDependent.throwException(cause);
}
}
该方法会对Socket中的字节进行解码,首先是处理帧头的数据,由processHeaderState(ByteBuf)处理,该方法会读取payloadLength、frameType、flags以及streamId信息,并进行正确性校验,如果校验失败,会抛出相应的Http2Exception异常。代码如下:
private void processHeaderState(ByteBuf in) throws Http2Exception {
if (in.readableBytes() < FRAME_HEADER_LENGTH) {
// Wait until the entire frame header has been read.
return;
}
// Read the header and prepare the unmarshaller to read the frame.
payloadLength = in.readUnsignedMedium();
if (payloadLength > maxFrameSize) {
throw connectionError(FRAME_SIZE_ERROR, "Frame length: %d exceeds maximum: %d", payloadLength,
maxFrameSize);
}
frameType = in.readByte();
flags = new Http2Flags(in.readUnsignedByte());
streamId = readUnsignedInt(in);
// We have consumed the data, next time we read we will be expecting to read the frame payload.
readingHeaders = false;
switch (frameType) {
case DATA:
verifyDataFrame();
break;
case HEADERS:
verifyHeadersFrame();
break;
case PRIORITY:
verifyPriorityFrame();
break;
case RST_STREAM:
verifyRstStreamFrame();
break;
case SETTINGS:
verifySettingsFrame();
break;
case PUSH_PROMISE:
verifyPushPromiseFrame();
break;
case PING:
verifyPingFrame();
break;
case GO_AWAY:
verifyGoAwayFrame();
break;
case WINDOW_UPDATE:
verifyWindowUpdateFrame();
break;
case CONTINUATION:
verifyContinuationFrame();
break;
default:
// Unknown frame type, could be an extension.
verifyUnknownFrame();
break;
}
}
当帧头信息处理完后,就处理fram payload了,这个逻辑由processPayloadState(ChannelHandlerContext, ByteBuf, Http2FrameListener)方法完成。它会根据frameType进行执行相应解码方法,当把帧的数据处理好后,再调用Http2FrameListener对应的onxxxx()方法,执行下一步的逻辑。processPayloadState(ChannelHandlerContext, ByteBuf, Http2FrameListener)方法的代码逻辑如下:
private void processPayloadState(ChannelHandlerContext ctx, ByteBuf in, Http2FrameListener listener)
throws Http2Exception {
if (in.readableBytes() < payloadLength) {
// Wait until the entire payload has been read.
return;
}
// Only process up to payloadLength bytes.
int payloadEndIndex = in.readerIndex() + payloadLength;
// We have consumed the data, next time we read we will be expecting to read a frame header.
readingHeaders = true;
// Read the payload and fire the frame event to the listener.
switch (frameType) {
case DATA:
//处理data帧数据
readDataFrame(ctx, in, payloadEndIndex, listener);
break;
case HEADERS:
//处理Header首帧数据
readHeadersFrame(ctx, in, payloadEndIndex, listener);
break;
...
case CONTINUATION:
//处理Header或PushPromise的连续帧数据
readContinuationFrame(in, payloadEndIndex, listener);
break;
default:
readUnknownFrame(ctx, in, payloadEndIndex, listener);
break;
}
in.readerIndex(payloadEndIndex);
}
Header帧体的处理分两种情况,分别是Header首帧和首帧后面跟着的CONTINUATION Header帧, 两种处理方法在processHeaderState()方法中的校验逻辑也不同,如下:
对于Header首帧的校验:
// Header首帧校验
private void verifyHeadersFrame() throws Http2Exception {
verifyAssociatedWithAStream();
verifyNotProcessingHeaders();
verifyPayloadLength(payloadLength);
int requiredLength = flags.getPaddingPresenceFieldLength() + flags.getNumPriorityBytes();
if (payloadLength < requiredLength) {
throw streamError(streamId, FRAME_SIZE_ERROR,
"Frame length too small." + payloadLength);
}
}
//Continuation 帧的校验
private void verifyContinuationFrame() throws Http2Exception {
verifyAssociatedWithAStream();
verifyPayloadLength(payloadLength);
if (headersContinuation == null) {
throw connectionError(PROTOCOL_ERROR, "Received %s frame but not currently processing headers.",
frameType);
}
if (streamId != headersContinuation.getStreamId()) {
throw connectionError(PROTOCOL_ERROR, "Continuation stream ID does not match pending headers. "
+ "Expected %d, but received %d.", headersContinuation.getStreamId(), streamId);
}
if (payloadLength < flags.getPaddingPresenceFieldLength()) {
throw streamError(streamId, FRAME_SIZE_ERROR,
"Frame length %d too small for padding.", payloadLength);
}
}
private void verifyNotProcessingHeaders() throws Http2Exception {
if (headersContinuation != null) {
throw connectionError(PROTOCOL_ERROR, "Received frame of type %s while processing headers on stream %d.",
frameType, headersContinuation.getStreamId());
}
}
private void verifyPayloadLength(int payloadLength) throws Http2Exception {
if (payloadLength > maxFrameSize) {
throw connectionError(PROTOCOL_ERROR, "Total payload length %d exceeds max frame length.", payloadLength);
}
}
private void verifyAssociatedWithAStream() throws Http2Exception {
if (streamId == 0) {
throw connectionError(PROTOCOL_ERROR, "Frame of type %s must be associated with a stream.", frameType);
}
}
Header首帧校验项:
Item | Method | Exception |
---|---|---|
streamId是否正常(!=0) | verifyAssociatedWithAStream | 异常:streamId==0 |
headersContinuation 是否为空 | verifyNotProcessingHeaders | headersContinuation!=null,headersContinuation是处理continuation header帧的一种手段,出现headersContinuation!=null时,说明流的帧数据出现错乱 |
payloadLength<=maxFrameSize | verifyPayloadLength | 帧的大小超出了上限 |
payloadLength >= (flags.getPaddingPresenceFieldLength() + flags.getNumPriorityBytes()) | 帧体长度太小 |
Header帧的处理逻辑在readHeadersFrame(ChannelHandlerContext, ByteBuf, int, Http2FrameListener)方法中, 该方法有个priority information的处理分支,这个暂时不看。代码逻辑如下:
private void readHeadersFrame(final ChannelHandlerContext ctx, ByteBuf payload, int payloadEndIndex,
Http2FrameListener listener) throws Http2Exception {
final int headersStreamId = streamId;
final Http2Flags headersFlags = flags;
final int padding = readPadding(payload);
verifyPadding(padding);
// The callback that is invoked is different depending on whether priority information
// is present in the headers frame.
if (flags.priorityPresent()) {
...
return;
}
// The priority fields are not present in the frame. Prepare a continuation that invokes
// the listener callback without priority information.
headersContinuation = new HeadersContinuation() {
@Override
public int getStreamId() {
return headersStreamId;
}
@Override
public void processFragment(boolean endOfHeaders, ByteBuf fragment, int len,
Http2FrameListener listener) throws Http2Exception {
final HeadersBlockBuilder hdrBlockBuilder = headersBlockBuilder();
hdrBlockBuilder.addFragment(fragment, len, ctx.alloc(), endOfHeaders);
if (endOfHeaders) {
listener.onHeadersRead(ctx, headersStreamId, hdrBlockBuilder.headers(), padding,
headersFlags.endOfStream());
}
}
};
// Process the initial fragment, invoking the listener's callback if end of headers.
int len = lengthWithoutTrailingPadding(payloadEndIndex - payload.readerIndex(), padding);
headersContinuation.processFragment(flags.endOfHeaders(), payload, len, listener);
resetHeadersContinuationIfEnd(flags.endOfHeaders());
}
该方法会创建一个HeadersContinuation对象,主要是用于对Continuation Header帧数据的累加。
帧数据的读取的代码逻辑是通过headersContinuation.processFragment()方法去读取,并继续解析向下调用,处理完后关闭并销毁headersContinuation对象。代码逻辑:
int len = lengthWithoutTrailingPadding(payloadEndIndex - payload.readerIndex(), padding);
headersContinuation.processFragment(flags.endOfHeaders(), payload, len, listener);
resetHeadersContinuationIfEnd(flags.endOfHeaders());
Header 处理完成后关闭并销毁headersContinuation对象:
private void resetHeadersContinuationIfEnd(boolean endOfHeaders) {
if (endOfHeaders) {
closeHeadersContinuation();
}
}
private void closeHeadersContinuation() {
if (headersContinuation != null) {
headersContinuation.close();
headersContinuation = null;
}
}
HeadersContinuation.processFragment()方法会调用HeadersBlockBuilder.addFragment()方法将Header帧的数据读入另一个ByteBuf对象headerBlock中,在Continuation Header帧数据读取时,也是同样的逻辑会读入到headerBlock中,这样Header完整的字节数据都会在headerBlock中。
代码逻辑:
public void processFragment(boolean endOfHeaders, ByteBuf fragment, int len,
Http2FrameListener listener) throws Http2Exception {
final HeadersBlockBuilder hdrBlockBuilder = headersBlockBuilder();
hdrBlockBuilder.addFragment(fragment, len, ctx.alloc(), endOfHeaders);
if (endOfHeaders) {
listener.onHeadersRead(ctx, headersStreamId, hdrBlockBuilder.headers(), padding,
headersFlags.endOfStream());
}
}
HeadersBlockBuilder的代码逻辑:
final HeadersBlockBuilder headersBlockBuilder() {
return builder;
}
final void addFragment(ByteBuf fragment, int len, ByteBufAllocator alloc,
boolean endOfHeaders) throws Http2Exception {
if (headerBlock == null) {
if (len > headersDecoder.configuration().maxHeaderListSizeGoAway()) {
headerSizeExceeded();
}
if (endOfHeaders) {
// Optimization - don't bother copying, just use the buffer as-is. Need
// to retain since we release when the header block is built.
headerBlock = fragment.readRetainedSlice(len);
} else {
headerBlock = alloc.buffer(len).writeBytes(fragment, len);
}
return;
}
if (headersDecoder.configuration().maxHeaderListSizeGoAway() - len <
headerBlock.readableBytes()) {
headerSizeExceeded();
}
if (headerBlock.isWritable(len)) {
// The buffer can hold the requested bytes, just write it directly.
headerBlock.writeBytes(fragment, len);
} else {
// Allocate a new buffer that is big enough to hold the entire header block so far.
ByteBuf buf = alloc.buffer(headerBlock.readableBytes() + len);
buf.writeBytes(headerBlock).writeBytes(fragment, len);
headerBlock.release();
headerBlock = buf;
}
}
HeadersContinuation.processFragment()方法中,在HeadersBlockBuilder.addFragment()方法将Header帧的数据读取并加入到headerBlock中后,会判断Header帧是否结束,如果结束了,则会执行hdrBlockBuilder.headers()将headerBlock中的字节解码在io.netty.handler.codec.http2.Http2Headers对象 ,再调用Http2FrameListener.onHeadersRead(ChannelHandlerContext ctx, int streamId, Http2Headers headers, int padding, boolean endOfStream)方法,继续执行后面的Header解析、创建StreamChannel、处理Header消息等逻辑。
Continuation Header帧是Header帧的数据补充,另外Continuation帧还可以对PushPromise帧进行数据补充。Continuation Header帧的处理方式跟Header帧没多大差别。
Continuation Header帧校验项:
Item | Method | Exception |
---|---|---|
streamId是否正常(!=0) | verifyAssociatedWithAStream | 异常:streamId==0 |
payloadLength<=maxFrameSize | verifyPayloadLength | 帧的大小超出了上限 |
headersContinuation 是否不为空 | headersContinuation==null,headersContinuation是处理continuation header帧的一种手段,出现headersContinuation=null时,说明流的帧数据出现错乱 | |
streamId != headersContinuation.getStreamId() | 如果当前帧的streamId与headersContinuation的streamId不同, 流的帧传输或读取出现了乱序 | |
payloadLength >= flags.getPaddingPresenceFieldLength() | 帧体长度太小 |
代码逻辑如下:
private void verifyContinuationFrame() throws Http2Exception {
verifyAssociatedWithAStream();
verifyPayloadLength(payloadLength);
if (headersContinuation == null) {
throw connectionError(PROTOCOL_ERROR, "Received %s frame but not currently processing headers.",
frameType);
}
if (streamId != headersContinuation.getStreamId()) {
throw connectionError(PROTOCOL_ERROR, "Continuation stream ID does not match pending headers. "
+ "Expected %d, but received %d.", headersContinuation.getStreamId(), streamId);
}
if (payloadLength < flags.getPaddingPresenceFieldLength()) {
throw streamError(streamId, FRAME_SIZE_ERROR,
"Frame length %d too small for padding.", payloadLength);
}
}
Continuation Header帧的数据读入更是同Header帧的数据读入没什么区别,都是调用HeadersContinuation.processFragment()方法,通过HeadersBlockBuilder.addFragment()方法将Continuation Header帧的数据读取并加入到headerBlock中,然后判断Header帧是否结束,如果结束了,则会执行hdrBlockBuilder.headers()将headerBlock中的字节解码在io.netty.handler.codec.http2.Http2Headers对象 ,再调用Http2FrameListener.onHeadersRead(ChannelHandlerContext ctx, int streamId, Http2Headers headers, int padding, boolean endOfStream)方法,继续执行后面的Header解析、创建StreamChannel、处理Header消息等逻辑。
代码逻辑如下:
private void readContinuationFrame(ByteBuf payload, int payloadEndIndex, Http2FrameListener listener)
throws Http2Exception {
// Process the initial fragment, invoking the listener's callback if end of headers.
headersContinuation.processFragment(flags.endOfHeaders(), payload,
payloadEndIndex - payload.readerIndex(), listener);
resetHeadersContinuationIfEnd(flags.endOfHeaders());
}
这里其实有个疑问,为什么headersContinuation是做为实例对象放在DefaultHttp2FrameReader中,Http2的多流之间的帧数据是可以中间穿插串起来的,如果相连的两个Header帧和Continuation Header帧分别是不同流的,那不是就出问题了么?
后来我明白过来了,虽然多流之间的帧数据是可以中间穿插串起来,但是仍是使用同一个NioChannel,在写数据时保证Header帧和Continuation Header帧是紧紧相连一起写出去的,那么出现问题的可能性就几乎没有了。
Header解码的动作入口是在headersContinuation.processFragment() -> hdrBlockBuilder.headers() -> DefaultHttp2HeadersDecoder.decodeHeaders() -> HpackDecoder.decode()。所以最终的解码工作是由io.netty.handler.codec.http2.HpackDecoder.decode()方法完成的。
调用栈如下:
HpackDecoder维护了一组状态常量,代表的是当前对Header的读取状态,不同的状态做的事情是不一样的,HpackDecoder通过一个While循环来读取Header,因为一个Frame里面包含若干个Header,根据读取到的数据判断,State会不断变化流转。
State | 说明 |
---|---|
READ_HEADER_REPRESENTATION | 读取header的初试状态 |
READ_INDEXED_HEADER | 读取被索引的完整header,即name和value均被索引 |
READ_INDEXED_HEADER_NAME | 读取被索引的header name |
READ_LITERAL_HEADER_NAME_LENGTH_PREFIX | name未被索引,读取name长度前缀,判断是否使用哈夫曼编码 |
READ_LITERAL_HEADER_NAME_LENGTH | name未被索引,读取name长度 |
READ_LITERAL_HEADER_NAME | name未被索引,读取header name |
READ_LITERAL_HEADER_VALUE_LENGTH_PREFIX | 读取value长度前缀,判断是否使用哈夫曼编码 |
READ_LITERAL_HEADER_VALUE_LENGTH | 读取value长度 |
READ_LITERAL_HEADER_VALUE | value未被索引,读取value |
更多详细的解读见Netty对HPACK头部压缩的支持
当HttpFrameCodec将帧解码之后,会通过Http2FrameListener进行封装处理,然后再调用netty的相应逻辑,与Http2MultiplexHandler的userEventTriggered()方法和channelRead()打通连起来。
Http2MultiplexHandler的两个方法主要逻辑:
Method | Desc |
---|---|
userEventTriggered() | 将Http2FrameCodec.DefaultHttp2FrameStream对象转换成Http2MultiplexHandlerStreamChannel对象,将触发Http2MultiplexHandlerStreamChannel对应pipeline的fireChannelRegistered()方法 |
channelRead() | 将Http2StreamFrame对象(DefaultHttp2HeadersFrame/DefaultHttp2DataFrame)传为参数,执行AbstractHttp2StreamChannel.pipeline的fireChannelRead()方法。完成从NioSocketChannel.ChannelHandler到AbstractHttp2StreamChannel.ChannelHandler的转换 |
HeadersContinuation.processFragment()方法调用的Http2FrameListener.onHeadersRead(ChannelHandlerContext ctx, int streamId, Http2Headers headers, int padding, boolean endOfStream)方法逻辑中,Http2FrameListener是DefaultHttp2ConnectionDecoder$FrameReadListener对象,在该对象的onHeadersRead()方法中,会在Http2Connection先通过streamId查找Http2Stream对象,如果没有,则执行Http2Connection.DefaultEndpoint.createStream()新建一个Http2Stream对象,并将其激活。
调用栈如下:
DefaultHttp2ConnectionDecoder$FrameReadListener.onHeadersRead()方法代码逻辑如下:
public void onHeadersRead(ChannelHandlerContext ctx, int streamId, Http2Headers headers, int streamDependency,
short weight, boolean exclusive, int padding, boolean endOfStream) throws Http2Exception {
Http2Stream stream = connection.stream(streamId);
boolean allowHalfClosedRemote = false;
if (stream == null && !connection.streamMayHaveExisted(streamId)) {
// 创建Http2Stream
stream = connection.remote().createStream(streamId, endOfStream);
// Allow the state to be HALF_CLOSE_REMOTE if we're creating it in that state.
allowHalfClosedRemote = stream.state() == HALF_CLOSED_REMOTE;
}
if (shouldIgnoreHeadersOrDataFrame(ctx, streamId, stream, "HEADERS")) {
return;
}
boolean isInformational = !connection.isServer() &&
HttpStatusClass.valueOf(headers.status()) == INFORMATIONAL;
if ((isInformational || !endOfStream) && stream.isHeadersReceived() || stream.isTrailersReceived()) {
throw streamError(streamId, PROTOCOL_ERROR,
"Stream %d received too many headers EOS: %s state: %s",
streamId, endOfStream, stream.state());
}
switch (stream.state()) {
case RESERVED_REMOTE:
stream.open(endOfStream);
break;
case OPEN:
case HALF_CLOSED_LOCAL:
// Allowed to receive headers in these states.
break;
case HALF_CLOSED_REMOTE:
if (!allowHalfClosedRemote) {
throw streamError(stream.id(), STREAM_CLOSED, "Stream %d in unexpected state: %s",
stream.id(), stream.state());
}
break;
case CLOSED:
throw streamError(stream.id(), STREAM_CLOSED, "Stream %d in unexpected state: %s",
stream.id(), stream.state());
default:
// Connection error.
throw connectionError(PROTOCOL_ERROR, "Stream %d in unexpected state: %s", stream.id(),
stream.state());
}
stream.headersReceived(isInformational);
encoder.flowController().updateDependencyTree(streamId, streamDependency, weight, exclusive);
//触发...
listener.onHeadersRead(ctx, streamId, headers, streamDependency, weight, exclusive, padding, endOfStream);
// If the headers completes this stream, close it.
if (endOfStream) {
lifecycleManager.closeStreamRemote(stream, ctx.newSucceededFuture());
}
}
Http2Connection.DefaultEndpoint.createStream() 方法代码逻辑如下:
public DefaultStream createStream(int streamId, boolean halfClosed) throws Http2Exception {
State state = activeState(streamId, IDLE, isLocal(), halfClosed);
checkNewStreamAllowed(streamId, state);
// Create and initialize the stream.
DefaultStream stream = new DefaultStream(streamId, state);
incrementExpectedStreamId(streamId);
addStream(stream);
stream.activate();
return stream;
}
创建好Http2Stream后,同时调用Http2Stream.activate()方法进行激活。
触发ChannelHandlerContext.fireUserEventTriggered(Http2FrameStreamEvent)方法的调用链:
Method | Desc | |
---|---|---|
- | DefaultHttp2FrameReader$HeadersContinuation.processFragment() | 从IO中读取帧数据到添加到DefaultHttp2FrameReader$HeadersBlockBuilder.headerBlock中,当Header数据结束时,调用 Http2FrameListener.onHeadersRead()方法 |
- | DefaultHttp2ConnectionDecoder$FrameReadListener.onHeadersRead() | 根据streamId从Http2Connection查找Http2Stream对象,如果没有,则执行Http2Connection.DefaultEndpoint.createStream()新建一个Http2Stream对象,并将其激活 (详情见上面的创建Http2Stream并触发ChannelHandlerContext.fireUserEventTriggered(Http2FrameStreamEvent))。再然后调用Http2EmptyDataFrameListener.onHeadersRead()方法向下执行 |
1 | Http2Connection.DefaultEndpoint.createStream() | 创建DefaultHttp2Connection.DefaultStream(Http2Stream)对象 |
2 | DefaultHttp2Connection.DefaultStream.activate() | 流的激活方法 |
3 | DefaultHttp2Connection.ActiveStreams.activate() | 激活流 |
4 | DefaultHttp2Connection.ActiveStreams.addToActiveStreams() | 添加到集合DefaultHttp2Connection.ActiveStreams.streams中,并调用Http2FrameCodec$ConnectionListener.onStreamActive()方法 |
5 | Http2FrameCodec.onStreamActive0() | 将DefaultHttp2Connection.DefaultStream对象包装成Http2FrameCodec.DefaultHttp2FrameStream对象 |
6 | Http2FrameCodec.onHttp2StreamStateChanged() | 调用ChannelHandlerContext.fireUserEventTriggered()方法,触发Http2FrameStreamEvent |
7 | ChannelHandlerContext.fireUserEventTriggered() | |
8 | ChannelHandlerContext.invokeUserEventTriggered() | 触发下一个AbstractChannelHandlerContext.invokeUserEventTriggered()并使用下一个ChannelHandler绑定的EventExecutor执行。 相当于触发ChannelInboundHandler链的userEventTriggered() |
9 | ChannelInboundHandler.userEventTriggered() | |
10 | Http2MultiplexHandler.userEventTriggered() | 将Http2FrameCodec.DefaultHttp2FrameStream对象转换成Http2MultiplexHandlerStreamChannel对象,并将新建的Http2MultiplexHandlerStreamChannel对象注册到EventLoop中 |
11 | SingleThreadEventLoop.register() | |
12 | AbstractHttp2StreamChannel.Http2ChannelUnsafe.register() | 触发AbstractHttp2StreamChannel.pipeline 的fireChannelRegistered()方法,但并没有使用上面传递过来的EventLoop |
13 | DefaultChannelPipeline.fireChannelRegistered() | 触发AbstractHttp2StreamChannel.pipeline 的fireChannelRegistered()方法,并执行第一个DefaultChannelPipeline.AbstractChannelHandlerContext的invokeChannelRegistered()方法 |
14 | ChannelInboundHandler.channelRegistered() | 执行ChannelInboundHandler的channelRegistered(ChannelHandlerContext)方法 |
Http2MultiplexHandler.userEventTriggered()方法会将Stream转换为Http2MultiplexHandlerStreamChannel,实现一个连接的多流Channel隔离,每个流可以对数据字节(ByteBuf)单独处理。
代码逻辑如下:
public void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exception {
if (evt instanceof Http2FrameStreamEvent) {
Http2FrameStreamEvent event = (Http2FrameStreamEvent) evt;
DefaultHttp2FrameStream stream = (DefaultHttp2FrameStream) event.stream();
if (event.type() == Http2FrameStreamEvent.Type.State) {
switch (stream.state()) {
case HALF_CLOSED_LOCAL:
if (stream.id() != Http2CodecUtil.HTTP_UPGRADE_STREAM_ID) {
// Ignore everything which was not caused by an upgrade
break;
}
// fall-through
case HALF_CLOSED_REMOTE:
// fall-through
case OPEN:
if (stream.attachment != null) {
// ignore if child channel was already created.
break;
}
final AbstractHttp2StreamChannel ch;
// We need to handle upgrades special when on the client side.
if (stream.id() == Http2CodecUtil.HTTP_UPGRADE_STREAM_ID && !isServer(ctx)) {
// We must have an upgrade handler or else we can't handle the stream
if (upgradeStreamHandler == null) {
throw connectionError(INTERNAL_ERROR,
"Client is misconfigured for upgrade requests");
}
// 客户端升级流场景,将Stream转换创建Http2MultiplexHandlerStreamChannel
ch = new Http2MultiplexHandlerStreamChannel(stream, upgradeStreamHandler);
ch.closeOutbound();
} else {
//将Stream转换创建Http2MultiplexHandlerStreamChannel
ch = new Http2MultiplexHandlerStreamChannel(stream, inboundStreamHandler);
}
// 触发Http2MultiplexHandlerStreamChannel.pipeline的fireChannelRegistered()方法
ChannelFuture future = ctx.channel().eventLoop().register(ch);
if (future.isDone()) {
registerDone(future);
} else {
future.addListener(CHILD_CHANNEL_REGISTRATION_LISTENER);
}
break;
case CLOSED:
AbstractHttp2StreamChannel channel = (AbstractHttp2StreamChannel) stream.attachment;
if (channel != null) {
channel.streamClosed();
}
break;
default:
// ignore for now
break;
}
}
return;
}
ctx.fireUserEventTriggered(evt);
}
最终Http2Headers对象会转换为DefaultHttp2HeadersFrame对象,经由Http2MultiplexHandler.channelRead()的处理,调用AbstractHttp2StreamChannel.pipeline的fireChannelRead()方法,DefaultHttp2HeadersFrame对象对象做为参数,执行 AbstractHttp2StreamChannel.pipeline中ChannelInboundHandler的channelRead()方法。完成从NioSocketChannel.ChannelHandler到AbstractHttp2StreamChannel.ChannelHandler的转换。
调用栈如下:
Method | Desc | |
---|---|---|
1 | DefaultHttp2FrameReader$HeadersContinuation.processFragment() | 从IO中读取帧数据到添加到DefaultHttp2FrameReader$HeadersBlockBuilder.headerBlock中,当Header数据结束时,调用 Http2FrameListener.onHeadersRead()方法 |
2 | DefaultHttp2ConnectionDecoder$FrameReadListener.onHeadersRead() | 根据streamId从Http2Connection查找Http2Stream对象,如果没有,则执行Http2Connection.DefaultEndpoint.createStream()新建一个Http2Stream对象,并将其激活 (详情见上面的创建Http2Stream并触发ChannelHandlerContext.fireUserEventTriggered(Http2FrameStreamEvent))。再然后调用Http2EmptyDataFrameListener.onHeadersRead()方法向下执行 |
3 | Http2EmptyDataFrameListener.onHeadersRead() | 调用 Http2FrameCodec$FrameListener.onHeadersRead()方法 |
4 | Http2FrameCodec$FrameListener.onHeadersRead() | 将Http2Headers对象转换为DefaultHttp2HeadersFrame对象,并调用onHttp2Frame()方法继续向下执行 |
5 | Http2FrameCodec.onHttp2Frame() | 将DefaultHttp2HeadersFrame对象做为参数触发DefaultChannelHandlerContext.fireChannelRead()方法 |
6 | AbstractChannelHandlerContext.fireChannelRead() | 触发DefaultChannelHandlerContext.invokeChannelRead()方法 |
7 | Http2MultiplexHandler.channelRead() | 将Http2StreamFrame对象(DefaultHttp2HeadersFrame/DefaultHttp2DataFrame)传为参数,执行Http2MultiplexHandler.Http2MultiplexHandlerStreamChannel.pipeline的fireChannelRead()方法。完成从NioSocketChannel.ChannelHandler到AbstractHttp2StreamChannel.ChannelHandler的转换 |
8 | AbstractHttp2StreamChannel.fireChildRead() | |
9 | AbstractHttp2StreamChannel.Http2ChannelUnsafe.doRead0() | 触发AbstractHttp2StreamChannel.pipeline 的fireChannelRegistered()方法 |
10 | DefaultChannelPipeline.fireChannelRead() | 触发AbstractHttp2StreamChannel.pipeline 的fireChannelRead()方法,并执行第一个DefaultChannelPipeline.AbstractChannelHandlerContext的invokeChannelRead()方法 |
AbstractChannelHandlerContext.invokeChannelRead() | netty 逻辑 | |
AbstractChannelHandlerContext.fireChannelRead() | netty 逻辑 | |
AbstractChannelHandlerContext.invokeChannelRead() | netty 逻辑 | |
ChannelInboundHandlerAdapter.channelRead() | netty 逻辑 |
DefaultHttp2ConnectionDecoder$FrameReadListener.onHeadersRead()相关代码逻辑:
public void onHeadersRead(ChannelHandlerContext ctx, int streamId, Http2Headers headers, int streamDependency,
short weight, boolean exclusive, int padding, boolean endOfStream) throws Http2Exception {
Http2Stream stream = connection.stream(streamId);
boolean allowHalfClosedRemote = false;
if (stream == null && !connection.streamMayHaveExisted(streamId)) {、
//创建Http2Stream,并触发Http2MultiplexHandlerStreamChannel.pipeline的fireUserEventTriggered()和fireChannelRegistered()
stream = connection.remote().createStream(streamId, endOfStream);
// Allow the state to be HALF_CLOSE_REMOTE if we're creating it in that state.
allowHalfClosedRemote = stream.state() == HALF_CLOSED_REMOTE;
}
...
stream.headersReceived(isInformational);
encoder.flowController().updateDependencyTree(streamId, streamDependency, weight, exclusive);
//Http2Headers对象转换为DefaultHttp2HeadersFrame对象,并将DefaultHttp2HeadersFrame对象做为参数触发DefaultChannelHandlerContext.fireChannelRead()方法,然后调用Http2MultiplexHandler.channelRead()方法
listener.onHeadersRead(ctx, streamId, headers, streamDependency, weight, exclusive, padding, endOfStream);
// If the headers completes this stream, close it.
if (endOfStream) {
lifecycleManager.closeStreamRemote(stream, ctx.newSucceededFuture());
}
}
Http2FrameCodec$FrameListener.onHeadersRead() 方法逻辑:
public void onHeadersRead(ChannelHandlerContext ctx, int streamId, Http2Headers headers,
int padding, boolean endOfStream) {
// 将Http2Headers对象转换为DefaultHttp2HeadersFrame对象,并调用onHttp2Frame()方法继续向下执行
onHttp2Frame(ctx, new DefaultHttp2HeadersFrame(headers, endOfStream, padding)
.stream(requireStream(streamId)));
}
Http2FrameCodec.onHttp2Frame()方法逻辑:
void onHttp2Frame(ChannelHandlerContext ctx, Http2Frame frame) {
ctx.fireChannelRead(frame);
}
Http2MultiplexHandler.channelRead()方法逻辑:
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
parentReadInProgress = true;
if (msg instanceof Http2StreamFrame) {
if (msg instanceof Http2WindowUpdateFrame) {
// We dont want to propagate update frames to the user
return;
}
Http2StreamFrame streamFrame = (Http2StreamFrame) msg;
DefaultHttp2FrameStream s =
(DefaultHttp2FrameStream) streamFrame.stream();
AbstractHttp2StreamChannel channel = (AbstractHttp2StreamChannel) s.attachment;
if (msg instanceof Http2ResetFrame) {
// Reset frames needs to be propagated via user events as these are not flow-controlled and so
// must not be controlled by suppressing channel.read() on the child channel.
channel.pipeline().fireUserEventTriggered(msg);
// RST frames will also trigger closing of the streams which then will call
// AbstractHttp2StreamChannel.streamClosed()
} else {
// 将Http2StreamFrame对象(DefaultHttp2HeadersFrame/DefaultHttp2DataFrame)传为参数,执行Http2MultiplexHandler.Http2MultiplexHandlerStreamChannel.pipeline的fireChannelRead()方法。
//完成从NioSocketChannel.ChannelHandler到AbstractHttp2StreamChannel.ChannelHandler的转换
channel.fireChildRead(streamFrame);
}
return;
}
if (msg instanceof Http2GoAwayFrame) {
// goaway frames will also trigger closing of the streams which then will call
// AbstractHttp2StreamChannel.streamClosed()
onHttp2GoAwayFrame(ctx, (Http2GoAwayFrame) msg);
}
// Send everything down the pipeline
ctx.fireChannelRead(msg);
}
HttpFrameCodec读取Data帧数据跟的逻辑跟读取Header帧数据的逻辑非常相似,不同的有几点:
跟读取Header帧和Continuation帧一样,读取入口也是在DefaultHttp2FrameReader.readFrame()方法中。主要代码逻辑如下:
public void readFrame(ChannelHandlerContext ctx, ByteBuf input, Http2FrameListener listener)
throws Http2Exception {
if (readError) {
input.skipBytes(input.readableBytes());
return;
}
try {
do {
if (readingHeaders) {
//校验
processHeaderState(input);
if (readingHeaders) {
// Wait until the entire header has arrived.
return;
}
}
// the first pass at payload processing now.
//读取
processPayloadState(ctx, input, listener);
if (!readingHeaders) {
// Wait until the entire payload has arrived.
return;
}
} while (input.isReadable());
} catch (Http2Exception e) {
readError = !Http2Exception.isStreamError(e);
throw e;
} catch (RuntimeException e) {
readError = true;
throw e;
} catch (Throwable cause) {
readError = true;
PlatformDependent.throwException(cause);
}
}
private void processHeaderState(ByteBuf in) throws Http2Exception {
if (in.readableBytes() < FRAME_HEADER_LENGTH) {
// Wait until the entire frame header has been read.
return;
}
// Read the header and prepare the unmarshaller to read the frame.
payloadLength = in.readUnsignedMedium();
if (payloadLength > maxFrameSize) {
throw connectionError(FRAME_SIZE_ERROR, "Frame length: %d exceeds maximum: %d", payloadLength,
maxFrameSize);
}
frameType = in.readByte();
flags = new Http2Flags(in.readUnsignedByte());
streamId = readUnsignedInt(in);
// We have consumed the data, next time we read we will be expecting to read the frame payload.
readingHeaders = false;
switch (frameType) {
case DATA:
verifyDataFrame();
break;
...
default:
// Unknown frame type, could be an extension.
verifyUnknownFrame();
break;
}
}
private void processPayloadState(ChannelHandlerContext ctx, ByteBuf in, Http2FrameListener listener)
throws Http2Exception {
if (in.readableBytes() < payloadLength) {
// Wait until the entire payload has been read.
return;
}
// Only process up to payloadLength bytes.
int payloadEndIndex = in.readerIndex() + payloadLength;
readingHeaders = true;
// Read the payload and fire the frame event to the listener.
switch (frameType) {
case DATA:
//读取Data帧数据
readDataFrame(ctx, in, payloadEndIndex, listener);
break;
...
}
in.readerIndex(payloadEndIndex);
}
//Data帧校验方法
private void verifyDataFrame() throws Http2Exception {
verifyAssociatedWithAStream();
verifyNotProcessingHeaders();
verifyPayloadLength(payloadLength);
if (payloadLength < flags.getPaddingPresenceFieldLength()) {
throw streamError(streamId, FRAME_SIZE_ERROR,
"Frame length %d too small.", payloadLength);
}
}
//Data帧数据读取方法
private void readDataFrame(ChannelHandlerContext ctx, ByteBuf payload, int payloadEndIndex,
Http2FrameListener listener) throws Http2Exception {
int padding = readPadding(payload);
verifyPadding(padding);
// Determine how much data there is to read by removing the trailing
// padding.
int dataLength = lengthWithoutTrailingPadding(payloadEndIndex - payload.readerIndex(), padding);
//将frame payload部分的数据切片之后返回新的一个ByteBuf对象直接向下传递
ByteBuf data = payload.readSlice(dataLength);
listener.onDataRead(ctx, streamId, data, padding, flags.endOfStream());
}
Item | Method | Exception |
---|---|---|
streamId是否正常(!=0) | verifyAssociatedWithAStream | 异常:streamId==0 |
headersContinuation 是否为空 | verifyNotProcessingHeaders | headersContinuation!=null,headersContinuation是处理continuation header帧的一种手段,出现headersContinuation!=null时,说明流的帧数据出现错乱 |
payloadLength<=maxFrameSize | verifyPayloadLength | 帧的大小超出了上限 |
payloadLength >= flags.getPaddingPresenceFieldLength() | 帧体长度太小 |
Method | Desc | |
---|---|---|
1 | DefaultHttp2FrameReader.readDataFrame() | 将frame payload部分的数据切片之后返回新的一个ByteBuf对象直接向下传递 |
2 | DefaultHttp2ConnectionDecoder$FrameReadListener.onDataRead() | |
3 | Http2FrameCodec$FrameListener.onDataRead() | 将ByteBuf类型的frame payload数据包装成DefaultHttp2DataFrame对象,向下传递执行 |
4 | Http2FrameCodec.onHttp2Frame() | 将DefaultHttp2DataFrame对象做为参数触发DefaultChannelHandlerContext.fireChannelRead()方法 |
5 | AbstractChannelHandlerContext.fireChannelRead() | 触发DefaultChannelHandlerContext.invokeChannelRead()方法 |
6 | Http2MultiplexHandler.channelRead() | 将Http2StreamFrame对象(DefaultHttp2HeadersFrame/DefaultHttp2DataFrame)传为参数,执行Http2MultiplexHandler.Http2MultiplexHandlerStreamChannel.pipeline的fireChannelRead()方法。完成从NioSocketChannel.ChannelHandler到AbstractHttp2StreamChannel.ChannelHandler的转换 |
7 | AbstractHttp2StreamChannel.fireChildRead() | |
8 | AbstractHttp2StreamChannel.Http2ChannelUnsafe.doRead0() | 触发AbstractHttp2StreamChannel.pipeline 的fireChannelRegistered()方法 |
9 | DefaultChannelPipeline.fireChannelRead() | 触发AbstractHttp2StreamChannel.pipeline 的fireChannelRead()方法,并执行第一个DefaultChannelPipeline.AbstractChannelHandlerContext的invokeChannelRead()方法 |
AbstractChannelHandlerContext.invokeChannelRead() | netty 逻辑 | |
AbstractChannelHandlerContext.fireChannelRead() | netty 逻辑 | |
AbstractChannelHandlerContext.invokeChannelRead() | netty 逻辑 | |
ChannelInboundHandlerAdapter.channelRead() | netty 逻辑 |
DefaultHttp2FrameReader.readDataFrame():
private void readDataFrame(ChannelHandlerContext ctx, ByteBuf payload, int payloadEndIndex,
Http2FrameListener listener) throws Http2Exception {
int padding = readPadding(payload);
verifyPadding(padding);
// Determine how much data there is to read by removing the trailing
// padding.
int dataLength = lengthWithoutTrailingPadding(payloadEndIndex - payload.readerIndex(), padding);
ByteBuf data = payload.readSlice(dataLength);
listener.onDataRead(ctx, streamId, data, padding, flags.endOfStream());
}
将frame payload部分的数据切片之后返回新的一个ByteBuf对象直接向下传递。
Http2FrameCodec$FrameListener.onDataRead() 方法的代码逻辑:
public int onDataRead(ChannelHandlerContext ctx, int streamId, ByteBuf data, int padding,
boolean endOfStream) {
onHttp2Frame(ctx, new DefaultHttp2DataFrame(data, endOfStream, padding)
.stream(requireStream(streamId)).retain());
// We return the bytes in consumeBytes() once the stream channel consumed the bytes.
return 0;
}
ByteBuf类型的frame payload数据包装成DefaultHttp2DataFrame对象,调用Http2FrameCodec.onHttp2Frame() 将DefaultHttp2DataFrame对象做为参数触发DefaultChannelHandlerContext.fireChannelRead()方法,这里的逻辑跟处理Header帧时一样都 是进入netty的逻辑了,会调用 io.netty.channel.ChannelHandlerContext.fireChannelRead(),直到运行到Http2MultiplexHandler.channelRead()方法中,根据Http2Stream拿到对应的Http2MultiplexHandlerStreamChannel对象,将DefaultHttp2DataFrame对象做为参数,执行Http2MultiplexHandler.Http2MultiplexHandlerStreamChannel.pipeline的fireChannelRead()方法。完成从NioSocketChannel.ChannelHandler到AbstractHttp2StreamChannel.ChannelHandler的转换。
Netty对HPACK头部压缩的支持