Tomcat NioEndpoint.Poller和Netty NioEventLoop OP_READ实现对比

一、Tomcat NioEndpoint.Poller

1. NioEndpoint.Poller#run while(true) 处理监听到的感兴趣事件（SelectionKey#interestOps(int)）
2. NioEndpoint.Poller#processKey 迭代处理发生的事件

• 2.1 unreg(sk, attachment, sk.readyOps()); 取消注册 socketchannel 感兴趣事件，这步很关键
• 2.2 processSocket(attachment, SocketEvent, true) 处理 socket 中的数据

protected void processKey(SelectionKey sk, NioSocketWrapper attachment) {
    try {
        if ( close ) {
            cancelledKey(sk);
        } else if ( sk.isValid() && attachment != null ) {
            if (sk.isReadable() || sk.isWritable() ) {
                if ( attachment.getSendfileData() != null ) {
                    processSendfile(sk,attachment, false);
                } else {
                	// 关注这一步
                    unreg(sk, attachment, sk.readyOps());
                    boolean closeSocket = false;
                    // Read goes before write
                    if (sk.isReadable()) {
                        if (!processSocket(attachment, SocketEvent.OPEN_READ, true)) {
                            closeSocket = true;
                        }
                    }
                    if (!closeSocket && sk.isWritable()) {
                        if (!processSocket(attachment, SocketEvent.OPEN_WRITE, true)) {
                            closeSocket = true;
                        }
                    }
                    if (closeSocket) {
                        cancelledKey(sk);
                    }
                }
            }
        } else {
            //invalid key
            cancelledKey(sk);
        }
    } catch ( CancelledKeyException ckx ) {
        cancelledKey(sk);
    } catch (Throwable t) {
        ExceptionUtils.handleThrowable(t);
        log.error("",t);
    }
}

3. AbstractEndpoint#processSocket

• 3.1 executor.execute(sc) 获取线程池分发processor

public boolean processSocket(SocketWrapperBase<S> socketWrapper,
        SocketEvent event, boolean dispatch) {
    try {
        if (socketWrapper == null) {
            return false;
        }
        SocketProcessorBase<S> sc = processorCache.pop();
        if (sc == null) {
            sc = createSocketProcessor(socketWrapper, event);
        } else {
            sc.reset(socketWrapper, event);
        }
        // 获取线程池分发processor
        Executor executor = getExecutor();
        if (dispatch && executor != null) {
            executor.execute(sc);
        } else {
            sc.run();
        }
    } catch (RejectedExecutionException ree) {
        getLog().warn(sm.getString("endpoint.executor.fail", socketWrapper) , ree);
        return false;
    } catch (Throwable t) {
        ExceptionUtils.handleThrowable(t);
        // This means we got an OOM or similar creating a thread, or that
        // the pool and its queue are full
        getLog().error(sm.getString("endpoint.process.fail"), t);
        return false;
    }
    return true;
}

二、Netty4.x NioEventLoop

1. NioEventLoop#run for ( ; ; ) 处理监听到的感兴趣事件
2. NioEventLoop#processSelectedKeys
3. NioEventLoop#processSelectedKeysOptimized 迭代处理发生的事件
4. NioEventLoop#processSelectedKey
   该方法下面判断处理读事件

if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
   unsafe.read();
}

5. AbstractNioByteChannel.NioByteUnsafe#read

• 5.1 doReadBytes(byteBuf) 读取 socketchannel 中的数据
• 5.2 allocHandle.lastBytesRead() <= 0 如果读到<=0就 break（-1 套接字关闭，0 非阻塞没读到数据），等待下次事件触发

@Override
public final void read() {
    // 省略代码
    ByteBuf byteBuf = null;
    boolean close = false;
    try {
        do {
            byteBuf = allocHandle.allocate(allocator);
            allocHandle.lastBytesRead(doReadBytes(byteBuf));
            if (allocHandle.lastBytesRead() <= 0) {
                // nothing was read. release the buffer.
                byteBuf.release();
                byteBuf = null;
                close = allocHandle.lastBytesRead() < 0;
                if (close) {
                    // There is nothing left to read as we received an EOF.
                    readPending = false;
                }
                break;
            }

            allocHandle.incMessagesRead(1);
            readPending = false;
            pipeline.fireChannelRead(byteBuf);
            byteBuf = null;
        } while (allocHandle.continueReading());

        allocHandle.readComplete();
        pipeline.fireChannelReadComplete();

        if (close) {
            closeOnRead(pipeline);
        }
    } catch (Throwable t) {
        handleReadException(pipeline, byteBuf, t, close, allocHandle);
    } finally {
       	// 省略代码
    }
}

6. ByteToMessageDecoder#channelRead 同步或者异步（ChannelPipeline#addLast(EventExecutorGroup, ChannelHandler)传了EventExecutorGroup，但同一套接字任务提交顺序执行）保存组合上次读到的数据，执行ChannelPipeline责任链

三、总结

相同点

1. 毋庸置疑，都是有 jdk NIO api 实现；
2. 监听感兴趣事件时都使用单个线程（也就是 Poller 和 NioEventLoop）死循环监听。

不同

1. tomcat 处理事件时，先 unregister 感兴趣事件，再读取数据，如果配置了 Executor 直接分发到另一个线程池中。读取完成后再次注册。
2. Netty 处理事件时使用 NioEventLoop 一个线程读取，假设没有读取完直接 break，通过 ByteToMessageDecoder#cumulation 保存上次读取的部分数据。