Elasticsearch Transport 模块创建及启动分析

Elasticsearch 通信模块的分析从宏观上介绍了ES Transport模块总体功能，于是就很好奇ElasticSearch是怎么把服务启动起来，以接收Client发送过来的Index索引操作、GET获取文档操作 等一系列操作的呢？本文分析：ElasticSearch6.3.2 Netty Http Server 服务的启动过程。ES节点启动，就是启动各个服务，初始化各个服务代码实现 在 org.elasticsearch.node.Node的构造方法中，从创建 org.elasticsearch.common.network.NetworkModule 对象开始，NetworkModule 就是ES中所有关于网络通信相关的功能的创建与注册吧。

final NetworkModule networkModule = new NetworkModule(settings, false, pluginsService.filterPlugins(NetworkPlugin.class),
threadPool, bigArrays, pageCacheRecycler, circuitBreakerService, namedWriteableRegistry, xContentRegistry,
networkService, restController);

在创建NetworkModule对象时，主要是创建2个用于通信的Server

• 一个是Sever是用来接收用户发起的各种操作请求的(external REST clients)，比如GET、INDEX、BULK WRITE、DELETE...这个Server叫HttpServerTransport（具体实现是Netty4HttpServerTransport）。
• 另一个Server用于节点之间的通信(transport layer)，比如：节点之间相互发送ping命令、集群各个节点之间的信息交换、还有，当GET index/_doc/1 这样的用户操作发送到coordinator 节点上，当docid为1的文档不在本机节点上，那么就会使用TcpTransport（具体实现是Netty4TcpTransport）将命令转发到目标节点上

A client can either be retrieved from a org.elasticsearch.node.Node started, or connected remotely to one or more nodes using org.elasticsearch.client.transport.TransportClient. Every node in the cluster can handle HTTP and Transport traffic by default. The transport layer is used exclusively for communication between nodes and the Java TransportClient; the HTTP layer is used only by external REST clients.

Netty4HttpServerTransport 对象创建如下，Netty4TcpTransport 也是类似的逻辑。
org.elasticsearch.common.network.NetworkModule#NetworkModule

                Map> httpTransportFactory = plugin.getHttpTransports(settings,threadPool,bigArrays,circuitBreakerService,namedWriteableRegistry, xContentRegistry, networkService, dispatcher);
                for (Map.Entry> entry : httpTransportFactory.entrySet()) {
                    registerHttpTransport(entry.getKey(), entry.getValue());
                }

Netty4Plugin#getHttpTransports 创建 Netty Http Server：Netty4HttpServerTransport

    @Override
    public Map> getHttpTransports(Settings settings, ThreadPool threadPool, BigArrays bigArrays,CircuitBreakerService，circuitBreakerService,NamedWriteableRegistry namedWriteableRegistry,NamedXContentRegistry xContentRegistry,NetworkService networkService,HttpServerTransport.Dispatcher dispatcher) {
        return Collections.singletonMap(NETTY_HTTP_TRANSPORT_NAME,
            () -> new Netty4HttpServerTransport(settings, networkService, bigArrays, threadPool, xContentRegistry, dispatcher));
    }

将构造好的 Transport 对象封装到 TransportService

//获取构造好的 Netty4Transport
final Transport transport = networkModule.getTransportSupplier().get();
//将 Netty4Transport 封装到 TransportService
final TransportService transportService = newTransportService(settings, transport, threadPool,
networkModule.getTransportInterceptor(), localNodeFactory, settingsModule.getClusterSettings(), taskHeaders);

然后其他需要使用通信功能的模块，只需要封装 TransportService 对象即可。比如执行用户SEARCH操作的搜索模块 TransportSearchAction，它有一个实例属性SearchTransportService，而SearchTransportService就封装了 TransportService，这样TransportSearchAction就能使用TcpTransport进行通信了。如下代码所示：
Node.java 构造方法：

//构造SearchTransportService对象时f需要TransportService，TransportService对象 是一个"公共连接对象"，许多服务都会用到它
final SearchTransportService searchTransportService =  new SearchTransportService(settings,transportService,SearchExecutionStatsCollector.makeWrapper(responseCollectorService));

这里额外提一句：各种Action对象所依赖的Service，应该都是在Node.java的构造方法里面创建的：比如TransportSearchAction依赖的SearchTransportService、ClusterService等都是在节点启动时创建的。

当Netty4HttpServerTransport创建完毕后，就需要绑定端口，启动服务。在org.elasticsearch.node.Node.start方法是ES节点中所有服务的启动入口(当然也包括Netty Http Server了)
org.elasticsearch.node.Node#start方法

        if (NetworkModule.HTTP_ENABLED.get(settings)) {
            injector.getInstance(HttpServerTransport.class).start();
        }

因为Netty4HttpServerTransport继承了AbstractLifecycleComponent，因此它的启动逻辑在org.elasticsearch.common.component.AbstractLifecycleComponent.start中实现，执行doStart()启动Netty Http Server，并绑定端口到9200
Netty4HttpServerTransport#doStart()

protected void doStart() {
        boolean success = false;
        try {
            this.serverOpenChannels = new Netty4OpenChannelsHandler(logger);//---> es for test

            serverBootstrap = new ServerBootstrap();//workerCount=8, elasticsearch[debug_node][http_server_worker]
            //channel一旦分配给EventLoopGroup里面的某个EventLoop线程后,该channel上的所有的事件都将由这个EventLoop线程处理
            serverBootstrap.group(new NioEventLoopGroup(workerCount, daemonThreadFactory(settings,
                HTTP_SERVER_WORKER_THREAD_NAME_PREFIX)));
            serverBootstrap.channel(NioServerSocketChannel.class);//处理连接请求,每个连接建立后创建一个'child channel'处理该连接的所有IO事件
            //为child channel 绑定一个handler, 即用该handler处理该 channel 上的io event
            serverBootstrap.childHandler(configureServerChannelHandler());//--->Netty4HttpRequestHandler
            //指定 child channel 一些配置参数 (父channel是处理连接请求的channel, child channel是已建立的连接的事件处理通道)
            serverBootstrap.childOption(ChannelOption.TCP_NODELAY, SETTING_HTTP_TCP_NO_DELAY.get(settings));
            serverBootstrap.childOption(ChannelOption.SO_KEEPALIVE, SETTING_HTTP_TCP_KEEP_ALIVE.get(settings));
            //---> TCP 发送缓冲区大小
            final ByteSizeValue tcpSendBufferSize = SETTING_HTTP_TCP_SEND_BUFFER_SIZE.get(settings);
            if (tcpSendBufferSize.getBytes() > 0) {
                serverBootstrap.childOption(ChannelOption.SO_SNDBUF, Math.toIntExact(tcpSendBufferSize.getBytes()));
            }
            //---> TCP 接收缓冲区大小
            final ByteSizeValue tcpReceiveBufferSize = SETTING_HTTP_TCP_RECEIVE_BUFFER_SIZE.get(settings);
            if (tcpReceiveBufferSize.getBytes() > 0) {
                serverBootstrap.childOption(ChannelOption.SO_RCVBUF, Math.toIntExact(tcpReceiveBufferSize.getBytes()));
            }

            serverBootstrap.option(ChannelOption.RCVBUF_ALLOCATOR, recvByteBufAllocator);
            serverBootstrap.childOption(ChannelOption.RCVBUF_ALLOCATOR, recvByteBufAllocator);

            final boolean reuseAddress = SETTING_HTTP_TCP_REUSE_ADDRESS.get(settings);
            serverBootstrap.option(ChannelOption.SO_REUSEADDR, reuseAddress);
            serverBootstrap.childOption(ChannelOption.SO_REUSEADDR, reuseAddress);

            this.boundAddress = createBoundHttpAddress();//--->ServerBootStrap绑定端口
            if (logger.isInfoEnabled()) {
                logger.info("{}", boundAddress);
            }
            success = true;
        } finally {
            if (success == false) {
                doStop(); // otherwise we leak threads since we never moved to started
            }
        }
    }

Netty Http Server的worker线程数量是：节点所在的机器上的可用CPU核数：(Runtime.getRuntime().availableProcessors()*2)
其他的一些默认配置如下：
TCP_NODELAY=true, SO_KEEPALIVE=true

ServerBootstrap(ServerBootstrapConfig(group: NioEventLoopGroup, channelFactory: NioServerSocketChannel.class, options: {RCVBUF_ALLOCATOR=io.netty.channel.FixedRecvByteBufAllocator@72ce8a9b, SO_REUSEADDR=true}, childGroup: NioEventLoopGroup, childOptions: {TCP_NODELAY=true, SO_KEEPALIVE=true, RCVBUF_ALLOCATOR=io.netty.channel.FixedRecvByteBufAllocator@72ce8a9b, SO_REUSEADDR=true}, childHandler: org.elasticsearch.http.netty4.Netty4HttpServerTransport$HttpChannelHandler@56ec6ac0))

ES Server 接收用户请求(GET/WRITE/DELETE...)的起始处理点 在哪里？

由于ES Server(实在找不到其他更好的名字来描述了...)是基于 Netty的，那肯定有个ChannelHandler负责处理发生在SocketChannel上的事件。而这个ChannelHandler就是：org.elasticsearch.http.netty4.Netty4HttpRequestHandler
org.elasticsearch.http.netty4.Netty4HttpServerTransport.HttpChannelHandler#initChannel 方法中注册了Netty4HttpRequestHandler，因此用户请求就交给Netty4HttpRequestHandler来处理了。

            ch.pipeline().addLast("handler", requestHandler);//Netty4HttpRequestHandler 业务逻辑处理

那根据Netty框架，毫无疑问 接收用户请求的起始处理点在 org.elasticsearch.http.netty4.Netty4HttpRequestHandler#channelRead0 方法里面了。因此，如果想debug一下INDEX操作、GET操作、DELETE操作的入口，那就从 org.elasticsearch.http.netty4.Netty4HttpRequestHandler#channelRead0方法开始吧。
既然所有的用户操作都是统一的入口，那么又是如何解析这些操作，并最终传递给合适的 TransportXXXAction 来处理的呢？这个以后再说吧。

原文：https://www.cnblogs.com/hapjin/p/11018479.html