netty客户端发送数据时异常被隐藏

问题描述

    最近需要用netty实现一个中间件通信，开始为了先快速把客户端和服务端通信的demo完成，只是采用了字符串的编解码方式(StringEncoder,StringDecoder)。客户端和服务端可以正常互发数据，一切运行正常。
    但是字符串的编解码并不适合业务实体类的传输，为了快速实现实体类传输，所以决定采用jboss-marshalling-serial序列化方式先完成demo，但是在客户端发送数据时，服务端却无法收到数据，客户端控制台也没有任何异常信息。
    先看整个demo实现代码，再查找问题原因。（先提前说明，示例代码是完全正确无逻辑bug的）

pom依赖

<dependencies>
    
	<dependency>
		<groupId>org.springframework.bootgroupId>
		<artifactId>spring-boot-starterartifactId>
	dependency>

	<dependency>
		<groupId>io.nettygroupId>
		<artifactId>netty-allartifactId>
		<version>4.1.56.Finalversion>
	dependency>

	<dependency>
		<groupId>org.jboss.marshallinggroupId>
		<artifactId>jboss-marshalling-serialartifactId>
		<version>2.0.10.Finalversion>
	dependency>
dependencies>

jboss-marshalling-serial序列化工具类

• netty提供的Marshalling编解码器采用消息头和消息体的方式
• JBoss Marshalling是一个Java对象序列化包,对jdk默认的序列化框架进行优化，但又保持跟Serializable接口的兼容,同时增加了一些可调用的参数和附加的特性
• 经过测试发现序列化后的流较protostuff,MessagePack还是比较大的,
• 序列化和反序列化的类必须是同一个类,否则抛出异常: io.netty.handler.codec.DecoderException: java.lang.ClassNotFoundException: com.bruce.netty.rpc.entity.UserInfo

public final class MarshallingCodeFactory {
     
    private static final InternalLogger log = InternalLoggerFactory.getInstance(MarshallingCodeFactory.class);

    /** 创建Jboss marshalling 解码器 */
    public static MyMarshallingDecoder buildMarshallingDecoder() {
     
        //参数serial表示创建的是Java序列化工厂对象,由jboss-marshalling-serial提供
        MarshallerFactory factory = Marshalling.getProvidedMarshallerFactory("serial");
        MarshallingConfiguration configuration = new MarshallingConfiguration();
        configuration.setVersion(5);
        DefaultUnmarshallerProvider provider = new DefaultUnmarshallerProvider(factory, configuration);

        return new MyMarshallingDecoder(provider, 1024);
    }

    /** 创建Jboss marshalling 编码器 */
    public static MarshallingEncoder buildMarshallingEncoder() {
     
        MarshallerFactory factory = Marshalling.getProvidedMarshallerFactory("serial");
        MarshallingConfiguration configuration = new MarshallingConfiguration();
        configuration.setVersion(5);
        DefaultMarshallerProvider provider = new DefaultMarshallerProvider(factory, configuration);
        return new MarshallingEncoder(provider);
    }

    public static class MyMarshallingDecoder extends MarshallingDecoder {
     
        public MyMarshallingDecoder(UnmarshallerProvider provider, int maxObjectSize) {
     
            super(provider, maxObjectSize);
        }
        @Override
        protected Object decode(ChannelHandlerContext ctx, ByteBuf in) throws Exception {
     
            log.info("读取数据长度:{}", in.readableBytes());
            return super.decode(ctx, in);
        }
    }
}

服务端代码实现

服务端业务处理器：(真实场景中不要在io线程执行耗时业务逻辑处理)

@ChannelHandler.Sharable
public class SimpleServerHandler extends ChannelInboundHandlerAdapter {
     
    private static final InternalLogger log = InternalLoggerFactory.getInstance(SimpleServerHandler.class);

    @Override
    public void handlerAdded(ChannelHandlerContext ctx) throws Exception {
     
        log.info("handlerAdded" + this.hashCode());
    }

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
     
        log.info("server channelRead:{}", msg);
        ctx.channel().writeAndFlush("hello netty");
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
     
        if (cause instanceof java.io.IOException) {
     
            log.warn("client close");
        } else {
     
            cause.printStackTrace();
        }
    }
}

服务端启动类

public class NettyServer {
     
    private static final InternalLogger log = InternalLoggerFactory.getInstance(NettyServer.class);

    public static void main(String[] args) throws Exception {
     
        EventLoopGroup acceptGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup =  new NioEventLoopGroup();
        Class<? extends ServerSocketChannel> serverSocketChannelClass = NioServerSocketChannel.class;
        ServerBootstrap bootstrap = new ServerBootstrap();
        bootstrap.group(acceptGroup, workerGroup)
                .channel(serverSocketChannelClass)
                .option(ChannelOption.SO_BACKLOG, 128)
                .option(ChannelOption.SO_REUSEADDR, true)
                .childOption(ChannelOption.SO_KEEPALIVE, false) //默认为false
                .handler(new LoggingHandler())
                .childHandler(new CustomCodecChannelInitializer());

        try {
     
            //sync() 将异步变为同步,绑定到8088端口
            ChannelFuture channelFuture = bootstrap.bind(8088).sync();
            log.info("server 启动成功");
        } catch (Exception e) {
     
            e.printStackTrace();
        }
        
        Thread serverShutdown = new Thread(() -> {
     
            log.info("执行jvm ShutdownHook, server shutdown");
            acceptGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        });
        //注册jvm ShutdownHook,jvm退出之前关闭服务资源
        Runtime.getRuntime().addShutdownHook(serverShutdown);
    }

    static class CustomCodecChannelInitializer extends ChannelInitializer<SocketChannel> {
     
        @Override
        protected void initChannel(SocketChannel ch) throws Exception {
     
            ChannelPipeline pipeline = ch.pipeline();
            pipeline.addLast(MarshallingCodeFactory.buildMarshallingDecoder());
            pipeline.addLast(MarshallingCodeFactory.buildMarshallingEncoder());
            pipeline.addLast(new SimpleServerHandler());
        }
    }
}

客户端代码实现

客户端业务处理器

public class SimpleClientHandler extends ChannelInboundHandlerAdapter {
     
    private static final InternalLogger log = InternalLoggerFactory.getInstance(SimpleClientHandler.class);

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
     
        log.info("client receive:{}", msg);
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
     
        cause.printStackTrace();
    }
}

客户端启动类

public class NettyClient {
     
    private static final InternalLogger log = InternalLoggerFactory.getInstance(NettyClient.class);

    public static void main(String[] args) {
     
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        Class<? extends SocketChannel> socketChannelClass = NioSocketChannel.class;

        Bootstrap bootstrap = new Bootstrap();
        bootstrap.group(workerGroup)
                .channel(socketChannelClass)
                .option(ChannelOption.TCP_NODELAY, true)
                .option(ChannelOption.SO_KEEPALIVE, false)
                .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 30000)
                .handler(new CustomCodecChannelInitializer());

        Channel clientChannel;
        try {
     
            ChannelFuture channelFuture = bootstrap.connect("127.0.0.1", 8088);
            //同步等待连接建立成功, 这里示例代码, 可以认为是一定会连接成功
            boolean b = channelFuture.awaitUninterruptibly(10, TimeUnit.SECONDS);
            clientChannel = channelFuture.channel();

            for (int i = 0; i < 10; i++) {
     
                Thread.sleep(1000);
                UserInfo userInfo = new UserInfo("bruce", 18);
                log.info("send user info");
                //连接成功后发送数据
                send(clientChannel, userInfo);
            }
            //实际上这个地方会永远阻塞等待
            clientChannel.closeFuture().sync();
        } catch (InterruptedException e) {
     
            e.printStackTrace();
        } finally {
     
            workerGroup.shutdownGracefully();
        }
    }

    static void send(Channel channel, UserInfo data) {
     
        //连接成功后发送数据
        ChannelFuture channelFuture1 = channel.writeAndFlush(data);
    }

    static class CustomCodecChannelInitializer extends ChannelInitializer<SocketChannel> {
     
        @Override
        protected void initChannel(SocketChannel ch) throws Exception {
     
            ChannelPipeline pipeline = ch.pipeline();
            pipeline.addLast(MarshallingCodeFactory.buildMarshallingDecoder());
            pipeline.addLast(MarshallingCodeFactory.buildMarshallingEncoder());
            pipeline.addLast(new SimpleClientHandler());
        }
    }
}

实体类UserInfo，

public class UserInfo {
     
    private String username;
    private int age;

    public UserInfo() {
     
    }

    public UserInfo(String username, int age) {
     
        this.username = username;
        this.age = age;
    }
    //getter / setter 省略
}

先启动服务端，再启动客户端可以在idea控制台发现：

服务端和客户端建立了连接，客户端在发送数据，$但是服务端却没有收到，并且控制台没有任何异常信息$


既然没有异常，只能先在客户端断点，确认客户端是否正常，根据经验直接查看MarshallingEncoder的编码方法MarshallingEncoder#encode，debug执行先确认UserInfo对象有没有被正确序列化。

在执行到marshaller.writeObject(msg)时出现了异常。

继续跟进断点会进入catch中，显示java.io.NotSerializableException，（脑中出现一句话：我大意了，没有…）已经可以知道UserInfo类没有继承序列化接口java.io.Serializable而抛出异常。UserInfo只需要继承java.io.Serializable就可以正常向客户端发送数据。

但是为什么控制台没有抛出异常呢 ！？

继续跟进断点NotSerializableException被包装在io.netty.handler.codec.EncoderException中抛出，序列化的buf也在finally中被释放。而EncoderException会被AbstractChannelHandlerContext#invokeWrite0方法的catch语句中被处理。

private void invokeWrite0(Object msg, ChannelPromise promise) {
     
    try {
     
        ((ChannelOutboundHandler) handler()).write(this, msg, promise);
    } catch (Throwable t) {
     
        notifyOutboundHandlerException(t, promise);
    }
}
 private static void notifyOutboundHandlerException(Throwable cause, ChannelPromise promise) {
     
    // Only log if the given promise is not of type VoidChannelPromise as tryFailure(...) is expected to return
    // false.
    PromiseNotificationUtil.tryFailure(promise, cause, promise instanceof VoidChannelPromise ? null : logger);
}

最终会执行到io.netty.util.concurrent.DefaultPromise#setValue0，主要目的就是为了记录这个异常信息，然后检查是否有GenericFutureListener监听这次发送请求的结果。如果有Listener则在nio线程中回调监听器方法。

private boolean setValue0(Object objResult) {
     
    if (RESULT_UPDATER.compareAndSet(this, null, objResult) ||
        RESULT_UPDATER.compareAndSet(this, UNCANCELLABLE, objResult)) {
     
        if (checkNotifyWaiters()) {
     
            notifyListeners();
        }
        return true;
    }
    return false;
}

private synchronized boolean checkNotifyWaiters() {
     
    if (waiters > 0) {
     
        notifyAll();
    }
    return listeners != null;
}

然而笔者的示例中并没有设置GenericFutureListener，checkNotifyWaiters方法返回的是false，不会执行notifyListeners();方法，所以整个异常被吞没。而Promise#tryFailure方法最终返回true。
再看方法io.netty.util.internal.PromiseNotificationUtil#tryFailure，虽然也是会处理Throwable，但是只在Promise#tryFailure返回false并且logger不为null时执行。所以这里也不会打印出日志。

public static void tryFailure(Promise<?> p, Throwable cause, InternalLogger logger) {
     
	if (!p.tryFailure(cause) && logger != null) {
     
		Throwable err = p.cause();
		if (err == null) {
     
			logger.warn("Failed to mark a promise as failure because it has succeeded already: {}", p, cause);
		} else if (logger.isWarnEnabled()) {
     
			logger.warn(
					"Failed to mark a promise as failure because it has failed already: {}, unnotified cause: {}",
					p, ThrowableUtil.stackTraceToString(err), cause);
		}
	}
}

如何打印出这些异常信息呢？

方案1 (异步处理)
在数据发送过后，给ChannelFuture添加监听器，用于监听此次发送的结果，当出现异常时，对异常进行处理。

static void send(Channel channel, UserInfo data) {
     
    //连接成功后发送数据
    ChannelFuture channelFuture1 = channel.writeAndFlush(data);
    channelFuture1.addListener(new ChannelFutureListener() {
     
        @Override
        public void operationComplete(ChannelFuture future) throws Exception {
     
            Throwable cause = future.cause();
            if (cause != null) {
     
                cause.printStackTrace();
            }
        }
    });
}

方案2 （不推荐，根据业务决定）
在数据发送过后，同步等待发送结果，判断是否存在异常。

static void send(Channel channel, UserInfo data) {
     
    //连接成功后发送数据
    ChannelFuture channelFuture1 = channel.writeAndFlush(data);
    while (!channelFuture1.isDone()) {
     
        try {
     
            //超时时间示例值，根据业务决定
            boolean notTimeout = channelFuture1.await(50);
        } catch (Exception e) {
     
            log.warn(e.getMessage());
        }
    }
    Throwable cause = channelFuture1.cause();
    if (cause != null) {
     
        cause.printStackTrace();
    }
}

没有监听ChannelFuture，异常就被隐藏是否合理呢？

这个问题见仁见智，对笔者有点代码洁癖来说，这里至少是可有优化一下的，不至于让开发者耗费时间去查找丢失的异常信息。优化逻辑也简单，在io.netty.util.concurrent.DefaultPromise#setFailure0中，如果既没有listeners也没有await等待时，则打印异常信息。
修DefaultPromise改代码如下:

private boolean setFailure0(Throwable cause) {
     
    if (listeners == null && waiters == 0) {
     
        logger.error("cause:", cause);
    }
    return setValue0(new CauseHolder(checkNotNull(cause, "cause")));
}

请看pr: https://github.com/netty/netty/pull/10917

总结

这里只是通过编码时没有注意到的细节（实体类没有实现序列化接口），来分析为什么异常被吞及处理方案，可以通过异常栈快速定位问题，但如果想要没有异常，则只能根据异常做相应的修改了。同时可以让我们更加了解netty的实现细节。
最后还是建议通过channel发送数据后，对返回的ChannelFuture做是否存在异常判断以及处理，防止出现类似的情况。