Netty进阶-协议

2. 协议设计与解析

2.1 为什么需要协议？

TCP/IP 中消息传输基于流的方式，没有边界。

协议的目的就是划定消息的边界，制定通信双方要共同遵守的通信规则

例如：在网络上传输

下雨天留客天留我不留

是中文一句著名的无标点符号句子，在没有标点符号情况下，这句话有数种拆解方式，而意思却是完全不同，所以常被用作讲述标点符号的重要性

一种解读

下雨天留客，天留，我不留

另一种解读

下雨天，留客天，留我不？留

如何设计协议呢？其实就是给网络传输的信息加上“标点符号”。但通过分隔符来断句不是很好，因为分隔符本身如果用于传输，那么必须加以区分。因此，下面一种协议较为常用

定长字节表示内容长度 + 实际内容

例如，假设一个中文字符长度为 3，按照上述协议的规则，发送信息方式如下，就不会被接收方弄错意思了

0f下雨天留客06天留09我不留

小故事

很久很久以前，一位私塾先生到一家任教。双方签订了一纸协议：“无鸡鸭亦可无鱼肉亦可白菜豆腐不可少不得束修金”。此后，私塾先生虽然认真教课，但主人家则总是给私塾先生以白菜豆腐为菜，丝毫未见鸡鸭鱼肉的款待。私塾先生先是很不解，可是后来也就想通了：主人把鸡鸭鱼肉的钱都会换为束修金的，也罢。至此双方相安无事。

年关将至，一个学年段亦告结束。私塾先生临行时，也不见主人家为他交付束修金，遂与主家理论。然主家亦振振有词：“有协议为证——无鸡鸭亦可，无鱼肉亦可，白菜豆腐不可少，不得束修金。这白纸黑字明摆着的，你有什么要说的呢？”

私塾先生据理力争：“协议是这样的——无鸡，鸭亦可；无鱼，肉亦可；白菜豆腐不可，少不得束修金。”

双方唇枪舌战，你来我往，真个是不亦乐乎！

这里的束修金，也作“束脩”，应当是泛指教师应当得到的报酬

2.2 redis 协议举例

@Slf4j
public class TestRedis {
    /* redis协议格式
    set name zhangsan
    *3
    $3
    set
    $4
    name
    $8
    zhangsan
     */
    public static void main(String[] args) {
        final byte[] LINE = {13, 10}; //回车换行 \r\n
        NioEventLoopGroup worker = new NioEventLoopGroup();
        try {
            Bootstrap bootstrap = new Bootstrap();
            bootstrap.channel(NioSocketChannel.class);
            bootstrap.group(worker);
            bootstrap.handler(new ChannelInitializer<SocketChannel>() {
                @Override
                protected void initChannel(SocketChannel ch) {
                    ch.pipeline().addLast(new LoggingHandler());
                    ch.pipeline().addLast(new ChannelInboundHandlerAdapter() {
                        @Override
                        public void channelActive(ChannelHandlerContext ctx) {
                            ByteBuf buf = ctx.alloc().buffer();
                            buf.writeBytes("*3".getBytes());
                            buf.writeBytes(LINE);
                            buf.writeBytes("$3".getBytes());
                            buf.writeBytes(LINE);
                            buf.writeBytes("set".getBytes());
                            buf.writeBytes(LINE);
                            buf.writeBytes("$4".getBytes());
                            buf.writeBytes(LINE);
                            buf.writeBytes("name".getBytes());
                            buf.writeBytes(LINE);
                            buf.writeBytes("$8".getBytes());
                            buf.writeBytes(LINE);
                            buf.writeBytes("zhangsan".getBytes());
                            buf.writeBytes(LINE);
                            ctx.writeAndFlush(buf);
                        }

                        @Override
                        public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
                            ByteBuf buf = (ByteBuf) msg;
                            System.out.println(buf.toString(Charset.defaultCharset()));
                        }
                    });
                }
            });
            ChannelFuture channelFuture = bootstrap.connect("localhost", 6379).sync();
            channelFuture.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            log.error("client error", e);
        } finally {
            worker.shutdownGracefully();
        }
    }
}

2.3 http 协议举例

@Slf4j
public class TestHttp {
    public static void main(String[] args) {
        NioEventLoopGroup boss = new NioEventLoopGroup();
        NioEventLoopGroup worker = new NioEventLoopGroup();
        try {
            ServerBootstrap serverBootstrap = new ServerBootstrap();
            serverBootstrap.channel(NioServerSocketChannel.class);
            serverBootstrap.group(boss, worker);
            serverBootstrap.childHandler(new ChannelInitializer<SocketChannel>() {
                @Override
                protected void initChannel(SocketChannel ch) throws Exception {
                    ch.pipeline().addLast(new LoggingHandler(LogLevel.DEBUG));
                    //http编码器，解码器
                    ch.pipeline().addLast(new HttpServerCodec());
                    ch.pipeline().addLast(new SimpleChannelInboundHandler<HttpRequest>() {
                        @Override
                        protected void channelRead0(ChannelHandlerContext ctx, HttpRequest msg) throws Exception {
                            // 获取请求
                            log.debug(msg.uri());
                            //返回响应
                            DefaultFullHttpResponse response = new DefaultFullHttpResponse(msg.protocolVersion(), HttpResponseStatus.OK);
                            byte[] bytes = "
Hello, world!
".getBytes();
                            //不加这个，浏览器一直转圈
                            response.headers().set(CONTENT_LENGTH, bytes.length);
                            response.content().writeBytes(bytes);
                            ctx.channel().writeAndFlush(response);
                        }
                    });
                }
            });
            ChannelFuture channelFuture = serverBootstrap.bind(8080).sync();
            channelFuture.channel().closeFuture().sync();
        } catch (Exception e) {
            log.error("server error", e);
        } finally {
            boss.shutdownGracefully();
            worker.shutdownGracefully();
        }
    }
}

浏览器输入：http://127.0.0.1:8080/index，结果

17:26:54 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x986cb0df, L:/127.0.0.1:8080 - R:/127.0.0.1:55551] REGISTERED
17:26:54 [DEBUG] [nioEventLoopGroup-3-2] i.n.h.l.LoggingHandler - [id: 0x0bc48b95, L:/127.0.0.1:8080 - R:/127.0.0.1:55552] REGISTERED
17:26:54 [DEBUG] [nioEventLoopGroup-3-2] i.n.h.l.LoggingHandler - [id: 0x0bc48b95, L:/127.0.0.1:8080 - R:/127.0.0.1:55552] ACTIVE
17:26:54 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x986cb0df, L:/127.0.0.1:8080 - R:/127.0.0.1:55551] ACTIVE
17:26:54 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x986cb0df, L:/127.0.0.1:8080 - R:/127.0.0.1:55551] READ: 662B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 47 45 54 20 2f 69 6e 64 65 78 20 48 54 54 50 2f |GET /index HTTP/|
|00000010| 31 2e 31 0d 0a 48 6f 73 74 3a 20 31 32 37 2e 30 |1.1..Host: 127.0|
|00000020| 2e 30 2e 31 3a 38 30 38 30 0d 0a 43 6f 6e 6e 65 |.0.1:8080..Conne|
|00000030| 63 74 69 6f 6e 3a 20 6b 65 65 70 2d 61 6c 69 76 |ction: keep-aliv|
|00000040| 65 0d 0a 73 65 63 2d 63 68 2d 75 61 3a 20 22 47 |e..sec-ch-ua: "G|
|00000050| 6f 6f 67 6c 65 20 43 68 72 6f 6d 65 22 3b 76 3d |oogle Chrome";v=|
|00000060| 22 31 31 37 22 2c 20 22 4e 6f 74 3b 41 3d 42 72 |"117", "Not;A=Br|
|00000070| 61 6e 64 22 3b 76 3d 22 38 22 2c 20 22 43 68 72 |and";v="8", "Chr|
|00000080| 6f 6d 69 75 6d 22 3b 76 3d 22 31 31 37 22 0d 0a |omium";v="117"..|
|00000090| 73 65 63 2d 63 68 2d 75 61 2d 6d 6f 62 69 6c 65 |sec-ch-ua-mobile|
|000000a0| 3a 20 3f 30 0d 0a 73 65 63 2d 63 68 2d 75 61 2d |: ?0..sec-ch-ua-|
|000000b0| 70 6c 61 74 66 6f 72 6d 3a 20 22 57 69 6e 64 6f |platform: "Windo|
|000000c0| 77 73 22 0d 0a 55 70 67 72 61 64 65 2d 49 6e 73 |ws"..Upgrade-Ins|
|000000d0| 65 63 75 72 65 2d 52 65 71 75 65 73 74 73 3a 20 |ecure-Requests: |
|000000e0| 31 0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a 20 4d |1..User-Agent: M|
|000000f0| 6f 7a 69 6c 6c 61 2f 35 2e 30 20 28 57 69 6e 64 |ozilla/5.0 (Wind|
|00000100| 6f 77 73 20 4e 54 20 31 30 2e 30 3b 20 57 69 6e |ows NT 10.0; Win|
|00000110| 36 34 3b 20 78 36 34 29 20 41 70 70 6c 65 57 65 |64; x64) AppleWe|
|00000120| 62 4b 69 74 2f 35 33 37 2e 33 36 20 28 4b 48 54 |bKit/537.36 (KHT|
|00000130| 4d 4c 2c 20 6c 69 6b 65 20 47 65 63 6b 6f 29 20 |ML, like Gecko) |
|00000140| 43 68 72 6f 6d 65 2f 31 31 37 2e 30 2e 30 2e 30 |Chrome/117.0.0.0|
|00000150| 20 53 61 66 61 72 69 2f 35 33 37 2e 33 36 0d 0a | Safari/537.36..|
|00000160| 41 63 63 65 70 74 3a 20 74 65 78 74 2f 68 74 6d |Accept: text/htm|
|00000170| 6c 2c 61 70 70 6c 69 63 61 74 69 6f 6e 2f 78 68 |l,application/xh|
|00000180| 74 6d 6c 2b 78 6d 6c 2c 61 70 70 6c 69 63 61 74 |tml+xml,applicat|
|00000190| 69 6f 6e 2f 78 6d 6c 3b 71 3d 30 2e 39 2c 69 6d |ion/xml;q=0.9,im|
|000001a0| 61 67 65 2f 61 76 69 66 2c 69 6d 61 67 65 2f 77 |age/avif,image/w|
|000001b0| 65 62 70 2c 69 6d 61 67 65 2f 61 70 6e 67 2c 2a |ebp,image/apng,*|
|000001c0| 2f 2a 3b 71 3d 30 2e 38 2c 61 70 70 6c 69 63 61 |/*;q=0.8,applica|
|000001d0| 74 69 6f 6e 2f 73 69 67 6e 65 64 2d 65 78 63 68 |tion/signed-exch|
|000001e0| 61 6e 67 65 3b 76 3d 62 33 3b 71 3d 30 2e 37 0d |ange;v=b3;q=0.7.|
|000001f0| 0a 53 65 63 2d 46 65 74 63 68 2d 53 69 74 65 3a |.Sec-Fetch-Site:|
|00000200| 20 6e 6f 6e 65 0d 0a 53 65 63 2d 46 65 74 63 68 | none..Sec-Fetch|
|00000210| 2d 4d 6f 64 65 3a 20 6e 61 76 69 67 61 74 65 0d |-Mode: navigate.|
|00000220| 0a 53 65 63 2d 46 65 74 63 68 2d 55 73 65 72 3a |.Sec-Fetch-User:|
|00000230| 20 3f 31 0d 0a 53 65 63 2d 46 65 74 63 68 2d 44 | ?1..Sec-Fetch-D|
|00000240| 65 73 74 3a 20 64 6f 63 75 6d 65 6e 74 0d 0a 41 |est: document..A|
|00000250| 63 63 65 70 74 2d 45 6e 63 6f 64 69 6e 67 3a 20 |ccept-Encoding: |
|00000260| 67 7a 69 70 2c 20 64 65 66 6c 61 74 65 2c 20 62 |gzip, deflate, b|
|00000270| 72 0d 0a 41 63 63 65 70 74 2d 4c 61 6e 67 75 61 |r..Accept-Langua|
|00000280| 67 65 3a 20 7a 68 2d 43 4e 2c 7a 68 3b 71 3d 30 |ge: zh-CN,zh;q=0|
|00000290| 2e 39 0d 0a 0d 0a                               |.9....          |
+--------+-------------------------------------------------+----------------+
17:26:54 [DEBUG] [nioEventLoopGroup-3-1] c.i.n.j.c.TestHttp - /index
17:26:54 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x986cb0df, L:/127.0.0.1:8080 - R:/127.0.0.1:55551] WRITE: 61B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d |HTTP/1.1 200 OK.|
|00000010| 0a 43 6f 6e 74 65 6e 74 2d 6c 65 6e 67 74 68 3a |.Content-length:|
|00000020| 20 32 32 0d 0a 0d 0a 3c 68 31 3e 48 65 6c 6c 6f | 22....
Hello|
|00000030| 2c 20 77 6f 72 6c 64 21 3c 2f 68 31 3e          |, world!
   |
+--------+-------------------------------------------------+----------------+

2.4 自定义协议要素

• 魔数，用来在第一时间判定是否是无效数据包
  • 魔数（Magic Number）通常是一个固定长度的字节序列，用于识别协议或数据结构的开头。魔数的存在可以帮助接收方快速验证数据的有效性，并确保它们理解并按照正确的协议进行处理。
• 版本号，可以支持协议的升级
• 序列化算法，消息正文到底采用哪种序列化反序列化方式，可以由此扩展，例如：json、protobuf、hessian、jdk
• 指令类型，是登录、注册、单聊、群聊… 跟业务相关
• 请求序号，为了双工通信，提供异步能力
• 正文长度
• 消息正文

编解码器

根据上面的要素，设计一个登录请求消息和登录响应消息，并使用 Netty 完成收发

@Slf4j
public class MessageCodec extends ByteToMessageCodec<Message> {
    // 编码 出站调用
    @Override
    protected void encode(ChannelHandlerContext ctx, Message msg, ByteBuf out) throws Exception {
        // 4 字节的魔数
        out.writeBytes(new byte[]{1, 2, 3, 4});
        // 1 字节的版本
        out.writeByte(1);
        // 1 字节的序列化方式 jdk 0 , json 1
        out.writeByte(0);
        // 1 字节的指令类型
        out.writeByte(msg.getMessageType());
        // 4 个字节的请求序号
        out.writeInt(msg.getSequenceId());
        // 1 字节，无意义，对齐填充，为了将协议头补齐至2的n次幂
        out.writeByte(0xff);
        // 将message序列化为字节数组
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        ObjectOutputStream oos = new ObjectOutputStream(bos);
        oos.writeObject(msg);
        byte[] bytes = bos.toByteArray();
        // 4 个字节的正文长度
        out.writeInt(bytes.length);
        // 消息正文
        out.writeBytes(bytes);
    }

    // 解码 进站调用
    @Override
    protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
        int magicNum = in.readInt();
        byte version = in.readByte();
        byte serializerType = in.readByte();
        byte messageType = in.readByte();
        int sequenceId = in.readInt();
        in.readByte();
        int length = in.readInt();
        byte[] bytes = new byte[length];
        in.readBytes(bytes, 0, length);
        ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(bytes));
        Message message = (Message) ois.readObject();
        log.debug("{}, {}, {}, {}, {}, {}", magicNum, version, serializerType, messageType, sequenceId, length);
        log.debug("{}", message);
        out.add(message);
    }
}

测试

public class TestMessageCodec {
    public static void main(String[] args) throws Exception {
        EmbeddedChannel channel = new EmbeddedChannel(
                new LoggingHandler(LogLevel.DEBUG),
                new LengthFieldBasedFrameDecoder(
                        1024, 12, 4, 0, 0
                ),//解决黏包半包
                new MessageCodec()
        );

        //出站 encode
        LoginRequestMessage message = new LoginRequestMessage("zhangsan", "123", "张三");
        channel.writeOutbound(message);

        //入站 decode
        ByteBuf buf = ByteBufAllocator.DEFAULT.buffer();
        new MessageCodec().encode(null, message, buf);
        //channel.writeInbound(buf);

        //模拟半包问题
        ByteBuf s1 = buf.slice(0, 100);
        ByteBuf s2 = buf.slice(100, buf.readableBytes() - 100);
        s1.retain();
        channel.writeInbound(s1); // release 1
        channel.writeInbound(s2);
    }
}

11:02:40 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ: 100B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 01 02 03 04 01 00 00 00 00 00 00 ff 00 00 00 fd |................|
|00000010| ac ed 00 05 73 72 00 36 63 6e 2e 69 74 63 61 73 |....sr.6cn.itcas|
|00000020| 74 2e 6e 65 74 74 79 2e 6a 69 6e 6a 69 65 2e 63 |t.netty.jinjie.c|
|00000030| 32 2e 70 72 6f 74 6f 63 6f 6c 2e 4c 6f 67 69 6e |2.protocol.Login|
|00000040| 52 65 71 75 65 73 74 4d 65 73 73 61 67 65 43 a4 |RequestMessageC.|
|00000050| f9 f0 14 8a f7 ce 02 00 03 4c 00 04 6e 61 6d 65 |.........L..name|
|00000060| 74 00 12 4c                                     |t..L            |
+--------+-------------------------------------------------+----------------+
11:02:40 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ COMPLETE
11:02:40 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ: 169B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 6a 61 76 61 2f 6c 61 6e 67 2f 53 74 72 69 6e 67 |java/lang/String|
|00000010| 3b 4c 00 08 70 61 73 73 77 6f 72 64 71 00 7e 00 |;L..passwordq.~.|
|00000020| 01 4c 00 08 75 73 65 72 6e 61 6d 65 71 00 7e 00 |.L..usernameq.~.|
|00000030| 01 78 72 00 2a 63 6e 2e 69 74 63 61 73 74 2e 6e |.xr.*cn.itcast.n|
|00000040| 65 74 74 79 2e 6a 69 6e 6a 69 65 2e 63 32 2e 70 |etty.jinjie.c2.p|
|00000050| 72 6f 74 6f 63 6f 6c 2e 4d 65 73 73 61 67 65 78 |rotocol.Messagex|
|00000060| d7 24 ab 71 27 ac 87 02 00 02 49 00 0b 6d 65 73 |.$.q'.....I..mes|
|00000070| 73 61 67 65 54 79 70 65 49 00 0a 73 65 71 75 65 |sageTypeI..seque|
|00000080| 6e 63 65 49 64 78 70 00 00 00 00 00 00 00 00 74 |nceIdxp........t|
|00000090| 00 06 e5 bc a0 e4 b8 89 74 00 03 31 32 33 74 00 |........t..123t.|
|000000a0| 08 7a 68 61 6e 67 73 61 6e                      |.zhangsan       |
+--------+-------------------------------------------------+----------------+
11:02:40 [DEBUG] [main] c.i.n.j.c.p.MessageCodec - 16909060, 1, 0, 0, 0, 253
11:02:40 [DEBUG] [main] c.i.n.j.c.p.MessageCodec - LoginRequestMessage(super=Message(sequenceId=0, messageType=0), username=zhangsan, password=123, name=张三)
11:02:40 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ COMPLETE

解读

什么时候可以加 @Sharable

• 当 handler 不保存状态时，就可以安全地在多线程下被共享，可以加@Sharable
• 但要注意对于编解码器类，不能继承 ByteToMessageCodec 或 CombinedChannelDuplexHandler 父类，他们的构造方法对 @Sharable 有限制
• 如果能确保编解码器不会保存状态，可以继承 MessageToMessageCodec 父类，再加@Sharable注解

@Slf4j
@ChannelHandler.Sharable
/**
 * 必须和 LengthFieldBasedFrameDecoder 一起使用，确保接到的 ByteBuf 消息是完整的
 */
public class MessageCodecSharable extends MessageToMessageCodec<ByteBuf, Message> {
    @Override
    protected void encode(ChannelHandlerContext ctx, Message msg, List<Object> outList) throws Exception {
        ByteBuf out = ctx.alloc().buffer();
        // 1. 4 字节的魔数
        out.writeBytes(new byte[]{1, 2, 3, 4});
        // 2. 1 字节的版本,
        out.writeByte(1);
        // 3. 1 字节的序列化方式 jdk 0 , json 1
        out.writeByte(0);
        // 4. 1 字节的指令类型
        out.writeByte(msg.getMessageType());
        // 5. 4 个字节
        out.writeInt(msg.getSequenceId());
        // 无意义，对齐填充
        out.writeByte(0xff);
        // 6. 获取内容的字节数组
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        ObjectOutputStream oos = new ObjectOutputStream(bos);
        oos.writeObject(msg);
        byte[] bytes = bos.toByteArray();
        // 7. 长度
        out.writeInt(bytes.length);
        // 8. 写入内容
        out.writeBytes(bytes);
        outList.add(out);
    }

    @Override
    protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
        int magicNum = in.readInt();
        byte version = in.readByte();
        byte serializerType = in.readByte();
        byte messageType = in.readByte();
        int sequenceId = in.readInt();
        in.readByte();
        int length = in.readInt();
        byte[] bytes = new byte[length];
        in.readBytes(bytes, 0, length);
        ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(bytes));
        Message message = (Message) ois.readObject();
        log.debug("{}, {}, {}, {}, {}, {}", magicNum, version, serializerType, messageType, sequenceId, length);
        log.debug("{}", message);
        out.add(message);
    }
}