Golang gRPC实践 在gRPC中使用FlatBuffers编码

介绍

gRPC默认使用Protocol Buffers编码,同时也支持其他编码如:JSON、FlatBuffers等。

FlatBuffers是一个跨平台的序列化库,旨在实现最大的内存效率。它允许您直接访问序列化数据,而无需首先对其进行解析/解包,同时仍具有良好的向前/向后兼容性。

项目地址:https://github.com/google/flatbuffers

FlatBuffers在解编码性能上要比Protocol Buffers快很多,这里有两篇详细介绍Protocol Buffers和FlatBuffers对比的文章:

https://blog.csdn.net/chosen0ne/article/details/43033575
https://juzii.gitee.io/2020/03/02/protobuf-vs-flatbuffer/

这里有一篇文章详细介绍了FlatBuffers以及schema的编写:

https://halfrost.com/flatbuffers_schema/

这里主要来演示一下如何在gRPC中使用FlatBuffers.

编码

编写fbs接口定义文件

api/fbs/greeter.fbs

namespace models;

table HelloReply {
  message:string;
}

table HelloRequest {
  name:string;
}

table ManyHellosRequest {
  name:string;
  num_greetings:int;
}

rpc_service Greeter {
  SayHello(HelloRequest):HelloReply;
  SayManyHellos(ManyHellosRequest):HelloReply (streaming: "server");
}

这里的定义和protobuf差不多,使用table定义结构体,使用rcp_service定义接口。

这里定义了三个结构体用于数据发送和接收,定义了两个接口用于演示。

生成gRPC代码

先安装flatc 从下面地址中下载对应版本的flatc即可

https://github.com/google/flatbuffers/releases/tag/v2.0.0

flatc --go --grpc -o api/ api/fbs/greeter.fbs

参数说明:

--go 指定生成的语言是go
--grpc 指定生成grpc代码
-o 可选,指定要生成的目标文件目录前缀
--go-namespace 可选,指定生成的包名,覆盖 fbs 文件中的定义

会在指定目录下生成一个models目录,里面即是生成的代码,这个目录名就是fbs文件中定义的namespace,也可以通过参数'--go-namespace来覆盖这个值,以指定新的目录,如:

flatc --go --go-namespace newmodels --grpc -o api/ api/fbs/greeter.fbs

建议通过fbs定义namespace,这个namespace也是Go文件的package名称。

生成的文件目录是这样的:

├── api
│   ├── fbs
│   │   └── greeter.fbs
│   └── models
│       ├── Greeter_grpc.go
│       ├── HelloReply.go
│       ├── HelloRequest.go
│       └── ManyHellosRequest.go

现在我们可以编写gRPC的代码了。

初始化go mod

在项目根目录执行:

go mod init github.com/safeie/grpc-flatbuffers-example
go mod tidy

编写gRPC服务端

cmd/server/main.go

package main

import (
    "context"
    "fmt"
    "log"
    "net"

    "github.com/safeie/grpc-flatbuffers-example/api/models"
    "google.golang.org/grpc"

    flatbuffers "github.com/google/flatbuffers/go"
)

var (
    greetings = [...]string{"Hi", "Hallo", "Ciao"}
)

type greeterServer struct {
    models.UnimplementedGreeterServer
}

func (s *greeterServer) SayHello(ctx context.Context, request *models.HelloRequest) (*flatbuffers.Builder, error) {
    v := request.Name()
    var m string
    if v == nil {
        m = "Unknown"
    } else {
        m = string(v)
    }
    b := flatbuffers.NewBuilder(0)
    idx := b.CreateString("Welcome " + m)
    models.HelloReplyStart(b)
    models.HelloReplyAddMessage(b, idx)
    b.Finish(models.HelloReplyEnd(b))
    return b, nil
}

func (s *greeterServer) SayManyHellos(request *models.ManyHellosRequest, stream models.Greeter_SayManyHellosServer) error {
    v := request.Name()
    var m string
    if v == nil {
        m = "Unknown"
    } else {
        m = string(v)
    }
    num := request.NumGreetings()
    b := flatbuffers.NewBuilder(0)

    for _, greeting := range greetings {
        idx := b.CreateString(fmt.Sprintf("%s %s ,num %d", greeting, m, num))
        models.HelloReplyStart(b)
        models.HelloReplyAddMessage(b, idx)
        b.Finish(models.HelloReplyEnd(b))
        if err := stream.Send(b); err != nil {
            return err
        }
    }

    return nil
}

func newServer() *greeterServer {
    return &greeterServer{}
}

func main() {
    lis, err := net.Listen("tcp", fmt.Sprintf("localhost:%d", 3000))
    if err != nil {
        log.Fatalf("Falied to listen: %v", err)
    }

    codec := &flatbuffers.FlatbuffersCodec{}
    grpcServer := grpc.NewServer(grpc.ForceServerCodec(codec))
    models.RegisterGreeterServer(grpcServer, newServer())
    if err := grpcServer.Serve(lis); err != nil {
        fmt.Println(err)
        panic(err)
    }
}

可以 go build测试,如果有依赖问题,回到根目录,执行 go mod tidy下载依赖就可以了。

编写gRPC客户端

cmd/client/main.go

package main

import (
    "context"
    "flag"
    "fmt"
    "io"
    "log"
    "math/rand"
    "time"

    flatbuffers "github.com/google/flatbuffers/go"
    "github.com/safeie/grpc-flatbuffers-example/api/models"
    "google.golang.org/grpc"
)

var (
    addr = "3000"
    name = flag.String("name", "Flatbuffers", "name to be sent go server :D")
)

func printSayHello(client models.GreeterClient, name string) {
    log.Printf("Name to be sent (%s)", name)
    b := flatbuffers.NewBuilder(0)
    i := b.CreateString(name)
    models.HelloRequestStart(b)
    models.HelloRequestAddName(b, i)
    b.Finish(models.HelloRequestEnd(b))

    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()
    request, err := client.SayHello(ctx, b, grpc.CallContentSubtype("flatbuffers"))
    if err != nil {
        log.Fatalf("%v.SayHello(_) = _, %v: ", client, err)
    }
    log.Printf("server said %q", request.Message())
}

func printSayManyHello(client models.GreeterClient, name string, num int32) {
    log.Printf("Name to be sent (%s), num to be sent (%d)", name, num)
    b := flatbuffers.NewBuilder(0)
    i := b.CreateString(name)
    models.ManyHellosRequestStart(b)
    models.ManyHellosRequestAddName(b, i)
    models.ManyHellosRequestAddNumGreetings(b, num)
    b.Finish(models.ManyHellosRequestEnd(b))

    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()
    stream, err := client.SayManyHellos(ctx, b, grpc.CallContentSubtype("flatbuffers"))
    if err != nil {
        log.Fatalf("%v.SayManyHellos(_) = _, %v", client, err)
    }
    for {
        request, err := stream.Recv()
        if err == io.EOF {
            break
        }
        if err != nil {
            log.Fatalf("%v.SayManyHellos(_) = _, %v", client, err)
        }
        log.Printf("server said %q", request.Message())
    }
}

func main() {
    flag.Parse()
    conn, err := grpc.Dial(fmt.Sprintf("localhost:%s", addr), grpc.WithInsecure(), grpc.WithCodec(flatbuffers.FlatbuffersCodec{}))
    if err != nil {
        log.Fatalf("Failed to dial: %v", err)
    }
    defer conn.Close()
    client := models.NewGreeterClient(conn)
    printSayHello(client, *name)

    num := rand.Int31()
    printSayManyHello(client, *name, num)
}

可以 go build测试,如果有依赖问题,回到根目录,执行 go mod tidy下载依赖就可以了。

运行测试

开一个命令行窗口运行:cd cmd/server && go run main.go

开一个命令行窗口运行:cd cmd/client && go run main.go

输出结果应该是这样的:

2021/12/15 18:04:16 Name to be sent (Flatbuffers)
2021/12/15 18:04:16 server said "Welcome Flatbuffers"
2021/12/15 18:04:16 Name to be sent (Flatbuffers), num to be sent (1298498081)
2021/12/15 18:04:16 server said "Hi Flatbuffers ,num 1298498081"
2021/12/15 18:04:16 server said "Hallo Flatbuffers ,num 1298498081"
2021/12/15 18:04:16 server said "Ciao Flatbuffers ,num 1298498081"

完成的示例项目代码:https://github.com/safeie/grpc-flatbuffers-example

参考链接:

https://github.com/google/flatbuffers/tree/master/grpc/examples/go/greeter

你可能感兴趣的:(Golang gRPC实践 在gRPC中使用FlatBuffers编码)