gRPC之proto数据验证

1、proto数据验证

本篇将介绍grpc_validator，它可以对gRPC数据的输入和输出进行验证。

这里复用上一篇文章的代码。

1.1 创建proto文件，添加验证规则

这里使用第三方插件go-proto-validators 自动生成验证规则。

地址：https://github.com/mwitkow/go-proto-validators

$ go get github.com/mwitkow/go-proto-validators

1.1.1 新建simple.proto文件

syntax = "proto3";
package proto;
option go_package = "./proto;proto";

// validator.proto是github.com/mwitkow/go-proto-validators/validator.proto
import "validator.proto";

message InnerMessage {
    // some_integer can only be in range (1, 100).
    int32 some_integer = 1 [(validator.field) = {int_gt: 0, int_lt: 100}];
    // some_float can only be in range (0;1).
    double some_float = 2 [(validator.field) = {float_gte: 0, float_lte: 1}];
}

message OuterMessage {
    // important_string must be a lowercase alpha-numeric of 5 to 30 characters (RE2 syntax).
    string important_string = 1 [(validator.field) = {regex: "^[a-z]{2,5}$"}];
    // proto3 doesn't have `required`, the `msg_exist` enforces presence of InnerMessage.
    InnerMessage inner = 2 [(validator.field) = {msg_exists: true}];
}

service Simple {
    rpc Route (InnerMessage) returns (OuterMessage) {};
}

validator.proto文件内容：

// Copyright 2016 Michal Witkowski. All Rights Reserved.
// See LICENSE for licensing terms.

// Protocol Buffers extensions for defining auto-generateable validators for messages.

// TODO(mwitkow): Add example.


syntax = "proto2";
package validator;

import "google/protobuf/descriptor.proto";

option go_package = "github.com/mwitkow/go-proto-validators;validator";

// TODO(mwitkow): Email [email protected] to get an extension ID.

extend google.protobuf.FieldOptions {
  optional FieldValidator field = 65020;
}

extend google.protobuf.OneofOptions {
  optional OneofValidator oneof = 65021;
}

message FieldValidator {
  // Uses a Golang RE2-syntax regex to match the field contents.
  optional string regex = 1;
  // Field value of integer strictly greater than this value.
  optional int64 int_gt = 2;
  // Field value of integer strictly smaller than this value.
  optional int64 int_lt = 3;
  // Used for nested message types, requires that the message type exists.
  optional bool msg_exists = 4;
  // Human error specifies a user-customizable error that is visible to the user.
  optional string human_error = 5;
  // Field value of double strictly greater than this value.
  // Note that this value can only take on a valid floating point
  // value. Use together with float_epsilon if you need something more specific.
  optional double float_gt = 6;
  // Field value of double strictly smaller than this value.
  // Note that this value can only take on a valid floating point
  // value. Use together with float_epsilon if you need something more specific.
  optional double float_lt = 7;
  // Field value of double describing the epsilon within which
  // any comparison should be considered to be true. For example,
  // when using float_gt = 0.35, using a float_epsilon of 0.05
  // would mean that any value above 0.30 is acceptable. It can be
  // thought of as a {float_value_condition} +- {float_epsilon}.
  // If unset, no correction for floating point inaccuracies in
  // comparisons will be attempted.
  optional double float_epsilon = 8;
  // Floating-point value compared to which the field content should be greater or equal.
  optional double float_gte = 9;
  // Floating-point value compared to which the field content should be smaller or equal.
  optional double float_lte = 10;
  // Used for string fields, requires the string to be not empty (i.e different from "").
  optional bool string_not_empty = 11;
  // Repeated field with at least this number of elements.
  optional int64 repeated_count_min = 12;
  // Repeated field with at most this number of elements.
  optional int64 repeated_count_max = 13;
  // Field value of length greater than this value.
  optional int64 length_gt = 14;
  // Field value of length smaller than this value.
  optional int64 length_lt = 15;
  // Field value of length strictly equal to this value.
  optional int64 length_eq = 16;
  // Requires that the value is in the enum.
  optional bool is_in_enum = 17;
  // Ensures that a string value is in UUID format.
  // uuid_ver specifies the valid UUID versions. Valid values are: 0-5.
  // If uuid_ver is 0 all UUID versions are accepted.
  optional int32 uuid_ver = 18;
}

message OneofValidator {
  // Require that one of the oneof fields is set.
  optional bool required = 1;
}

1.1.2 编译simple.proto文件

$ go get github.com/mwitkow/go-proto-validators/protoc-gen-govalidators
$ go install github.com/mwitkow/go-proto-validators/protoc-gen-govalidators

$ protoc --govalidators_out=. --go_out=plugins=grpc:. simple.proto

编译完成后，自动生成simple.pb.go和simple.validator.pb.go文件，simple.pb.go文件不再介绍，我们

看下simple.validator.pb.go文件。

// Code generated by protoc-gen-gogo. DO NOT EDIT.
// source: simple.proto

package proto

import (
	fmt "fmt"
	math "math"
	proto "github.com/golang/protobuf/proto"
	_ "github.com/mwitkow/go-proto-validators"
	regexp "regexp"
	github_com_mwitkow_go_proto_validators "github.com/mwitkow/go-proto-validators"
)

// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf

func (this *InnerMessage) Validate() error {
	if !(this.SomeInteger > 0) {
		return github_com_mwitkow_go_proto_validators.FieldError("SomeInteger", fmt.Errorf(`value '%v' must be greater than '0'`, this.SomeInteger))
	}
	if !(this.SomeInteger < 100) {
		return github_com_mwitkow_go_proto_validators.FieldError("SomeInteger", fmt.Errorf(`value '%v' must be less than '100'`, this.SomeInteger))
	}
	if !(this.SomeFloat >= 0) {
		return github_com_mwitkow_go_proto_validators.FieldError("SomeFloat", fmt.Errorf(`value '%v' must be greater than or equal to '0'`, this.SomeFloat))
	}
	if !(this.SomeFloat <= 1) {
		return github_com_mwitkow_go_proto_validators.FieldError("SomeFloat", fmt.Errorf(`value '%v' must be lower than or equal to '1'`, this.SomeFloat))
	}
	return nil
}

var _regex_OuterMessage_ImportantString = regexp.MustCompile(`^[a-z]{2,5}$`)

func (this *OuterMessage) Validate() error {
	if !_regex_OuterMessage_ImportantString.MatchString(this.ImportantString) {
		return github_com_mwitkow_go_proto_validators.FieldError("ImportantString", fmt.Errorf(`value '%v' must be a string conforming to regex "^[a-z]{2,5}$"`, this.ImportantString))
	}
	if nil == this.Inner {
		return github_com_mwitkow_go_proto_validators.FieldError("Inner", fmt.Errorf("message must exist"))
	}
	if this.Inner != nil {
		if err := github_com_mwitkow_go_proto_validators.CallValidatorIfExists(this.Inner); err != nil {
			return github_com_mwitkow_go_proto_validators.FieldError("Inner", err)
		}
	}
	return nil
}

里面自动生成了message中属性的验证规则。

1.2 把grpc_validator验证拦截器添加到服务端

package main

import (
	"context"
	grpc_middleware "github.com/grpc-ecosystem/go-grpc-middleware"
	grpc_auth "github.com/grpc-ecosystem/go-grpc-middleware/auth"
	grpc_zap "github.com/grpc-ecosystem/go-grpc-middleware/logging/zap"
	grpc_recovery "github.com/grpc-ecosystem/go-grpc-middleware/recovery"
	grpc_validator "github.com/grpc-ecosystem/go-grpc-middleware/validator"
	pb "middleware/proto"
	"middleware/server/middleware/auth"
	"middleware/server/middleware/cred"
	"middleware/server/middleware/recovery"
	"middleware/server/middleware/zap"
	"google.golang.org/grpc"
	"log"
	"net"
)

// SimpleService 定义我们的服务
type SimpleService struct{}

// Route 实现Route方法
func (s *SimpleService) Route(ctx context.Context, req *pb.InnerMessage) (*pb.OuterMessage, error) {
	res := pb.OuterMessage{
		ImportantString: "hello grpc validator",
		Inner:           req,
	}
	return &res, nil
}

const (
	// Address 监听地址
	Address string = ":8000"
	// Network 网络通信协议
	Network string = "tcp"
)

func main() {
	// 监听本地端口
	listener, err := net.Listen(Network, Address)
	if err != nil {
		log.Fatalf("net.Listen err: %v", err)
	}
	// 新建gRPC服务器实例
	grpcServer := grpc.NewServer(
		// TLS+Token认证
		cred.TLSInterceptor(),
		grpc.StreamInterceptor(grpc_middleware.ChainStreamServer(
			// 参数验证
			grpc_validator.StreamServerInterceptor(),
			// grpc_zap日志记录
			grpc_zap.StreamServerInterceptor(zap.ZapInterceptor()),
			// grpc_auth认证
			grpc_auth.StreamServerInterceptor(auth.AuthInterceptor),
			// grpc_recovery恢复
			grpc_recovery.StreamServerInterceptor(recovery.RecoveryInterceptor()),
		)),
		grpc.UnaryInterceptor(grpc_middleware.ChainUnaryServer(
			// 参数验证
			grpc_validator.UnaryServerInterceptor(),
			// grpc_zap日志记录
			grpc_zap.UnaryServerInterceptor(zap.ZapInterceptor()),
			// grpc_auth认证
			grpc_auth.UnaryServerInterceptor(auth.AuthInterceptor),
			// grpc_recovery恢复
			grpc_recovery.UnaryServerInterceptor(recovery.RecoveryInterceptor()),
		)),
	)
	// 在gRPC服务器注册我们的服务
	pb.RegisterSimpleServer(grpcServer, &SimpleService{})
	log.Println(Address + " net.Listing with TLS and token...")
	//用服务器 Serve() 方法以及我们的端口信息区实现阻塞等待，直到进程被杀死或者 Stop() 被调用
	err = grpcServer.Serve(listener)
	if err != nil {
		log.Fatalf("grpcServer.Serve err: %v", err)
	}
}

[root@zsx middleware]# go run server.go
2023/02/11 21:42:01 :8000 net.Listing with TLS and token...

1.3 客户端

package main

import (
	"context"
	"middleware/token"
	pb "middleware/proto"
	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials"
	"log"
)

// Address 连接地址
const Address string = ":8000"

var grpcClient pb.SimpleClient

func main() {
	//从输入的证书文件中为客户端构造TLS凭证
	creds, err := credentials.NewClientTLSFromFile("./cert/server/server.pem", "test.example.com")
	if err != nil {
		log.Fatalf("Failed to create TLS credentials %v", err)
	}
	//构建Token
	token := auth.Token{
		Value: "bearer grpc.auth.token",
	}
	// 连接服务器
	conn, err := grpc.Dial(Address, grpc.WithTransportCredentials(creds), grpc.WithPerRPCCredentials(&token))
	if err != nil {
		log.Fatalf("net.Connect err: %v", err)
	}
	defer conn.Close()
	// 建立gRPC连接
	grpcClient = pb.NewSimpleClient(conn)
	route()
}

// route 调用服务端Route方法
func route() {
	// 创建发送结构体
	req := pb.InnerMessage{
		SomeInteger: 99,
		SomeFloat:   1,
	}
	// 调用我们的服务(Route方法)
	// 同时传入了一个 context.Context ，在有需要时可以让我们改变RPC的行为，比如超时/取消一个正在运行的RPC
	res, err := grpcClient.Route(context.Background(), &req)
	if err != nil {
		log.Fatalf("Call Route err: %v", err)
	}
	// 打印返回值
	log.Println(res)
}

[root@zsx middleware]# go run client.go
2023/02/11 21:44:11 important_string:"hello grpc validator" inner:{some_integer:99 some_float:1}

运行后，当输入不符合要求的数据：

// 创建发送结构体
req := pb.InnerMessage{
	SomeInteger: 199,
	SomeFloat:   1,
}

验证失败后，会有以下错误返回：

[root@zsx middleware]# go run client.go
2023/02/11 21:45:25 Call Route err: rpc error: code = InvalidArgument desc = invalid field SomeInteger: value '199' must be less than '100'
exit status 1

# 项目结构
$ tree middleware/
middleware/
├── cert
│   ├── ca.crt
│   ├── ca.csr
│   ├── ca.key
│   ├── ca.srl
│   ├── client
│   │   ├── client.csr
│   │   ├── client.key
│   │   └── client.pem
│   ├── openssl.cnf
│   └── server
│       ├── server.csr
│       ├── server.key
│       └── server.pem
├── client.go
├── go.mod
├── go.sum
├── log
│   └── debug.log
├── proto
│   ├── simple.pb.go
│   └── simple.validator.pb.go
├── server
│   └── middleware
│       ├── auth
│       │   └── auth.go
│       ├── cred
│       │   └── cred.go
│       ├── recovery
│       │   └── recovery.go
│       └── zap
│           └── zap.go
├── server.go
├── simple.proto
├── token
│   └── token.go
└── validator.proto

12 directories, 25 files

1.3 其他类型验证规则设置

enum验证：

syntax = "proto3";
package proto;
option go_package = "./proto;enum";

import "validator.proto";

message SomeMsg {
    Action do = 1 [(validator.field) = {is_in_enum: true}];
}
enum Action {
    ALLOW = 0;
    DENY = 1;
    CHILL = 2;
}

UUID验证：

syntax = "proto3";
package proto;
option go_package = "./proto;uuid";

import "validator.proto";

message UUIDMsg {
    // user_id must be a valid version 4 UUID.
    string user_id = 1 [(validator.field) = {uuid_ver: 4, string_not_empty: true}];
}

1.4 总结

go-grpc-middleware中grpc_validator集成go-proto-validators，我们只需要在编写proto时设好验证规

则，并把grpc_validator添加到gRPC服务端，就能完成gRPC的数据验证，很简单也很方便。