BRPC系列一、编译及简单示例的运行与分析

2020年03月22日00:13:34， 待完善…

前言

如果你要学BRPC，强烈建议查看官方文档https://github.com/apache/incubator-brpc/blob/master/README_cn.md，里面的内容很详细。

写BRPC系列博客的原因并不是要把源文档抄一遍搬到博客上来，而是在看源文档的过程中记录一些重要的点，自己实际写相关代码时候遇到的问题，思考一些流程的必要性。对应部分我都会贴上源文档的链接，便于查询。

BRPC编译问题

在按照官方页面进行编译时（Mac），出现“gnu-getopt must be installed and used“错误，

按照https://www.jianshu.com/p/c28e050955fb的解决方法，将getopt加入环境变量。

gnu-getopt must be installed and used
用brew install gnu-getopt安装即可，并加入路径
export PATH="/usr/local/opt/gnu-getopt/bin:$PATH"

EechoService示例

client端的流程

以https://github.com/apache/incubator-brpc/blob/master/example/echo_c++/client.cpp为例，

流程

1. 定义channel（负责连接）， 以及相关的ChannelOption（设置使用的协议、延时等）进行初始化。
2. 实例化Server_Stub对象对channel进行封装。
3. 实例化Request和Response类型的对象，填入Request信息
4. 实例化Controller对象，并设置相关的参数（controller负责管理此次连接的信息，比如状态位显示当前调用是否成功，比如此次连接延时等等）
5. 使用stub.RPCMethod(&controller, & request, & response, done)即可完成RPC方法的调用。
6. 调用成功，则response对象中已经返回RPCMethod的调用结果。

client代码如下， 可以自行对应。

#include 
#include 
#include 
#include 
#include "echo.pb.h"

DEFINE_string(attachment, "", "Carry this along with requests");
DEFINE_string(protocol, "baidu_std", "Protocol type. Defined in src/brpc/options.proto");
DEFINE_string(connection_type, "", "Connection type. Available values: single, pooled, short");
DEFINE_string(server, "0.0.0.0:8000", "IP Address of server");
DEFINE_string(load_balancer, "", "The algorithm for load balancing");
DEFINE_int32(timeout_ms, 100, "RPC timeout in milliseconds");
DEFINE_int32(max_retry, 3, "Max retries(not including the first RPC)"); 
DEFINE_int32(interval_ms, 1000, "Milliseconds between consecutive requests");

int main(int argc, char* argv[]) {
    // Parse gflags. We recommend you to use gflags as well.
    GFLAGS_NS::ParseCommandLineFlags(&argc, &argv, true);
    
    // A Channel represents a communication line to a Server. Notice that 
    // Channel is thread-safe and can be shared by all threads in your program.
    brpc::Channel channel;
    
    // Initialize the channel, NULL means using default options.
    brpc::ChannelOptions options;
    options.protocol = FLAGS_protocol;
    options.connection_type = FLAGS_connection_type;
    options.timeout_ms = FLAGS_timeout_ms/*milliseconds*/;
    options.max_retry = FLAGS_max_retry;
    if (channel.Init(FLAGS_server.c_str(), FLAGS_load_balancer.c_str(), &options) != 0) {
        LOG(ERROR) << "Fail to initialize channel";
        return -1;
    }

    // Normally, you should not call a Channel directly, but instead construct
    // a stub Service wrapping it. stub can be shared by all threads as well.
    example::EchoService_Stub stub(&channel);

    // Send a request and wait for the response every 1 second.
    int log_id = 0;
    while (!brpc::IsAskedToQuit()) {
        // We will receive response synchronously, safe to put variables
        // on stack.
        example::EchoRequest request;
        example::EchoResponse response;
        brpc::Controller cntl;

        request.set_message("hello world");

        cntl.set_log_id(log_id ++);  // set by user
        // Set attachment which is wired to network directly instead of 
        // being serialized into protobuf messages.
        cntl.request_attachment().append(FLAGS_attachment);

        // Because `done'(last parameter) is NULL, this function waits until
        // the response comes back or error occurs(including timedout).
        stub.Echo(&cntl, &request, &response, NULL);
        if (!cntl.Failed()) {
            LOG(INFO) << "Received response from " << cntl.remote_side()
                << " to " << cntl.local_side()
                << ": " << response.message() << " (attached="
                << cntl.response_attachment() << ")"
                << " latency=" << cntl.latency_us() << "us";
        } else {
            LOG(WARNING) << cntl.ErrorText();
        }
        usleep(FLAGS_interval_ms * 1000L);
    }

    LOG(INFO) << "EchoClient is going to quit";
    return 0;
}

划重点，上面的代码中，核心是stub.RPCMethod(&controller, & request, & response, done），必须要懂里面的参数。

其中的request和response很容易理解，就是调用service的Request信息（可能包含了需要处理的数据或者参数信息），以及service返回的Response信息（Service处理完的返回结果）。

比较难理解的是第一个参数controller，以及最后一个参数done。

Controller

// A Controller mediates a single method call. The primary purpose of
// the controller is to provide a way to manipulate settings per RPC-call 
// and to find out about RPC-level errors.

这是src/brpc/controller.h中Control类的注释，大意是"一个Controller对象对应一次RPC方法调用，用于管理RPC调用的选项以及查找RPC层面的错误"。

我们可以看看Controller类里有啥（感兴趣的读者可以看源文件，这里只截取部分）：

.....
149     // ------------------------------------------------------------------
150     //                      Client-side methods
151     // These calls shall be made from the client side only.  Their results
152     // are undefined on the server side (may crash).
153     // ------------------------------------------------------------------
154 
155     // Set/get timeout in milliseconds for the RPC call. Use
156     // ChannelOptions.timeout_ms on unset.
157     void set_timeout_ms(int64_t timeout_ms);
158     int64_t timeout_ms() const { return _timeout_ms; }
159 
160     // Set/get the delay to send backup request in milliseconds. Use
161     // ChannelOptions.backup_request_ms on unset.
162     void set_backup_request_ms(int64_t timeout_ms);
163     int64_t backup_request_ms() const { return _backup_request_ms; }
164 
165     // Set/get maximum times of retrying. Use ChannelOptions.max_retry on unset.
166     // <=0 means no retry.
167     // Conditions of retrying:
168     //   * The connection is broken. No retry if the connection is still on.
169     //     Use backup_request if you want to issue another request after some
170     //     time.
171     //   * Not timed out.
172     //   * retried_count() < max_retry().
173     //   * Retry may work for the error. E.g. No retry when the request is
174     //     incorrect (EREQUEST), retrying is pointless.
175     void set_max_retry(int max_retry);
176     int max_retry() const { return _max_retry; }
......

如上，有设置time_out时间以及重试次数等等选项。我们可以这么理解，controller对底层socket设置进行了进一步的封装，这样用户在使用BRPC时，对一个RPC方法调用，只需要关心延时，调用是否成功等信息，而无需考虑相关的如何重传等底层逻辑。

注释中出现” Client-side methods“的字样，相对的，当Client进行RPC方法调用将controller对象传到server端后，server端也可以调用里面的方法，比如主动关闭连接等。

...
395     // Tell RPC to close the connection instead of sending back response.
396     // If this controller was not SetFailed() before, ErrorCode() will be
397     // set to ECLOSE.
398     // NOTE: the underlying connection is not closed immediately.
399     void CloseConnection(const char* reason_fmt, ...);
400 
401     // True if CloseConnection() was called.
402     bool IsCloseConnection() const { return has_flag(FLAGS_CLOSE_CONNECTION); }
...

要注意的是（一些方法只能在Client端或Server端进行调用，比如set_timeout_ms只能在client端进行调用，如果在服务端再进行调用，会出现不可预测的错误）

另一个参数涉及到了BRPC调用方式（1. 同步 2. 异步 3. 半同步）

简单来说，同步类似简单的函数调用，发起调用RPC后，Client一直阻塞等待，收到Server端返回Response（即完成RPC调用后）后，继续执行。

异步则是，发起RPC调用后，Client端继续执行（此时RPC调用正在执行…），在RPC调用结束返回Rsponse后，此时调用Client端注册的回调函数，进行进一步的处理（如对Response结果进行处理）

在上面EchoService的例子中，是同步方式的RPC调用，因此done只需设为NULL即可。

若要进行异步方式的RPC调用，则需要借助NewCallback生成done函数，注册相应的回调函数。

更加详细的代码实例，可以参考这篇博客：https://blog.csdn.net/u012414189/article/details/84573786

Server端的流程

以https://github.com/apache/incubator-brpc/blob/master/example/echo_c%2B%2B/server.cpp为例

Server流程

1. 实例化Server对象
2. 注册自定义的服务（如EchoServiceImpl）
3. 实例化ServerOptions对象，在里面设置Server的选项（如time_out的时间）
4. 启动Server

相关代码如下：

#include 
#include 
#include 
#include "echo.pb.h"

DEFINE_bool(echo_attachment, true, "Echo attachment as well");
DEFINE_int32(port, 8000, "TCP Port of this server");
DEFINE_int32(idle_timeout_s, -1, "Connection will be closed if there is no "
             "read/write operations during the last `idle_timeout_s'");
DEFINE_int32(logoff_ms, 2000, "Maximum duration of server's LOGOFF state "
             "(waiting for client to close connection before server stops)");

// Your implementation of example::EchoService
// Notice that implementing brpc::Describable grants the ability to put
// additional information in /status.
namespace example {
class EchoServiceImpl : public EchoService {
public:
    EchoServiceImpl() {};
    virtual ~EchoServiceImpl() {};
    virtual void Echo(google::protobuf::RpcController* cntl_base,
                      const EchoRequest* request,
                      EchoResponse* response,
                      google::protobuf::Closure* done) {
        // This object helps you to call done->Run() in RAII style. If you need
        // to process the request asynchronously, pass done_guard.release().
        brpc::ClosureGuard done_guard(done);

        brpc::Controller* cntl =
            static_cast<brpc::Controller*>(cntl_base);

        // The purpose of following logs is to help you to understand
        // how clients interact with servers more intuitively. You should 
        // remove these logs in performance-sensitive servers.
        LOG(INFO) << "Received request[log_id=" << cntl->log_id() 
                  << "] from " << cntl->remote_side() 
                  << " to " << cntl->local_side()
                  << ": " << request->message()
                  << " (attached=" << cntl->request_attachment() << ")";

        // Fill response.
        response->set_message(request->message());

        // You can compress the response by setting Controller, but be aware
        // that compression may be costly, evaluate before turning on.
        // cntl->set_response_compress_type(brpc::COMPRESS_TYPE_GZIP);

        if (FLAGS_echo_attachment) {
            // Set attachment which is wired to network directly instead of
            // being serialized into protobuf messages.
            cntl->response_attachment().append(cntl->request_attachment());
        }
    }
};
}  // namespace example

int main(int argc, char* argv[]) {
    // Parse gflags. We recommend you to use gflags as well.
    GFLAGS_NS::ParseCommandLineFlags(&argc, &argv, true);

    // Generally you only need one Server.
    brpc::Server server;

    // Instance of your service.
    example::EchoServiceImpl echo_service_impl;

    // Add the service into server. Notice the second parameter, because the
    // service is put on stack, we don't want server to delete it, otherwise
    // use brpc::SERVER_OWNS_SERVICE.
    if (server.AddService(&echo_service_impl, 
                          brpc::SERVER_DOESNT_OWN_SERVICE) != 0) {
        LOG(ERROR) << "Fail to add service";
        return -1;
    }

    // Start the server.
    brpc::ServerOptions options;
    options.idle_timeout_sec = FLAGS_idle_timeout_s;
    if (server.Start(FLAGS_port, &options) != 0) {
        LOG(ERROR) << "Fail to start EchoServer";
        return -1;
    }

    // Wait until Ctrl-C is pressed, then Stop() and Join() the server.
    server.RunUntilAskedToQuit();
    return 0;
}

疑问（未解决）

BRPC支持各种协议，但各种协议有适用的场景么？

包的定义（以baidu_std协议分析为例）

链接： https://github.com/apache/incubator-brpc/blob/master/docs/cn/baidu_std.md

释义	说明
包头	4字节（Magic number ： BRPC） + 4字节（包体长度）+4字节（包体中元数据的长度）
包体（元数据）	用于描述请求/响应，可设置各种参数。可在元数据上进一步拓展
包体 （数据）	自定义的Protobuf Message。用于存放参数或返回结果。
包体 （附件）	某些场景下需要通过RPC来传递二进制数据，例如文件上传下载，多媒体转码等等。将这些二进制数据打包在Protobuf内会增加不必要的内存拷贝。因此协议允许使用附件的方式直接传送二进制数据。

附件总是放在包体的最后，紧跟数据部分。消息包需要携带附件时，应将RpcMeta中的attachment_size设为附件的实际字节数。|

常用链接

（来自BRPC文档）
总体介绍：https://github.com/brpc/brpc
Client：https://github.com/brpc/brpc/blob/master/docs/cn/client.md
Server：https://github.com/brpc/brpc/blob/master/docs/cn/server.md
内置服务：https://github.com/brpc/brpc/blob/master/docs/cn/builtin_service.md
protobuf：https://developers.google.com/protocol-buffers/

Ref：

1. https://zhuanlan.zhihu.com/p/101008345
2. https://blog.csdn.net/breaksoftware/article/details/81564405
3. https://github.com/apache/incubator-brpc/blob/master/README_cn.md