基于Google Protobuff和Mina的RPC

基于Google Protobuff和Mina的RPC

RPC(Remote procedure call):In computer science, a remote procedure call (RPC) is an inter-process communication(IPC) that allows a computer program to cause a subroutine or procedure to execute in another address space (commonly on another computer on a shared network) without the programmer explicitly coding the details for this remote interaction. That is, the programmer writes essentially the same code whether the subroutine is local to the executing program, or remote.

简单思想：RPC框架将客户端调用的[服务名称、方法名称、方法参数]打包，通过网络发送到服务端，服务端将数据包解包，在本地查找请求的服务、在服务中调用使用传来的参数调用请求的方法，最后将执行结果打包，通过网络再返回给客户端。类似Restful请求，后者根据URI来定为服务的请求，最终通过HTTP返回JSON格式是数据，RPC一般都封装了底层的网络服务，传输是基于socket的字节流数据包，当然也可以简单的使用HTTP来传输JSON格式的请求和应答。Restful的本质思想略有不同。

RPC中的事件流：

1. The client calls the client stub. The call is a local procedure call, with parameters pushed on to the stack in the normal way.
2. The client stub packs the parameters into a message and makes a system call to send the message. Packing the parameters is called marshalling.
3. The client's local operating system sends the message from the client machine to the server machine.
4. The local operating system on the server machine passes the incoming packets to the server stub.
5. The server stub unpacks the parameters from the message. Unpacking the parameters is called unmarshalling.
6. Finally, the server stub calls the server procedure. The reply traces the same steps in the reverse direction.

Why using Protobuff？

为了让多种不同的Clients（不同语言的客户端方法、对象、参数都不相同）可以访问共同的Server，通常使用IDL(Interface description language)来描述不同平台的RPC服务。大家都使用server使用IDL定义的接口，来调用server上的方法，这样才有共同的语言。

What is Protobuff？

Protobuff是Google提供的一种跨语言的对象序列化方案，同时又支持了IDL的功能，在Google内部广泛使用。类似的有Facebook的thrift，但是后者提供了RPC的实现，而Protobuff只是提供了机制，没有具体的实现，网上有很多对二者的比较。Protobuff的官方文档https://developers.google.com/protocol-buffers/docs/overview

有了共同的语言，还需要解决的是数据传输，客户端的请求和数据需要传送到服务端，此时就需要网络通信了。Java里面使用Mina和Nettry都可以，网上也有很多二者比较。

Protobuff官方列出了一些第三方的RPC实现https://code.google.com/p/protobuf/wiki/ThirdPartyAddOns#RPC_Implementations

简单学习下https://github.com/jcai/fepss-rpc的实现（貌似Protobuff现在不按这个模式搞了，使用code generator plugins了）

定义RPC沟通的IDL

.proto文件中定义了错误码、RPC请求对象、RPC响应对象
package com.aquar.rpc;

option java_package = “com.aquar.rpc”;
option java_outer_classname = “RpcProtobuf”;
option optimize_for = SPEED;

// Possible error reasons
enum ErrorReason {

BAD_REQUEST_DATA = 0;  
BAD_REQUEST_PROTO = 1;  
SERVICE_NOT_FOUND = 2;  
METHOD_NOT_FOUND = 3;  
RPC_ERROR = 4;  
RPC_FAILED = 5;  
CLIENT_FAILED=6;  

}

message Request {
// RPC　request id, used to identify different request
optional uint32 id = 1;

// RPC service full name
required string service_name = 2;

// RPC method name
required string method_name = 3;

// RPC request proto
required bytes request_proto = 4;
}

message Response {
// RPC　request id
optional uint32 id = 1;

// RPC response proto
optional bytes response_proto = 2;

// Eror, if any
optional string error = 3;

// Was callback invoked
optional bool callback = 4 [default = false];

// Error Reason
optional ErrorReason error_reason = 5;
}

编译生成RpcProtobuf类文件
protoc -I=./ –java_out=./src ./rpc.proto

Server端定义

定义Mina用到的编码和解码对象

字节流中结构为 整数+Message对象，其中整数表明整个消息大小，Message为RPC请求或响应对象

Encoder：

Message msg = (Message) message;
int size = msg.getSerializedSize();
IoBuffer buffer = IoBuffer.allocate(SizeContext.computeTotal(size));
CodedOutputStream cos = CodedOutputStream.newInstance(buffer.asOutputStream());
cos.writeRawVarint32(size);
msg.writeTo(cos);
cos.flush();
buffer.flip();
out.write(buffer);

Decoder: extends CumulativeProtocolDecoder

@Override
protected boolean doDecode(IoSession session, IoBuffer in, ProtocolDecoderOutput out) throws Exception {
    SizeContext ctx = SizeContext.get(session, in);
    if(ctx.hasEnoughData(in)) {
        try {
            decodeMessage(in, out, ctx);
            return true;
        } finally {
            ctx.shiftPositionAndReset(session, in);
        }
    }
    return false;
}

private void decodeMessage(IoBuffer in, ProtocolDecoderOutput out,
        SizeContext ctx) throws IOException {
    Message.Builder builder = newBuilder();
    ctx.getInputStream(in).readMessage(builder, ExtensionRegistry.getEmptyRegistry());
    out.write(builder.build());
}
protected abstract Message.Builder newBuilder();

其中使用了SizeContext类来辅助统计字节个数以及buffer相关计算

和普通的Mina服务端一样初始化服务端

int processorCount = Runtime.getRuntime().availableProcessors();
acceptor = new NioSocketAcceptor(processorCount);
acceptor.setReuseAddress(true);
acceptor.getSessionConfig().setReuseAddress(true);
acceptor.getSessionConfig().setReceiveBufferSize(1024);
acceptor.getSessionConfig().setSendBufferSize(1024);
acceptor.getSessionConfig().setTcpNoDelay(true);
acceptor.getSessionConfig().setSoLinger(-1);
acceptor.setBacklog(1024);

acceptor.setDefaultLocalAddress(new InetSocketAddress(port));
DefaultIoFilterChainBuilder chain = acceptor.getFilterChain();
chain.addLast("protobuf", new ProtocolCodecFilter(
        new ProtobufEncoder(), new ProtobufDecoder() {
            @Override
            protected Builder newBuilder() {
                return RpcProtobuf.Request.newBuilder();
            }
        }));

acceptor.setHandler(ioHandler);
acceptor.bind(new InetSocketAddress(port));

服务端的IoHandler

• 在messageReceived中根据请求的服务名称，从服务列表中找到需要的服务service
• 从service中findMethodByName(rpcRequest.getMethodName())找到方法method
• 解析出请求的对象request = service.getRequestPrototypemethod).newBuilderForType().mergeFrom(rpcRequest.getRequestProto());
• service调用方法method service.callMethod(method, controller, request, callback);
• 生成RpcResponse对象，写入session responseBuilder.setCallback(true).setResponseProto(callback.response.toByteString());

其中Callback类主要用来保存具体的Service的method方法需要返回的对象，可以看作一个wrapper，实际的实现方法将需要返回给客户端的对象塞到callback里面，服务端在从callback中将该对象取出来，存入RpcResponse中，客户端通过解析RpcResponse的ResponseProto就得到了服务端方法返回的对象值。

private Map<String, Service> services;

@Override
public void messageReceived(IoSession session, Object message)
        throws Exception {
    Request rpcRequest = (Request) message;
    if (rpcRequest == null) {
        throw new RpcException(ErrorReason.BAD_REQUEST_DATA,
                "request data is null!");
    }
    // Get the service/method
    Service service = services.get(rpcRequest.getServiceName());
    if (service == null) {
        throw new RpcException(ErrorReason.SERVICE_NOT_FOUND,
                "could not find service: " + rpcRequest.getServiceName());
    }
    MethodDescriptor method = service.getDescriptorForType()
            .findMethodByName(rpcRequest.getMethodName());
    if (method == null) {
        throw new RpcException(ErrorReason.METHOD_NOT_FOUND, String.format(
                "Could not find method %s in service %s", rpcRequest
                        .getMethodName(), service.getDescriptorForType()
                        .getFullName()));
    }

    // Parse request
    Message.Builder builder = null;
    try {
        builder = service.getRequestPrototype(method).newBuilderForType()
                .mergeFrom(rpcRequest.getRequestProto());
        if (!builder.isInitialized()) {
            throw new RpcException(ErrorReason.BAD_REQUEST_PROTO,
                    "Invalid request proto");
        }
    } catch (InvalidProtocolBufferException e) {
        throw new RpcException(ErrorReason.BAD_REQUEST_PROTO, e);
    }
    Message request = builder.build();

    // Call method
    RpcControllerImpl controller = new RpcControllerImpl();
    Callback callback = new Callback();
    try {
        service.callMethod(method, controller, request, callback);
    } catch (RuntimeException e) {
        throw new RpcException(ErrorReason.RPC_ERROR, e);
    }

    // Build and return response (callback is optional)
    Builder responseBuilder = Response.newBuilder();
    if (callback.response != null) {
        responseBuilder.setCallback(true).setResponseProto(
                callback.response.toByteString());
    } else {
        // Set whether callback was called
        responseBuilder.setCallback(callback.invoked);
    }
    if (controller.failed()) {
        responseBuilder.setError(controller.errorText());
        responseBuilder.setErrorReason(ErrorReason.RPC_FAILED);
    }
    Response rpcResponse = responseBuilder.build();
    outputResponse(session, rpcResponse);
}
/**
 * Callback that just saves the response and the fact that it was invoked.
 */
private class Callback implements RpcCallback<Message> {

    private Message response;
    private boolean invoked = false;

    public void run(Message response) {
        this.response = response;
        invoked = true;
    }
}

RpcChannel的实现
RpcChannel is Abstract interface for an RPC channel. An RpcChannel represents a communication line to a Service which can be used to call that Service's methods. The Service may be running on another machine. Normally, you should not call an RpcChannel directly, but instead construct a stub Service wrapping it. Starting with version 2.3.0, RPC implementations should not try to build on this, but should instead provide code generator plugins which generate code specific to the particular RPC implementation. This way the generated code can be more appropriate for the implementation in use and can avoid unnecessary layers of indirection.

RpcChannel主要在客户端实现，可以和服务端使用完全不同的语言或者通信框架

在callMethod的实现中主要处理：

• 初始化Mina客户端
• 实现客户端的IoHandler:

a. 在sessionOpened()中根据callMethod传入的参数创建RpcRequest对象，并写入session
b. 在messageReceived()方法中，解析Server传来的RpcReponse，将它转为客户端所调用的方法的返回值，即RpcReponse中的response_proto所对应的返回值
c. 最终回调给客户端请求服务方法时传入的RpcCallback对象中，从而将实际的返回值传给了客户端

         *      java.lang.Throwable)
        final RpcController controller, final Message request,
        final Message responsePrototype, final RpcCallback<Message> done) {
    // check rpc request
    if (!request.isInitialized()) {
        throw new RpcException(ErrorReason.BAD_REQUEST_DATA,
                "request uninitialized!");
    }

    // using MINA IoConnector
    IoConnector connector = new NioSocketConnector();

    // add protocol buffer codec
    DefaultIoFilterChainBuilder chain = connector.getFilterChain();
    chain.addLast("protobuf", new ProtocolCodecFilter(
            new ProtobufEncoder(), new ProtobufDecoder() {
                @Override
                protected Message.Builder newBuilder() {
                    return RpcProtobuf.Response.newBuilder();
                }
            }));

    // connector handler
    connector.setHandler(new IoHandlerAdapter() {
        @Override
        public void messageReceived(IoSession session, Object message)
                throws Exception {
            Response rpcResponse = (Response) message;
            handleResponse(responsePrototype, rpcResponse, controller, done);
            session.close(true);
        }

        /**
         * @see org.apache.mina.core.service.IoHandlerAdapter#sessionOpened(org.apache.mina.core.session.IoSession)
         */
        @Override
        public void sessionOpened(IoSession session) throws Exception {
            ((SocketSessionConfig) session.getConfig()).setKeepAlive(true);

            // Create request protocol buffer
            Request rpcRequest = Request.newBuilder()
                    .setRequestProto(request.toByteString())
                    .setServiceName(method.getService().getFullName())
                    .setMethodName(method.getName()).build();
            // Write request
            session.write(rpcRequest);
        }

        /**
         * @see org.apache.mina.core.service.IoHandlerAdapter#exceptionCaught(org.apache.mina.core.session.IoSession,
         *      java.lang.Throwable)
         */
        @Override
        public void exceptionCaught(IoSession session, Throwable cause)
                throws Exception {
            StringBuilder errorBuilder = new StringBuilder();
            errorBuilder.append("client has runtime exception!\n");
            ByteArrayOutputStream out = new ByteArrayOutputStream();
            cause.printStackTrace(new PrintStream(out));
            errorBuilder.append(out.toString());
            controller.setFailed(errorBuilder.toString());
        }

    });

    // connect remote server
    ConnectFuture cf = connector.connect(new InetSocketAddress(host, port));
    try {
        cf.awaitUninterruptibly();// wait to connect remote server
        cf.getSession().getCloseFuture().awaitUninterruptibly();
    } finally {
        connector.dispose();
    }
}

private void handleResponse(Message responsePrototype,
        Response rpcResponse, RpcController controller,
        RpcCallback<Message> callback) {
    // Check for error
    if (rpcResponse.hasError()) {
        ErrorReason reason = rpcResponse.getErrorReason();
        controller
                .setFailed(reason.name() + " : " + rpcResponse.getError());
        return;
    }

    if ((callback == null) || !rpcResponse.getCallback()) {
        // No callback needed
        return;
    }

    if (!rpcResponse.hasResponseProto()) {
        // Callback was called with null on server side
        callback.run(null);
        return;
    }
    try {
        Message.Builder builder = responsePrototype.newBuilderForType()
                .mergeFrom(rpcResponse.getResponseProto());
        Message response = builder.build();
        //调用客户端请求时定义的回调接口，将实际的返回值response传个客户端
        callback.run(response);
    } catch (InvalidProtocolBufferException e) {
        throw new RuntimeException(e);
    }
}

定义一个Service

自定义一个查找游戏的服务，proto文件：
package com.aquar.rpc.services;

option java_package = “com.aquar.rpc.services”;
option java_outer_classname = “GameServiceProto”;
option java_generic_services = true;

message Game{

optional string gameName = 1;  

}

message Result{

optional string result=1;  
optional bool success=2;  

}

service GameService {
rpc findGame(Game) returns(Result);
}

protoc -I=./ –java_out=./src ./GameService.proto

Server端启动

主要是初始化服务列表，把服务端端口绑定运行起来

public class ServerDemo {
public static String host = "127.0.0.1";
public static int port = 5566;
public static void main(String[] args) {
    Map<String, Service> services = new HashMap<String, Service>();
    services.put(GameService.getDescriptor().getFullName(), new GameServiceImpl());
    ServerIoHandler ioHandler = new ServerIoHandler(services);
    RpcServer server = new RpcServer(host, port, ioHandler);
    try {
        server.start();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
}

}

Client端请求

• 创建RpcChannel对象
• 创建服务Stub
• 创建请求方法的参数以及服务端执行完毕后用到的RpcCallback 回调对象
• 调用服务定义的方法

public static void main(String[] args) {
    RpcChannelImpl channel = new RpcChannelImpl(ServerDemo.host, ServerDemo.port);
    RpcControllerImpl controller = channel.newRpcController();
    Stub service = GameService.newStub(channel); 
    Game request = Game.newBuilder().setGameName("FIFA").build();
    RpcCallback<Result> done = new RpcCallback<GameServiceProto.Result>() {

        @Override
        public void run(Result result) {
            if (result.getSuccess()) {
                System.out.println("Client get " + result.getResult());
            }
        }
    };
    System.out.println("Client request Gameservice for findGame: " + request.getGameName());
    service.findGame(controller, request, done);
}