rpc框架--grpc-java

grpc-java 初探

环境准备

1. 下载protobuf
2. 新建maven项目

使用protoc命令行生成类文件

安装protocbuf

安装步骤略

编译 protoc-gen-grpc-java插件

下载grpc-java后，执行一下命令

cd grpc-java/compile
# 编译插件
../gradlew java_pluginExecutable
# 测试插件
../gradlew test

若看到BUILD SUCCESSFUL字样，则编译成功

编辑proto文件

syntax = "proto3";

option java_multiple_files = true;
option java_package = "io.grpc.examples.helloworld";
# java_package 用来指定生成的java类所在的包，如果该proto文件被指定编译为其他语言时，java参数失效，默认包为package
option java_outer_classname = "HelloWorldProto";
option objc_class_prefix = "HLW";

package helloworld;

# 定义服务
service Greeter {
  # 定义grpc方法，并且制订消息的请求类型和反回类型
  rpc SayHello (HelloRequest) returns (HelloReply) {}
}

# 消息类型都是protocol buffer的消息格式
message HelloRequest {
  string name = 1;
}

message HelloReply {
  string message = 1;
}

grpc可以有四种服务类型
1. 简单的rcp模式，client端使用stub发送请求到服务端，然后进入等待模式，直到响应发送回client端

rpc GetFeature(Point) returns (Feature) {}
2. 服务端流式rpc，client端发送请求到服务端，返回消息序列的流数据（gets a stream to read a sequence of messages back）,client从返回的流数据读取消息，直至不再有数据，如果指定服务端返回流数据类型，需使用stream关键字
rpc ListFeatures(Rectangle) returns (stream Feature) {}
3. 客户端流式rpc，client使用指定的stream方式写消息序列并发送给服务端，一旦客户端写消息完成，会等待服务端全部读取并做出响应
rpc RecordRoute(stream Point) returns (RouteSummary) {}
4. 双向流式rpc，双向流式操作独立，客户端和服务端可以以任何序列随意读写，比如，服务端可以在等待client端发送所有消息之前写响应消息，或者服务端可以选择读一个消息写一个消息或者其他的读写组合。消息的顺序都是被保存的
rpc RouteChat(stream RouteNote) returns (stream RouteNote) {}

生成client端和server端代码

使用protocol编译器protoc和指定的grpc的一个java插件。

使用maven的protobuf插件生成相关代码
命令行：
protoc --java_out=./java/ ./proto/hello.proto

生成grpc通讯代码

protoc --plugin=protoc-gen-grpc-java=/Users/liuyu9/Documents/personal/golang/src/github.com/grpc-java/compiler/build/exe/java_plugin/protoc-gen-grpc-java --grpc-java_out=./java ./proto/hello.proto   

文件列表如下：

# GreeterGrpc包含服务端和客户端要实现的一些基本类
-rw-r--r--  1 root  staff   7352 Oct 25 18:48 GreeterGrpc.java

# 包含了protocol buffer发送、序列化、反序列化我们请求和响应代码
-rw-r--r--  1 root  staff  16207 Oct 25 18:53 HelloReply.java
-rw-r--r--  1 root  staff    501 Oct 25 18:53 HelloReplyOrBuilder.java
-rw-r--r--  1 root  staff  16192 Oct 25 18:53 HelloRequest.java
-rw-r--r--  1 root  staff    493 Oct 25 18:53 HelloRequestOrBuilder.java
-rw-r--r--  1 root  staff   3011 Oct 25 18:53 HelloWorldProto.java

编写服务端文件

编写服务端文件的工作有两部分：
1. 重写proto文件定义的服务生成的服务基类，确定服务的工作内容
2. 运行grpc服务，监听客户端请求并做出响应

package com.weibo.dorteam.grpc;

import io.grpc.Server;
import io.grpc.ServerBuilder;
import io.grpc.stub.StreamObserver;

import io.grpc.examples.helloworld.GreeterGrpc;
import io.grpc.examples.helloworld.HelloReply;
import io.grpc.examples.helloworld.HelloRequest;

import java.io.IOException;
import java.util.logging.Logger;

public class helloServer {
    private static final Logger logger = Logger.getLogger(helloServer.class.getName());
    private int port = 50051;
    private Server server;

    private void start() throws IOException {
      // 使用ServerBuilder来构建和启动服务，通过使用forPort方法来指定监听的地址和端口
      // 创建一个实现方法的服务GreeterImpl的实例，并通过addService方法将该实例纳入
      // 调用build() start()方法构建和启动rpcserver
      server = ServerBuilder.forPort(port)
          .addService(new GreeterImpl())
          .build()
          .start();
      logger.info("Server started, listening on " + port);

        Runtime.getRuntime().addShutdownHook(new Thread() {
            @Override
            public void run() {
              // Use stderr here since the logger may have been reset by its JVM shutdown hook.
              System.err.println("*** shutting down gRPC server since JVM is shutting down");
              helloServer.this.stop();
              System.err.println("*** server shut down");
            }
          });
        }

        private void stop() {
          if (server != null) {
            server.shutdown();
          }
        }

        /**
         * Await termination on the main thread since the grpc library uses daemon threads.
         */
        private void blockUntilShutdown() throws InterruptedException {
          if (server != null) {
            server.awaitTermination();
          }
        }

        /**
         * Main launches the server from the command line.
         */
        public static void main(String[] args) throws IOException, InterruptedException {
          final helloServer server = new helloServer();
          server.start();
          server.blockUntilShutdown();
        }

    // 我们的服务GreeterImpl继承了生成抽象类GreeterGrpc.GreeterImplBase，实现了服务的所有方法
    private class GreeterImpl extends GreeterGrpc.GreeterImplBase {

        @Override
        public void sayHello(HelloRequest req, StreamObserver responseObserver) {
          HelloReply reply = HelloReply.newBuilder().setMessage("Hello " + req.getName()).build();
          // 使用响应监视器的onNext方法返回HelloReply
          responseObserver.onNext(reply);
          // 使用onCompleted方法指定本次调用已经完成
          responseObserver.onCompleted();
        }
      }
}

消息类型，StreamObserver一个响应监视器，是服务器调生成响应结果使用的特定接口

编写客户端文件

为了调用服务端的方法，首先需要创建一个或两个stub：
阻塞式stub：rpc调用会阻塞等待服务端响应后返回消息或异常信息
非阻塞式stub：非阻塞调用方式，响应异步返回

package com.weibo.dorteam.grpc;

import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;
import io.grpc.StatusRuntimeException;
import io.grpc.examples.helloworld.GreeterGrpc;
import io.grpc.examples.helloworld.HelloReply;
import io.grpc.examples.helloworld.HelloRequest;

import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;

public class helloClient {


    private static final Logger logger = Logger.getLogger(helloClient.class.getName());

    private final ManagedChannel channel;
    private final GreeterGrpc.GreeterBlockingStub blockingStub;

    /** Construct client connecting to HelloWorld server at {@code host:port}. */
    // 首先，我们需要为stub创建一个grpc的channel，指定我们连接服务端的地址和端口
    // 使用ManagedChannelBuilder方法来创建channel
    public helloClient(String host, int port) {
        channel = ManagedChannelBuilder.forAddress(host, port)
        // Channels are secure by default (via SSL/TLS). For the example we disable TLS to avoid
        // needing certificates.
        .usePlaintext(true)
        .build();
        // 使用我们从proto文件生成的GreeterGrpc类提供的newBlockingStub方法指定channel创建stubs
        blockingStub = GreeterGrpc.newBlockingStub(channel);
    }

    public void shutdown() throws InterruptedException {
        channel.shutdown().awaitTermination(5, TimeUnit.SECONDS);
    }
    // 调用服务端方法
    /** Say hello to server. */
    public void greet(String name) {
        logger.info("Will try to greet " + name + " ...");
        // 创建并定制protocol buffer对象，使用该对象调用服务端的sayHello方法，获得response
        HelloRequest request = HelloRequest.newBuilder().setName(name).build();
        HelloReply response;
        try {
            response = blockingStub.sayHello(request);
        // 如果有异常发生，则异常被编码成Status，可以从StatusRuntimeException异常中捕获
        } catch (StatusRuntimeException e) {
            logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
            return;
        }
        logger.info("Greeting: " + response.getMessage());
    }

    /**
     * Greet server. If provided, the first element of {@code args} is the name to use in the
     * greeting.
     */
    public static void main(String[] args) throws Exception {
        helloClient client = new helloClient("localhost", 50051);
        try {
            /* Access a service running on the local machine on port 50051 */
            String user = "world";
            if (args.length > 0) {
                user = args[0]; /* Use the arg as the name to greet if provided */
            }
            client.greet(user);
        } finally {
            client.shutdown();
        }
    }
}

mvn编译运行

服务端运行：

#  不需要参数
mvn exec:java -Dexec.mainClass="com.weibo.dorteam.grpc.helloServer"

# 需要参数
mvn exec:java -Dexec.mainClass="com.weibo.dorteam.grpc.helloServer" -Dexec.args="arg0 arg1 arg2"

# 传递classpath
mvn exec:java -Dexec.mainClass="com.weibo.dorteam.grpc.helloServer" -Dexec.classpathScope=runtime

客户端运行：

mvn exec:java -Dexec.mainClass="com.weibo.dorteam.grpc.helloClient"

---

使用maven依赖生成类文件

添加pom依赖

<dependency>
      <groupId>io.grpcgroupId>
      <artifactId>grpc-allartifactId>
      <version>1.0.1version>
 dependency>
<build> 
    <extensions> 
        <extension> 
            <groupId>kr.motd.mavengroupId> 
            <artifactId>os-maven-pluginartifactId> 
            <version>1.4.1.Finalversion> 
        extension> 
    extensions> 
    <plugins> 
        <plugin> 
            <groupId>org.xolstice.maven.pluginsgroupId> 
            <artifactId>protobuf-maven-pluginartifactId> 
            <version>0.5.0version> 
            <configuration> 
                 
                <protocArtifact>com.google.protobuf:protoc:3.0.0:exe:${os.detected.classifier}protocArtifact> 
                <pluginId>grpc-javapluginId> 
                <pluginArtifact>io.grpc:protoc-gen-grpc-java:0.13.2:exe:${os.detected.classifier}pluginArtifact> 
                <protocExecutable>/Users/peng/protoc-3.0.0-beta-2-osx-x86_64/protocprotocExecutable>
            configuration> 
            <executions> 
                <execution> 
                    <goals> 
                        <goal>compilegoal> 
                        <goal>compile-customgoal> 
                    goals> 
                execution> 
            executions> 
        plugin> 
    plugins> 
build>

注：protoc的版本需要匹配protobuf-java,如果不是直接依赖protobuf-java，则依赖grpc时，间接依赖protobuf-java，版本为3.0.0

编辑proto文件

同上

生成相关类文件

mvn generated-sources

默认生成路径为${basedir}/target/generated-sources

编辑服务端文件

同上

编辑客户端文件

同上

使用python编写客户端文件访问java服务端

环境准备

1. python版本2.7及以上
2. 安装pip管理软件
3. pip install grpc
4. python，查看importgrpc是否成功，缺啥装啥
   这里装环境太麻烦，下载了个docker镜像，直接使用了，镜像名grpc/python，用着很方便

编辑proto文件

准备一份proto文件

syntax = "proto3";
service Greeter {
  rpc SayHello (HelloRequest) returns (HelloReply) {}
}

message HelloRequest {
  string name = 1;
}

message HelloReply {
  string message = 1;
}

使用protoc命令生成相关类文件

docker run -i -t -v /data0/liuyu9/test/grpc/python-grpc:/python-grpc grpc/python python -m grpc.tools.protoc -I /python-grpc --python_out=/python-grpc --grpc_python_out=/python-grpc /python-grpc/hello.proto

指定protos文件的路径，以及输出路径，并指定proto 文件

创建客户端

from __future__ import print_function

import grpc

import hello_pb2


def run():
  channel = grpc.insecure_channel('**.**.**.**:50051')
  stub = hello_pb2.GreeterStub(channel)
  response = stub.SayHello(hello_pb2.HelloRequest(name='you'))
  print("Greeter client received: " + response.message)

if __name__ == '__main__':
  run()

使用python的client端去访问java的server端

docker run -i -t -v /data0/liuyu9/test/grpc/python-grpc:/python-grpc grpc/python python /python-grpc/helloClient.py
输出结果为：hello you

若不适用grpc/python的镜像，物理机的python需要安装grpc，安装过程中，版本及各种依赖包的冲突非常麻烦。。。没有耐心了

---

遇到问题

I. pom execution报错：
感觉像是包依赖冲突的问题。。。未解

Multiple annotations found at this line:
    - Execution default of goal org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile-custom failed: An API 
     incompatibility was encountered while executing org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile-custom: 
     java.lang.NoSuchMethodError: org.codehaus.plexus.util.DirectoryScanner.setupMatchPatterns()V 
     ----------------------------------------------------- realm = plugin>org.xolstice.maven.plugins:protobuf-maven-plugin:
     0.5.0--726595060 strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy urls[0] = file:/Users/liuyu9/.m2/
     repository/org/xolstice/maven/plugins/protobuf-maven-plugin/0.5.0/protobuf-maven-plugin-0.5.0.jar urls[1] = file:/Users/
     liuyu9/.m2/repository/org/sonatype/sisu/sisu-inject-bean/1.4.2/sisu-inject-bean-1.4.2.jar urls[2] = file:/Users/liuyu9/.m2/
     repository/org/sonatype/sisu/sisu-guice/2.1.7/sisu-guice-2.1.7-noaop.jar urls[3] = file:/Users/liuyu9/.m2/repository/com/
     google/guava/guava/18.0/guava-18.0.jar urls[4] = file:/Users/liuyu9/.m2/repository/org/apache/maven/plugin-tools/
     maven-plugin-annotations/3.4/maven-plugin-annotations-3.4.jar urls[5] = file:/Users/liuyu9/.m2/repository/org/codehaus/
     plexus/plexus-utils/3.0.22/plexus-utils-3.0.22.jar urls[6] = file:/Users/liuyu9/.m2/repository/org/codehaus/plexus/plexus-
     component-annotations/1.6/plexus-component-annotations-1.6.jar urls[7] = file:/Users/liuyu9/.m2/repository/org/
     sonatype/aether/aether-util/1.7/aether-util-1.7.jar urls[8] = file:/Users/liuyu9/.m2/repository/org/codehaus/plexus/plexus-
     interpolation/1.14/plexus-interpolation-1.14.jar urls[9] = file:/Users/liuyu9/.m2/repository/org/sonatype/plexus/plexus-
     sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.jar urls[10] = file:/Users/liuyu9/.m2/repository/org/sonatype/plexus/plexus-
     cipher/1.4/plexus-cipher-1.4.jar Number of foreign imports: 4 import: Entry[import org.sonatype.plexus.build.incremental 
     from realm ClassRealm[plexus.core, parent: null]] import: Entry[import org.codehaus.plexus.util.Scanner from realm 
     ClassRealm[plexus.core, parent: null]] import: Entry[import org.codehaus.plexus.util.AbstractScanner from realm 
     ClassRealm[plexus.core, parent: null]] import: Entry[import from realm ClassRealm[project>grpcTest:grpcTest:0.0.1-
     SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]] ----------------------------------------------------- 
     (org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile-custom:default:generate-sources)
    - Execution default of goal org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile failed: An API 
     incompatibility was encountered while executing org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile: 
     java.lang.NoSuchMethodError: org.codehaus.plexus.util.DirectoryScanner.setupMatchPatterns()V 
     ----------------------------------------------------- realm = plugin>org.xolstice.maven.plugins:protobuf-maven-plugin:
     0.5.0--726595060 strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy urls[0] = file:/Users/liuyu9/.m2/
     repository/org/xolstice/maven/plugins/protobuf-maven-plugin/0.5.0/protobuf-maven-plugin-0.5.0.jar urls[1] = file:/Users/
     liuyu9/.m2/repository/org/sonatype/sisu/sisu-inject-bean/1.4.2/sisu-inject-bean-1.4.2.jar urls[2] = file:/Users/liuyu9/.m2/
     repository/org/sonatype/sisu/sisu-guice/2.1.7/sisu-guice-2.1.7-noaop.jar urls[3] = file:/Users/liuyu9/.m2/repository/com/
     google/guava/guava/18.0/guava-18.0.jar urls[4] = file:/Users/liuyu9/.m2/repository/org/apache/maven/plugin-tools/
     maven-plugin-annotations/3.4/maven-plugin-annotations-3.4.jar urls[5] = file:/Users/liuyu9/.m2/repository/org/codehaus/
     plexus/plexus-utils/3.0.22/plexus-utils-3.0.22.jar urls[6] = file:/Users/liuyu9/.m2/repository/org/codehaus/plexus/plexus-
     component-annotations/1.6/plexus-component-annotations-1.6.jar urls[7] = file:/Users/liuyu9/.m2/repository/org/
     sonatype/aether/aether-util/1.7/aether-util-1.7.jar urls[8] = file:/Users/liuyu9/.m2/repository/org/codehaus/plexus/plexus-
     interpolation/1.14/plexus-interpolation-1.14.jar urls[9] = file:/Users/liuyu9/.m2/repository/org/sonatype/plexus/plexus-
     sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.jar urls[10] = file:/Users/liuyu9/.m2/repository/org/sonatype/plexus/plexus-
     cipher/1.4/plexus-cipher-1.4.jar Number of foreign imports: 4 import: Entry[import org.sonatype.plexus.build.incremental 
     from realm ClassRealm[plexus.core, parent: null]] import: Entry[import org.codehaus.plexus.util.Scanner from realm 
     ClassRealm[plexus.core, parent: null]] import: Entry[import org.codehaus.plexus.util.AbstractScanner from realm 
     ClassRealm[plexus.core, parent: null]] import: Entry[import from realm ClassRealm[project>grpcTest:grpcTest:0.0.1-
     SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]] ----------------------------------------------------- 
     (org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile:default:generate-sources)

原因：M2E的bug eclipse版本过低

https://github.com/sergei-ivanov/maven-protoc-plugin/issues/10
https://bugs.eclipse.org/bugs/show_bug.cgi?id=422106

更新eclipse为最新版，删除所有.project文件夹，重新更新project

II. 编译protoc-gen-grpc-java插件时报错：

fatal error: ‘google/protobuf/io/zero_copy_stream.h’ file not found

原因：找不到头文件，因为macos系统没有将/usr/local作为搜索头文件和库的默认路径
解决：export CXXFLAGS=”-I/usr/local/include” LDFLAGS=”-L/usr/local/lib”

III. 生成的grpc通讯代码GreeterGrpc编译报错

报错内容：

Multiple markers at this line
    - Method breakpoint:GreeterGrpc$GreeterImplBase [entry] - bindService()
    - The method bindService() of type GreeterGrpc.GreeterImplBase must override a superclass 
     method
    - implements io.grpc.BindableService.bindService

Multiple markers at this line
    - The method invoke(Req, StreamObserver) of type GreeterGrpc.MethodHandlers 
     must override a superclass method
    - implements io.grpc.stub.ServerCalls.UnaryRequestMethod.invoke

Multiple markers at this line
    - implements io.grpc.stub.ServerCalls.StreamingRequestMethod.invoke
    - The method invoke(StreamObserver) of type GreeterGrpc.MethodHandlers must 
     override a superclass method

原因：同问题I，由于M2E插件版本过低有bug
解决办法：更新M2E，或者更新eclipse为最新版

参考文献：

https://github.com/grpc/grpc-java/tree/master/compiler
http://www.grpc.io/docs/guides/