Google Cloud Pubsub使用详解（一）

Pubsub简介和Pub使用详解

一.简介

google cloud pub/sub是一种具有传递和接受消息的事件驱动以及流分析系统，跟kafka比较相似。它具有以下特点：

• 集成支持C++,C#,go,java,node.js,python,ruby

• 消息传递有序且发送效率支持从零到数百GB/秒

• 无需分区的自动扩展和自动配置

• 可跨区域系统（pubsub服务器在全球所有的Google Cloud区域中运行）

• 易上手且成本低

二.Pubsub配置

Google cloud上配置pubsub是非常容易上手的，直接访问Google Cloud Pubsub，在上面创建project(项目），Topic（主题），Subscription（订阅）。

Topic：发布者向其发送消息的命名资源（数据往这发送）

Subscription：一种命名资源，表示来自单个特定主题的消息流，该消息流将传递给订阅应用程序（从Topic订阅发送的信息）

message：发布者发送到主题并最终传递给订阅者的数据和（可选）属性的组合。

发布者应用程序创建并向主题发送消息。订阅者应用程序创建对主题的订阅，以从中接收消息。通讯可以是一对多（扇出），多对一（扇入）和多对多。

三.消息流

三.账户密钥

pubsub配置完成之后为了使第三方可以使用，我们需要为它生成一个账户密钥。Google cloud Pubsub客户端库会在使用的时候自动的读取GOOGLE_APPLICATION_CREDENTIALS 这个环境变量，这个环境变量指向的便是你生成的账户密钥。Idea或者eclipse在使用的时候需要将次环境变量配置到项目的Environment variables中。下面是一个账户密钥的事例：

{
  "type": "service_account",
  "project_id": "concise-option-296909",
  "private_key_id": "9555ce9d54842de70af939f24893e1b0ec9567b9",
  "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQDNBE8fesU33n9g\ntFiWTjtlaIEpAsAzdlF6+Okglv0wLZi6q++cyQ5dqIwzMfob9T+06gK9OpCuPdKf\nhbpujp2Lq8a5m1TrDXMtgdfrnsZs+lGJmwrtG5OcXWxzG/czFfHf8xCYDQVpWkZF\nEK3El4kM7g9Lm2cgQFpylb1ZEduBSW0S1RELuU9fnbEIMAnM9Yrv6vadB9O9rBG0\nvtiFzmqyEE1mNCL+WjeQwbi4G907iagi/cXznXJN/PevPKsWhJ0rIhNY2lQbeuxy\ndYwmqCNuGleToxCD7XQotxI1D0ota6tyXAbc9RSmZDx5Ok1FVhhFrzK/z6xqiUnC\nJqKLOkBZAgMBAAECggEADJQoGqFYul3PaOqHyOkR4d3wlsiRbFblFutwukwpSqxx\ng3qLIA6B8CMoPvfhBZgdiYcWFfAi8l3er2TwFFpxtiEa90ZdhnJzz6X4zn8WY5dU\nK2I04GlhvCVLQyDDIQJGrdvm971MyLL+PryQPNCjWwS8MYJeaG8qXX9sd6uN0TfI\n+JaMERsUeiKOLjSrSdmjpmMguydPB6iClMxBEzcD9rA3jdTqZ+iJ2RJ8iqKjnEdI\nsg2ryHuXgensnQCTu92U+BdZLr12H2x69RgcSNbqXx3ECVqy5xjvsbW8rzpuMOPw\ngONS14UbTlWTeg4Txb4UrpYPUI8sa/mAEMNSVxNMEQKBgQDpDr60PhMCfxjm3555\nsniKH8Im139/hjE9auppXs2NfjX8oBdtoWQuiYZZenngA14Ed3KMKvCHJe7GfnHb\n+ClcCgTPF/KPm0GoEMGQ87C1twHiTzoUA3M5f2WqhXTqXFSdtcAjAmYb/BAnGp1D\n8U0jBbVAjMJuX12oZOJHGlqjjQKBgQDhMumQVL0u0CeQHEpqsthU6fVMaUdAOk91\nJvOTKEVbaCyu/v/aTOu8p3UT6pnjUA1FvdszR7p64a1i+11SHVFVRL83AWO7yTW9\nwXO36FZniaEjg0PaU4ooqqCkzqoi/tuASE14lj+XWyLBKcybEhYsZ9j5OhZpU7DR\nCIDVRZqW/QKBgGyX40JG/kKWiDrjjYs1Hrd1btj8Zrs8AuuUSt+IPg0k9vDcg64g\nmnaYvBIrYGiCCSfZpHUteN/wS98Pgew2KMdrH6yDgxFpDWHLVPY+pmM1f9o8glpa\n9TCgbPXUs5KULRVDX+CwOHXnsHsYCpli8UdVXIxUSjB1nUVepJTu8bbtAoGAIJpO\nVibufKiGEccGvwvFZgW5DabwOcw3VK6e9N8Pq3wZe/xO7jvUpStZN9jvDom4YjBq\nW7I2vTOO9pWNnSXBcKbTSBNbbQRkU2R+bFGmsLasLYgDesg9aEXLnBCPcSGUha3y\nCVM3QaJgFbcHeCvWKfee4fYW6IjFQH4KqNW628ECgYA/itvSAuWYY2/8le+J1DJz\nEQf0Ri3gHZ2HWLGo771avxLfaV2WRZRf34PyHbzr6qiLDKDIn959nL8qdQAaHtI3\nwC2ys+TuNlQW/3QHUwizdHzWiZm2CGvGhVKy3mYZ1hw9ipz9MOU+ZIxRnegJn2Ua\nvf0A9D9eMvMklMk0CxNgIg==\n-----END PRIVATE KEY-----\n",
  "client_email": "[email protected]",
  "client_id": "117770248371861669412",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/test-487%40concise-option-296909.iam.gserviceaccount.com"
}

四.Pub实例

Gradle：

// google cloud pubsub
implementation 'com.google.cloud:google-cloud-pubsub:1.108.7'

java版：

package com.common;

import com.google.api.core.ApiFuture;
import com.google.api.core.ApiFutureCallback;
import com.google.api.core.ApiFutures;
import com.google.cloud.pubsub.v1.Publisher;
import com.google.common.util.concurrent.MoreExecutors;
import com.google.protobuf.ByteString;
import com.google.pubsub.v1.PubsubMessage;
import com.google.pubsub.v1.TopicName;
import java.io.IOException;

public class PubDemo {
    // 初始化TopicName
    private final TopicName topicName = TopicName.of("projectId", "topic");
    private Publisher publisher;

    public PubDemo() {
        initPublisher();
    }

    // 初始化Publisher
    private void initPublisher() {
        try{
          publisher = Publisher.newBuilder(topicName).build();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    public void sendMessage(String message) {
        // 创建Pubsub发送数据的message类型
        PubsubMessage pubsubMessage = PubsubMessage.newBuilder().setData(ByteString.copyFromUtf8(message)).build();
        // 发送message
        ApiFuture apiFuture = publisher.publish(pubsubMessage);
        //异步处理发送的结果
        ApiFutures.addCallback(
                apiFuture,
                callback(), // 回调处理
                MoreExecutors.directExecutor() // 处理的线程
        );
    }

    private ApiFutureCallback callback() {
       return new ApiFutureCallback<>() {
           @Override
           public void onFailure(Throwable t) {
               System.out.println("Send message to google pubsub failed");
           }

           @Override
           public void onSuccess(String result) {
               System.out.println("Send message to google pubsub success.");
           }
       };
    } 
}

Koltin版：

import com.google.api.core.ApiFutures
import com.google.cloud.pubsub.v1.Publisher
import com.google.common.util.concurrent.MoreExecutors
import com.google.protobuf.ByteString
import com.google.pubsub.v1.PubsubMessage
import com.google.pubsub.v1.TopicName
import com.google.api.core.ApiFutureCallback

class NeuralyzerPubsubProducer() {
    private val topicName: TopicName = TopicName.of("projectId", "topic")
    private val publisher: Publisher = Publisher.newBuilder(topicName).build()

    fun sendMessage(message: String) {
        val result = publisher.publish(message.pubsubMessage())
        ApiFutures.addCallback(
                result,
                callback(message),
                MoreExecutors.directExecutor()
        )
    }

    private fun String.pubsubMessage() = PubsubMessage.newBuilder().setData(ByteString.copyFromUtf8(this)).build()

    // 异步回调
    private fun callback(message: String) = object : ApiFutureCallback {
        override fun onFailure(t: Throwable) {
            logger.error("Send message to google pubsub failed, message: $message")
        }

        override fun onSuccess(result: String?) {
            logger.info("Send message to google pubsub succeeded")
        }
    }
}