Kubernetes 上的RabbitMQ集群搭建(按官方文档搭建)

先看完再做

官方文档：

https://www.rabbitmq.com/kubernetes/operator/using-operator.html
https://www.rabbitmq.com/kubernetes/operator/install-operator.html
https://www.rabbitmq.com/kubernetes/operator/quickstart-operator.html

前期准备

全部阿里云上搭建
在搭建前需要：
一个Kubernetes集群

购买SLB实例

创建一个规格为 slb.s1.small 的内网实例即可，可用区不要求，可跨可用区负载。
因为是给同一个VPC的服务使用(这些第三方插件在单独一个集群)，所以只需要一个内网的slb即可。

存储

使用alicloud-disk-efficiency创建云盘，不需要共享磁盘

开始配置

一、安装RabbitMQ Cluster Kubernetes Operator

wget -O 0-cluster-operator.yaml https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml
sed -i 's#rabbitmq-system#alpha-rabbitmq#g' *.yaml
kubectl apply  -f 0-cluster-operator.yaml 

1. 自动创建alpha-rabbitmq命名空间(如果换成自己的命名空间，记得把下面所有yaml中的namespace值改掉)

二、创建密文（可选）

1-secret.yaml

# 因为我的MQ镜像在内网,所以创建了一个密文，如果要用公网的，可以不创建
---
apiVersion: v1
data:
  .dockerconfigjson: eabcdefghijikjlasdfbuasdfbucwersfgX0=
kind: Secret
metadata:
  name: aliyun-registry
  namespace: alpha-rabbitmq
type: kubernetes.io/dockerconfigjson

1. 创建一个aliyun-registry的镜像密文，拉取我内网的MQ镜像，如果要用公网，可以不创建

三、部署Statefulset

2-statefulset.yaml

apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
  name: rabbitmq-cluster-node
  namespace: alpha-rabbitmq
spec:
  # 如果用官方镜像，把这三行删除
  image: "registry-vpc.cn-beijing.aliyuncs.com/bbbbbbbbb/rabbitmq:latest"
  imagePullSecrets: 
  - name: aliyun-registry
  #####
  persistence:
      storage: 20Gi
      storageClassName: alicloud-disk-efficiency
  rabbitmq:
      additionalPlugins: 
        - rabbitmq_peer_discovery_k8s
        - rabbitmq_management
        - rabbitmq_prometheus
      additionalConfig: |
        cluster_partition_handling = pause_minority
        vm_memory_high_watermark_paging_ratio = 0.99
        disk_free_limit.relative = 1.0
        collect_statistics_interval = 10000
        # 在Queues页面,给每个队列加个totals,统计队列有多少消息
        management.enable_queue_totals = true
  replicas: 3
  # 额外配置了一个15692的端口给prometheus,如果不需要可以把整个 service部分 和 statefulSet.spec.template.spec.containers.ports部分删掉
  override:
    service:
      spec:
        ports:
          - name: prometheus
            protocol: TCP
            port: 15692
    statefulSet:
      spec:
        template:
          spec:
            initContainers:
              - name: "rabbitmq-config"
                image: registry-vpc.cn-beijing.aliyuncs.com/acegear/busybox:1.32.0
                volumeMounts:
                  - name: rabbitmq-config-rw
                    mountPath: /etc/rabbitmq/conf.d/
                command:
                - sh
                - -c
                # 开启web的统计信息及指标收集,即开启Exchanges和Channels标签页
                # 报错：点Channels时报Stats in management UI are disabled on this node
                # 请看：https://stackoverflow.com/questions/63708061/how-to-enable-stats-in-rabbitmq-management-ui
                # 暂时没找到怎么直接用扩展配置设置,就先用命令写吧
                - echo "management_agent.disable_metrics_collector = false" > /etc/rabbitmq/conf.d/management_agent.disable_metrics_collector.conf
            volumes:
              - name: rabbitmq-config-rw
                emptyDir: {}
            containers:
              - name: rabbitmq
                ports:
                  - containerPort: 15692 # opens an additional port on the rabbitmq server container
                    name: prometheus
                    protocol: TCP
                volumeMounts:
                - name: rabbitmq-config-rw
                  mountPath: "/etc/rabbitmq/conf.d/"

执行命令将pod创建起来

kubectl apply -f 2-statefulset.yaml

八、创建客户端连接服务

阿里云官方文档

3-client-service.yaml

apiVersion: v1
kind: Service
metadata:
  annotations:
    # 将slb-id更改为自己
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: lb-c4bdaaaaaaaaagy7jql1k
    service.beta.kubernetes.io/alicloud-loadbalancer-force-override-listeners: 'true'
  namespace: alpha-rabbitmq
  name: rabbitmq-client
  # labels:
  #   app: rabbitmq
  #   type: LoadBalancer
spec:
  type: LoadBalancer
  ports:
    - name: amqp
      port: 5672
      protocol: TCP
      targetPort: 5672
    - name: management
      port: 15672
      protocol: TCP
      targetPort: 15672
  selector:
    app.kubernetes.io/name: rabbitmq-cluster-node
  sessionAffinity: None

九、获取用户名及密码

#用户名
kubectl get secrets -n alpha-rabbitmq rabbitmq-cluster-node-default-user -o jsonpath={.data.username} | base64 --decode


#密码
kubectl get secrets -n alpha-rabbitmq rabbitmq-cluster-node-default-user -o jsonpath={.data.password} | base64 --decode