thanos部署(三) -- sidecar和querier

本篇thanos部署是基于prometheus-operator，prometheus-operator为thanos提供了在CRD和controller上的支持。

sidecar

1. 创建minio的访问配置(s3协议)

# cat thanos-storage-minio.yaml
type: s3
config:
  bucket: thanos
  endpoint: minio.minio.svc.cluster.local:9000
  access_key: minio
  secret_key: minio123
  insecure: true
  signature_version2: false

创建secret保存配置：

$ kubectl create secret generic thanos-objectstorage --from-file=thanos.yaml=thanos-storage-minio.yaml -n monitoring

2. 修改prometheus的CRD，增加thanos的配置

# vi ../prometheus/prometheus-prometheus.yaml
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  labels:
    prometheus: k8s
  name: k8s
  namespace: monitoring
spec:
  ...
  thanos:
    image: thanosio/thanos:v0.18.0
    objectStorageConfig:
      key: thanos.yaml
      name: thanos-objstore-config

更新CRD:

# kubectl replace -f prometheus-prometheus.yaml

Prometheus-Operator监听到CRD的修改后：

• 为每个Prometheus Pod增加1个sidecar container；
• 为prometheus-operated这个svc增加1个10901的grpc访问端口；
  

3. thanos使用的端口统计：

• 10900: cluster-port;
• 10901: grpc;
• 10902: http;

querier

1. 部署querier deployment
   由于querier组件需要与sidecar和store组件对接，故querier需要配置：

• sidecar的访问地址；
• store的访问地址；(该配置待store组件部署ok后再修改)

# cat thanos-querier-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: thanos-querier
  namespace: monitoring
  labels:
    app: thanos-querier
spec:
  replicas: 1
  selector:
    matchLabels:
      app: thanos-querier
  template:
    metadata:
      labels:
        app: thanos-querier
    spec:
      containers:
      - name: thanos
        image: thanosio/thanos:v0.18.0
        args:
        - "query"
        - "--log.level=debug"
        - "--query.replica-label=prometheus_replica"
        - "--store=dnssrv+prometheus-operated:10901"
        ports:
        - name: http
          containerPort: 10902
        - name: grpc
          containerPort: 10901
        - name: cluster
          containerPort: 10900

这里仅配置了sidecar的访问地址：--store=dnssrv+prometheus-operated:10901

2. 部署querier service

querier Service用NodePort将服务暴露出来，可以使用NodePort访问querier WEB UI做查询；

# cat thanos-querier-svc.yaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    prometheus.io/path: /metrics
    prometheus.io/port: "10902"
    prometheus.io/scrape: "true"
  name: thanos-querier
  labels:
    app: thanos-querier
  namespace: monitoring
spec:
  selector:
    app: thanos-querier
  ports:
  - port: 9090
    targetPort: http
    name: http-querier
    nodePort: 32700
  type: NodePort

暴露的NodePort=32700，访问WEB UI如下：

访问“store”菜单，可以看到querier对接的存储列表：sidecar已被识别

3. 创建querier-ServiceMonitor将其加入prometheus监控

# cat thanos-querier-serviceMonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: thanos-querier
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: thanos-querier
  namespaceSelector:
    matchNames:
      - "monitoring"
  endpoints:
  - port: http-querier
    interval: 15s

serviceMonitor被创建后，可以在prometheus WEB UI的target列表中查看：