一文学好部署安装p8s operator

概述

为了在Kubernetes中能够方便管理和部署Prometheus,我们使用ConfigMap管理Prometheus配置文件。
promethuse中间也刚好8个字符,我们也称为p8s.

p8s Operator架构原理

从概念上来讲Operator就是针对管理特定应用程序的,在Kubernetes基本的Resource和Controller的概念上,以扩展Kubernetes api的形式,帮助用户创建,配置和管理复杂的有状态应用程序,从而实现特定应用程序的常见操作以及运维自动化。因此,通过声明式方式,Kubernetes允许用户添加自己的自定义资源(Custom Resource),并且通过实现自定义Controller来实现对Kubernetes的扩展。用户只需要关心应用程序的最终状态。


p8s Operator架构

Prometheus的本质就是一组用户自定义的CRD资源以及Controller的实现,Prometheus Operator负责监听这些自定义资源的变化,并且根据这些资源的定义自动化的完成如Prometheus Server自身以及配置的自动化管理工作:

  • Prometheus:声明式创建和管理Prometheus Server实例;
  • ServiceMonitor:负责声明式的管理监控配置;
  • PrometheusRule:负责声明式的管理告警配置;
  • Alertmanager:声明式的创建和管理Alertmanager实例。

部署

创建命名空间

kubectl create namespace monitoring

部署operator

下载Prometheus Operator的源码,并部署

git clone https://github.com/coreos/prometheus-operator.git
cd prometheus-operator/
kubectl -n monitoring apply -f bundle.yaml
kubectl -n monitoring get pods

创建Prometheus实例

声明Prometheus资源:

apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  name: inst
  namespace: monitoring
spec:
  serviceAccountName: prometheus
  serviceMonitorSelector:
    matchLabels:
      team: frontend
  ruleSelector:
    matchLabels:
      role: alert-rules
      prometheus: example
  alerting:
    alertmanagers:
    - name: alertmanager-example
      namespace: monitoring
      port: web
  resources:
    requests:
      memory: 400Mi

创建一个Prometheus实例:

kubectl apply -f  prometheus-inst.yaml
kubectl -n monitoring get statefulset
kubectl -n monitoring get pods

ServiceMonitor管理监控配置

创建service
kubectl apply -f prometheus-service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app: prometheus-operator
  name: prometheus-operator-svc
  namespace: monitoring
spec:
  ports:
  - name: operator
    port: 9090
    protocol: TCP
    targetPort: 9090
  selector:
    app: prometheus
    prometheus: inst
  sessionAffinity: None
  type: NodePort

在master节点通过port-forward访问任意Pod实例

kubectl port-forward deployments/example-app 8080:8080
#新打开master节点查看指标
curl localhost:8080/metrics
#指标列表
# HELP codelab_api_http_requests_in_progress The current number of API HTTP requests in progress.
# TYPE codelab_api_http_requests_in_progress gauge
codelab_api_http_requests_in_progress 1
# HELP codelab_api_request_duration_seconds A histogram of the API HTTP request durations in seconds.
# TYPE codelab_api_request_duration_seconds histogram
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.0001"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.00015000000000000001"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.00022500000000000002"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.0003375"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.00050625"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.000759375"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.0011390624999999999"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.0017085937499999998"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.0025628906249999996"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.0038443359374999994"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.00576650390625"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.008649755859375"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.0129746337890625"} 6342
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.01946195068359375"} 83643
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.029192926025390625"} 83814
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.043789389038085935"} 85925
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.0656840835571289"} 89587
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.09852612533569335"} 89587
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.14778918800354002"} 89587
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.22168378200531003"} 89587
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.33252567300796504"} 89587
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.49878850951194753"} 89587
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="0.7481827642679213"} 89587
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="1.122274146401882"} 89587
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="1.683411219602823"} 89587
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="200",le="+Inf"} 89587
codelab_api_request_duration_seconds_sum{method="GET",path="/api/bar",status="200"} 1530.125271866964
codelab_api_request_duration_seconds_count{method="GET",path="/api/bar",status="200"} 89587
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.0001"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.00015000000000000001"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.00022500000000000002"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.0003375"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.00050625"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.000759375"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.0011390624999999999"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.0017085937499999998"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.0025628906249999996"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.0038443359374999994"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.00576650390625"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.008649755859375"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.0129746337890625"} 9
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.01946195068359375"} 206
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.029192926025390625"} 207
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.043789389038085935"} 259
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.0656840835571289"} 355
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.09852612533569335"} 355
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.14778918800354002"} 355
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.22168378200531003"} 355
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.33252567300796504"} 355
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.49878850951194753"} 355
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="0.7481827642679213"} 355
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="1.122274146401882"} 355
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="1.683411219602823"} 355
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/bar",status="500",le="+Inf"} 355
codelab_api_request_duration_seconds_sum{method="GET",path="/api/bar",status="500"} 9.830579268000006
codelab_api_request_duration_seconds_count{method="GET",path="/api/bar",status="500"} 355
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.0001"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.00015000000000000001"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.00022500000000000002"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.0003375"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.00050625"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.000759375"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.0011390624999999999"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.0017085937499999998"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.0025628906249999996"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.0038443359374999994"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.00576650390625"} 1
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.008649755859375"} 10183
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.0129746337890625"} 151198
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.01946195068359375"} 151602
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.029192926025390625"} 155266
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.043789389038085935"} 161339
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.0656840835571289"} 161339
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.09852612533569335"} 161339
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.14778918800354002"} 161339
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.22168378200531003"} 161339
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.33252567300796504"} 161339
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.49878850951194753"} 161339
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="0.7481827642679213"} 161339
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="1.122274146401882"} 161339
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="1.683411219602823"} 161339
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="200",le="+Inf"} 161339
codelab_api_request_duration_seconds_sum{method="GET",path="/api/foo",status="200"} 1832.2875421590238
codelab_api_request_duration_seconds_count{method="GET",path="/api/foo",status="200"} 161339
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.0001"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.00015000000000000001"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.00022500000000000002"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.0003375"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.00050625"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.000759375"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.0011390624999999999"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.0017085937499999998"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.0025628906249999996"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.0038443359374999994"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.00576650390625"} 0
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.008649755859375"} 43
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.0129746337890625"} 727
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.01946195068359375"} 729
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.029192926025390625"} 933
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.043789389038085935"} 1230
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.0656840835571289"} 1230
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.09852612533569335"} 1230
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.14778918800354002"} 1230
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.22168378200531003"} 1230
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.33252567300796504"} 1230
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.49878850951194753"} 1230
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="0.7481827642679213"} 1230
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="1.122274146401882"} 1230
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="1.683411219602823"} 1230
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/foo",status="500",le="+Inf"} 1230
codelab_api_request_duration_seconds_sum{method="GET",path="/api/foo",status="500"} 22.503760457999984
codelab_api_request_duration_seconds_count{method="GET",path="/api/foo",status="500"} 1230
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.0001"} 5415
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.00015000000000000001"} 5419
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.00022500000000000002"} 5419
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.0003375"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.00050625"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.000759375"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.0011390624999999999"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.0017085937499999998"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.0025628906249999996"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.0038443359374999994"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.00576650390625"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.008649755859375"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.0129746337890625"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.01946195068359375"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.029192926025390625"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.043789389038085935"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.0656840835571289"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.09852612533569335"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.14778918800354002"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.22168378200531003"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.33252567300796504"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.49878850951194753"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="0.7481827642679213"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="1.122274146401882"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="1.683411219602823"} 5420
codelab_api_request_duration_seconds_bucket{method="GET",path="/api/nonexistent",status="404",le="+Inf"} 5420
codelab_api_request_duration_seconds_sum{method="GET",path="/api/nonexistent",status="404"} 0.12959933899999979
codelab_api_request_duration_seconds_count{method="GET",path="/api/nonexistent",status="404"} 5420
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.0001"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.00015000000000000001"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.00022500000000000002"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.0003375"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.00050625"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.000759375"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.0011390624999999999"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.0017085937499999998"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.0025628906249999996"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.0038443359374999994"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.00576650390625"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.008649755859375"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.0129746337890625"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.01946195068359375"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.029192926025390625"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.043789389038085935"} 1939
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.0656840835571289"} 18377
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.09852612533569335"} 18392
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.14778918800354002"} 18989
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.22168378200531003"} 19780
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.33252567300796504"} 19780
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.49878850951194753"} 19780
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="0.7481827642679213"} 19780
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="1.122274146401882"} 19780
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="1.683411219602823"} 19780
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="200",le="+Inf"} 19780
codelab_api_request_duration_seconds_sum{method="POST",path="/api/bar",status="200"} 1130.1983095349967
codelab_api_request_duration_seconds_count{method="POST",path="/api/bar",status="200"} 19780
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.0001"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.00015000000000000001"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.00022500000000000002"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.0003375"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.00050625"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.000759375"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.0011390624999999999"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.0017085937499999998"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.0025628906249999996"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.0038443359374999994"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.00576650390625"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.008649755859375"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.0129746337890625"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.01946195068359375"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.029192926025390625"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.043789389038085935"} 16
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.0656840835571289"} 187
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.09852612533569335"} 187
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.14778918800354002"} 250
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.22168378200531003"} 343
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.33252567300796504"} 343
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.49878850951194753"} 343
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="0.7481827642679213"} 343
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="1.122274146401882"} 343
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="1.683411219602823"} 343
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/bar",status="500",le="+Inf"} 343
codelab_api_request_duration_seconds_sum{method="POST",path="/api/bar",status="500"} 32.82294161699999
codelab_api_request_duration_seconds_count{method="POST",path="/api/bar",status="500"} 343
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.0001"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.00015000000000000001"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.00022500000000000002"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.0003375"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.00050625"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.000759375"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.0011390624999999999"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.0017085937499999998"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.0025628906249999996"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.0038443359374999994"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.00576650390625"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.008649755859375"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.0129746337890625"} 3
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.01946195068359375"} 5586
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.029192926025390625"} 15023
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.043789389038085935"} 15026
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.0656840835571289"} 15853
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.09852612533569335"} 16042
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.14778918800354002"} 16042
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.22168378200531003"} 16042
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.33252567300796504"} 16042
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.49878850951194753"} 16042
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="0.7481827642679213"} 16042
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="1.122274146401882"} 16042
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="1.683411219602823"} 16042
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="200",le="+Inf"} 16042
codelab_api_request_duration_seconds_sum{method="POST",path="/api/foo",status="200"} 363.95353269799966
codelab_api_request_duration_seconds_count{method="POST",path="/api/foo",status="200"} 16042
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.0001"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.00015000000000000001"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.00022500000000000002"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.0003375"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.00050625"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.000759375"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.0011390624999999999"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.0017085937499999998"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.0025628906249999996"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.0038443359374999994"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.00576650390625"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.008649755859375"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.0129746337890625"} 0
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.01946195068359375"} 103
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.029192926025390625"} 297
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.043789389038085935"} 298
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.0656840835571289"} 483
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.09852612533569335"} 527
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.14778918800354002"} 527
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.22168378200531003"} 527
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.33252567300796504"} 527
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.49878850951194753"} 527
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="0.7481827642679213"} 527
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="1.122274146401882"} 527
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="1.683411219602823"} 527
codelab_api_request_duration_seconds_bucket{method="POST",path="/api/foo",status="500",le="+Inf"} 527
codelab_api_request_duration_seconds_sum{method="POST",path="/api/foo",status="500"} 19.897669178000008
codelab_api_request_duration_seconds_count{method="POST",path="/api/foo",status="500"} 527
# HELP go_gc_duration_seconds A summary of the GC invocation durations.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 0.000159671
go_gc_duration_seconds{quantile="0.25"} 0.00035243400000000005
go_gc_duration_seconds{quantile="0.5"} 0.00042174400000000005
go_gc_duration_seconds{quantile="0.75"} 0.0004949970000000001
go_gc_duration_seconds{quantile="1"} 0.0007201330000000001
go_gc_duration_seconds_sum 0.278104232
go_gc_duration_seconds_count 663
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 38
# HELP go_memstats_alloc_bytes Number of bytes allocated and still in use.
# TYPE go_memstats_alloc_bytes gauge
go_memstats_alloc_bytes 1.803888e+06
# HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.
# TYPE go_memstats_alloc_bytes_total counter
go_memstats_alloc_bytes_total 1.937319376e+09
# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.
# TYPE go_memstats_buck_hash_sys_bytes gauge
go_memstats_buck_hash_sys_bytes 1.484967e+06
# HELP go_memstats_frees_total Total number of frees.
# TYPE go_memstats_frees_total counter
go_memstats_frees_total 2.1147933e+07
# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata.
# TYPE go_memstats_gc_sys_bytes gauge
go_memstats_gc_sys_bytes 204800
# HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use.
# TYPE go_memstats_heap_alloc_bytes gauge
go_memstats_heap_alloc_bytes 1.803888e+06
# HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used.
# TYPE go_memstats_heap_idle_bytes gauge
go_memstats_heap_idle_bytes 2.875392e+06
# HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use.
# TYPE go_memstats_heap_inuse_bytes gauge
go_memstats_heap_inuse_bytes 2.793472e+06
# HELP go_memstats_heap_objects Number of allocated objects.
# TYPE go_memstats_heap_objects gauge
go_memstats_heap_objects 7016
# HELP go_memstats_heap_released_bytes_total Total number of heap bytes released to OS.
# TYPE go_memstats_heap_released_bytes_total counter
go_memstats_heap_released_bytes_total 0
# HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system.
# TYPE go_memstats_heap_sys_bytes gauge
go_memstats_heap_sys_bytes 5.668864e+06
# HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection.
# TYPE go_memstats_last_gc_time_seconds gauge
go_memstats_last_gc_time_seconds 1.6642818001861356e+19
# HELP go_memstats_lookups_total Total number of pointer lookups.
# TYPE go_memstats_lookups_total counter
go_memstats_lookups_total 128949
# HELP go_memstats_mallocs_total Total number of mallocs.
# TYPE go_memstats_mallocs_total counter
go_memstats_mallocs_total 2.1154949e+07
# HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures.
# TYPE go_memstats_mcache_inuse_bytes gauge
go_memstats_mcache_inuse_bytes 4832
# HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system.
# TYPE go_memstats_mcache_sys_bytes gauge
go_memstats_mcache_sys_bytes 16384
# HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures.
# TYPE go_memstats_mspan_inuse_bytes gauge
go_memstats_mspan_inuse_bytes 35616
# HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system.
# TYPE go_memstats_mspan_sys_bytes gauge
go_memstats_mspan_sys_bytes 65536
# HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place.
# TYPE go_memstats_next_gc_bytes gauge
go_memstats_next_gc_bytes 4.194304e+06
# HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations.
# TYPE go_memstats_other_sys_bytes gauge
go_memstats_other_sys_bytes 1.249361e+06
# HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator.
# TYPE go_memstats_stack_inuse_bytes gauge
go_memstats_stack_inuse_bytes 622592
# HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator.
# TYPE go_memstats_stack_sys_bytes gauge
go_memstats_stack_sys_bytes 622592
# HELP go_memstats_sys_bytes Number of bytes obtained by system. Sum of all system allocations.
# TYPE go_memstats_sys_bytes gauge
go_memstats_sys_bytes 9.312504e+06
# HELP http_request_duration_microseconds The HTTP request latencies in microseconds.
# TYPE http_request_duration_microseconds summary
http_request_duration_microseconds{handler="api",quantile="0.5"} 11578.693
http_request_duration_microseconds{handler="api",quantile="0.9"} 30188.677
http_request_duration_microseconds{handler="api",quantile="0.99"} 59723.244
http_request_duration_microseconds_sum{handler="api"} 4.944684391777816e+09
http_request_duration_microseconds_count{handler="api"} 294623
http_request_duration_microseconds{handler="prometheus",quantile="0.5"} NaN
http_request_duration_microseconds{handler="prometheus",quantile="0.9"} NaN
http_request_duration_microseconds{handler="prometheus",quantile="0.99"} NaN
http_request_duration_microseconds_sum{handler="prometheus"} 0
http_request_duration_microseconds_count{handler="prometheus"} 0
# HELP http_request_size_bytes The HTTP request sizes in bytes.
# TYPE http_request_size_bytes summary
http_request_size_bytes{handler="api",quantile="0.5"} 71
http_request_size_bytes{handler="api",quantile="0.9"} 109
http_request_size_bytes{handler="api",quantile="0.99"} 109
http_request_size_bytes_sum{handler="api"} 2.2355889e+07
http_request_size_bytes_count{handler="api"} 294623
http_request_size_bytes{handler="prometheus",quantile="0.5"} NaN
http_request_size_bytes{handler="prometheus",quantile="0.9"} NaN
http_request_size_bytes{handler="prometheus",quantile="0.99"} NaN
http_request_size_bytes_sum{handler="prometheus"} 0
http_request_size_bytes_count{handler="prometheus"} 0
# HELP http_requests_total Total number of HTTP requests made.
# TYPE http_requests_total counter
http_requests_total{code="0",handler="api",method="get"} 250926
http_requests_total{code="0",handler="api",method="post"} 35822
http_requests_total{code="404",handler="api",method="get"} 5420
http_requests_total{code="500",handler="api",method="get"} 1585
http_requests_total{code="500",handler="api",method="post"} 870
# HELP http_response_size_bytes The HTTP response sizes in bytes.
# TYPE http_response_size_bytes summary
http_response_size_bytes{handler="api",quantile="0.5"} 0
http_response_size_bytes{handler="api",quantile="0.9"} 0
http_response_size_bytes{handler="api",quantile="0.99"} 35
http_response_size_bytes_sum{handler="api"} 265805
http_response_size_bytes_count{handler="api"} 294623
http_response_size_bytes{handler="prometheus",quantile="0.5"} NaN
http_response_size_bytes{handler="prometheus",quantile="0.9"} NaN
http_response_size_bytes{handler="prometheus",quantile="0.99"} NaN
http_response_size_bytes_sum{handler="prometheus"} 0
http_response_size_bytes_count{handler="prometheus"} 0
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 220.38
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 1.048576e+06
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 11
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 9.936896e+06
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.66427690008e+09
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 3.50339072e+08

为了采集部署在Kubernetes下应用的监控数据,p8s中需要配置:

  • 在Prometheus配置文件中定义单独的Job
  • 使用kubernetes_sd定义整个服务发现过程
    但是在Prometheus Operator中,则可以直接声明一个ServiceMonitor对象,如下所示:
    kubectl create -f example-app-service-monitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: example-app
  namespace: monitoring
  labels:
    team: frontend
spec:
  namespaceSelector:
    matchNames:
    - default
  selector:
    matchLabels:
      app: example-app
  endpoints:
  - port: web

你可能感兴趣的:(一文学好部署安装p8s operator)