原创 持续集成系列(9)------使用jmxexporter自动监控java容器

目标

使用Prometheus自动发现并监控容器中的jmx数据（使用jmx_exporter）
效果图：

将jmx_prometheus_javaagent打包到镜像中

Dockerfile

FROM java:jdk8-alpine
ADD jmx_prometheus_javaagent-0.12.0.jar /opt/
ADD config.yml /opt/
ADD target/${app_name}.jar /opt/${app_name}.jar
ENTRYPOINT ["java", "-server", "-javaagent:/opt/jmx_prometheus_javaagent-0.12.0.jar=1234:/opt/config.yml", "-jar", "-Dspring.profiles.active=test", "-Dfile.encoding=UTF-8","-Duser.timezone=GMT+08","/opt/${app_name}.jar"]

更新Prometheus配置新增采集java的job

prometheus-additional.yaml

  - job_name: 'java-pods'
    tls_config:
      ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
    bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
    kubernetes_sd_configs:
    - role: endpoints
    relabel_configs:
    - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape, __meta_kubernetes_service_annotation_prometheus_io_jvm_scrape]
      regex: true;true
      action: keep
    - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_app_metrics_path]
      action: replace
      target_label: __metrics_path__
      regex: (.+)
    - source_labels: [__meta_kubernetes_pod_ip, __meta_kubernetes_service_annotation_prometheus_io_jvm_port]
      action: replace
      target_label: __address__
      regex: (.+);(.+)
      replacement: $1:$2
    - source_labels: [__meta_kubernetes_namespace]
      action: replace
      target_label: kubernetes_namespace
    - source_labels: [__meta_kubernetes_pod_name]
      action: replace
      target_label: kubernetes_pod_name

配置更新

kubectl create secret generic additional-scrape-configs --from-file=prometheus-additional.yaml -n monitoring --dry-run -oyaml > additional-scrape-configs.yaml
kubectl replace -f additional-scrape-configs.yaml

pod配置

需要监控的pod的service中增加annotations

  annotations:
    prometheus.io/scrape: "true"
    prometheus.io/jvm-scrape: "true"
    prometheus.io/jvm-port: "1234"
    prometheus.io/jvm-path: "/metrics"

添加后可以在Prometheus的前端的target中看到自动注册的pod