【kafka-exporter监控模板修改】

背景：kafka的jmx指标数据要展示到监控需要prometheus的java agent来采集，jmx指标直接展示不太友好，还需要通过python或java api展示出来，在通过prometheus或其他手段上报到监控，比较麻烦，好在有开源的jmx_exporter可以直接使用，有延时等详细的指标，但是yml模板比较粗糙需要了解jmx的指标含义并知道具体格式然后了解yml中规则配置然后进行改写，如下是我的改写可以采集一些详细的需要信息，jmx的指标含义可以参考kafka官网

1.jmx-exporter 指标收集

github：https://github.com/prometheus/jmx_exporter

配置文件：https://github.com/prometheus/jmx_exporter/blob/master/example_configs/kafka-2_0_0.yml ## kafka服务侧的配置 在kafka-server-start.sh中最后一行之前添加如下配置

export KAFKA_OPTS="-javaagent:$base_dir/../jmx_prometheus_javaagent-0.16.1.jar=9011:$base_dir/../kafka-xxt-2_0_0.yml"

注释：这里我对原有yml文件进行了改写，命名为kafka-xxt-2_0_0.yml，可共使用，路径即为agent和yml文件存放的路径，建议放到kafka的安装目录下

2.模板改写

lowercaseOutputName: true

rules:
# Special cases and very specific rules
- pattern : kafka.server, name=RequestHandlerAvgIdlePercent><>OneMinuteRate
  name: kafka_server_kafkarequesthandlerpool_requesthandleravgidlepercent
  type: GAUGE
#- pattern : kafka.server<>Value
#  name: kafka_server_$1_$2
#  type: GAUGE
#  labels:
#    "$3": "$4"
- pattern : kafka.server, name=(.+), topic=(.*)><>(\w+)Rate
  name: kafka_server_brokertopicmetrics_$1
  type: GAUGE
  labels:
    "topic": "$2"
    "rate": "$3"
- pattern : kafka.server, name=(.+)><>(\w+)Rate
  name: kafka_server_brokertopicmetrics_$1
  type: GAUGE
  labels:
    "rate": "$2"
- pattern : kafka.server, name=(.+)><>Count
  name: kafka_server_brokertopicmetrics_$1_count
  type: GAUGE
- pattern : kafka.server, name=(\w+)><>(\w+)Rate
  name: kafka_server_replicamanager_$1
  type: GAUGE
  labels:
    "rate": "$2"
- pattern : kafka.server, name=(\w+)><>Value
  name: kafka_server_replicamanager_$1_value
  type: GAUGE
- pattern : kafka.network, name=(.+)Ms, request=(.*)><>(\d+)thPercentile
  name: kafka_network_requestmetrics_$1
  type: GAUGE
  labels:
    "reqeust": "$2"
    "quantile": "0.$3"
- pattern : kafka.server, name=(\w+)><>(\w+)Rate
  name: kafka_server_SessionExpireListener_$1
  type: GAUGE
  labels:
    "rate": "$2"
- pattern : kafka.server, name=(\w+)><>Count
  name: kafka_server_SessionExpireListener_$1_count
  type: GAUGE
- pattern : kafka.server-server-metrics, listener=(\w+), networkProcessor=(\d+)><>connection-([\w-]+)
  name: kafka_socket-server-metrics_$1_networkprocessor_$2
  type: GAUGE
  labels:
     "connection": "$3"
#- pattern : kafka.server<>connection-count
#  name: kafka_socketservermetric_$1_networkprocessor_$2_connectioncount
#  type: GAUGE
- pattern : kafka.network, name=(\w+)><>Value
  name: requestchannel_$1_value
  type: GAUGE
- pattern : kafka.log, name=LogFlushRateAndTimeMs><>(\w+)Rate
  name: kafka_log_logflushstats_logflushrateandtimems
  type: GAUGE
  labels:
    "rate": "$1"
- pattern : kafka.log, name=LogFlushRateAndTimeMs><>(\d+)thPercentile
  name: kafka_log_logflushstats_logflushrateandtimems
  type: GAUGE
  labels:
    "quantile": "0.$1"

具体的指标含义和样式可以参数kafka官网和通过python 查询jmx指标获取，然后和模板规则对对照，明白具体样式在做修改