[jmx]zookeeper/kafka/hive/hadoop/presto/presto等组件的jmx监控汇集

背景

项目借用jmx监控，也调研了一些时间，那就记个笔记吧，有些之前整理过，直接上链接了~ 都测试过了，均可使用。
jmx的jar包是：jmx_prometheus_javaagent-0.16.0.jar 或 jmx_prometheus_javaagent-0.15.0.jar

注意事项——本文必读

jmx的jar包路径、端口号 和 yaml文件的路径请自行修改
如：
{{install_dir}}/monitor/jmx_prometheus_javaagent-0.16.1.jar表示的是jar包路径

{{ prometh_port }} 是监控的可访问端口号，通过curl http://ip:port可以进行访问的。

{{install_dir}}/monitor/hadoop/hms.yaml 是yaml文件地址

zookeeper的jmx监控配置

参考我的博客：
https://blog.csdn.net/MyNameIsWangYi/article/details/124904575?spm=1001.2014.3001.5501

hadoop的jmx监控

https://blog.csdn.net/MyNameIsWangYi/article/details/124691276?spm=1001.2014.3001.5501

kafka的jmx监控

实际操作：（路径自己修改哦）
1.编辑 kafka/bin/kafka-server-start.sh，第17行增加如下内容：

export KAFKA_OPTS="-javaagent:{{ install_dir }}/monitor/jmx_prometheus_javaagent-0.16.1.jar={{ prometheus_port }}:{{ install_dir }}/monitor/hadoop/kafka.yaml"

2.{{ install_dir }}/monitor/hadoop/kafka.yaml内容如下：

#hostPort: {{inventory_hostname}}:{{jmx_port}}
#lowercaseOutputName: true

rules:
# Special cases and very specific rules
- pattern : kafka.server<type=(.+), name=(.+), clientId=(.+), topic=(.+), partition=(.*)><>Value
  name: kafka_server_$1_$2
  type: GAUGE
  labels:
    clientId: "$3"
    topic: "$4"
    partition: "$5"
- pattern : kafka.server<type=(.+), name=(.+), clientId=(.+), brokerHost=(.+), brokerPort=(.+)><>Value
  name: kafka_server_$1_$2
  type: GAUGE
  labels:
    clientId: "$3"
    broker: "$4:$5"
- pattern : kafka.coordinator.(\w+)<type=(.+), name=(.+)><>Value
  name: kafka_coordinator_$1_$2_$3
  type: GAUGE

# Generic per-second counters with 0-2 key/value pairs
- pattern: kafka.(\w+)<type=(.+), name=(.+)PerSec\w*, (.+)=(.+), (.+)=(.+)><>Count
  name: kafka_$1_$2_$3_total
  type: COUNTER
  labels:
    "$4": "$5"
    "$6": "$7"
- pattern: kafka.(\w+)<type=(.+), name=(.+)PerSec\w*, (.+)=(.+)><>Count
  name: kafka_$1_$2_$3_total
  type: COUNTER
  labels:
    "$4": "$5"
- pattern: kafka.(\w+)<type=(.+), name=(.+)PerSec\w*><>Count
  name: kafka_$1_$2_$3_total
  type: COUNTER

- pattern: kafka.server<type=(.+), client-id=(.+)><>([a-z-]+)
  name: kafka_server_quota_$3
  type: GAUGE
  labels:
    resource: "$1"
    clientId: "$2"

- pattern: kafka.server<type=(.+), user=(.+), client-id=(.+)><>([a-z-]+)
  name: kafka_server_quota_$4
  type: GAUGE
  labels:
    resource: "$1"
    user: "$2"
    clientId: "$3"
# Generic gauges with 0-2 key/value pairs
- pattern: kafka.(\w+)<type=(.+), name=(.+), (.+)=(.+), (.+)=(.+)><>Value
  name: kafka_$1_$2_$3
  type: GAUGE
  labels:
    "$4": "$5"
    "$6": "$7"
- pattern: kafka.(\w+)<type=(.+), name=(.+), (.+)=(.+)><>Value
  name: kafka_$1_$2_$3
  type: GAUGE
  labels:
    "$4": "$5"
- pattern: kafka.(\w+)<type=(.+), name=(.+)><>Value
  name: kafka_$1_$2_$3
  type: GAUGE

# Emulate Prometheus 'Summary' metrics for the exported 'Histogram's.
#
# Note that these are missing the '_sum' metric!
- pattern: kafka.(\w+)<type=(.+), name=(.+), (.+)=(.+), (.+)=(.+)><>Count
  name: kafka_$1_$2_$3_count
  type: COUNTER
  labels:
    "$4": "$5"
    "$6": "$7"
- pattern: kafka.(\w+)<type=(.+), name=(.+), (.+)=(.*), (.+)=(.+)><>(\d+)thPercentile
  name: kafka_$1_$2_$3
  type: GAUGE
  labels:
    "$4": "$5"
    "$6": "$7"
    quantile: "0.$8"
- pattern: kafka.(\w+)<type=(.+), name=(.+), (.+)=(.+)><>Count
  name: kafka_$1_$2_$3_count
  type: COUNTER
  labels:
    "$4": "$5"
- pattern: kafka.(\w+)<type=(.+), name=(.+), (.+)=(.*)><>(\d+)thPercentile
  name: kafka_$1_$2_$3
  type: GAUGE
  labels:
    "$4": "$5"
    quantile: "0.$6"
- pattern: kafka.(\w+)<type=(.+), name=(.+)><>Count
  name: kafka_$1_$2_$3_count
  type: COUNTER
- pattern: kafka.(\w+)<type=(.+), name=(.+)><>(\d+)thPercentile
  name: kafka_$1_$2_$3
  type: GAUGE
  labels:
    quantile: "0.$4"

下面shell请忽略（个人笔记）：

  shell: sed -i '17iexport KAFKA_OPTS="-javaagent:{{ install_dir }}/monitor/jmx_prometheus_javaagent-0.16.1.jar={{ prometheus_port }}:{{ install_dir }}/monitor/hadoop/kafka.yaml"' {{  install_dir  }}/kafka/bin/kafka-server-start.sh

hive的jmx监控

如下位置修改，别的位置不一定生效哦！！！！！！

hiveserver2的jmx监控

编辑hive/bin/hive文件，加入如下内容：

if [[ "$SERVICE" =~ ^hiveserver2$ ]] ; then
  export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -javaagent:{{install_dir}}/monitor/jmx_prometheus_javaagent-0.16.1.jar={{ prometheus_port }}:{{install_dir}}/monitor/hadoop/hs2.yaml"
fi

hs2.yaml内容：

rules:
    - pattern: '.*'

metastore的jmx监控

编辑hive/bin/hive文件，加入如下内容：

if [[ "$SERVICE" =~ ^metastore$ ]] ; then
  export HADOOP_OPTS="$HADOOP_OPTS -javaagent:{{install_dir}}/monitor/jmx_prometheus_javaagent-0.16.1.jar={{ prometheus_port }}:{{install_dir}}/monitor/hadoop/hms.yaml"
fi

hms.yaml文件内容：

rules:
    - pattern: '.*'

下面shell请忽略（个人笔记）：

  shell: restult=`cat -n  {{ install_dir }}/hive/bin/hive | grep 'help|version|' | awk '{print $1+2}'`;hs2Go=`cat {{ install_dir }}/hive/bin/hive | grep "hs2.yaml" `;if [[  -z "$hs2Go"   ]];then sed -i " "$restult"r  {{ install_dir }}/hive/bin/hjmx.txt" {{ install_dir }}/hive/bin/hive ;fi

  shell: restult=`cat -n  {{ install_dir }}/hive/bin/hive | grep 'help|version|' | awk '{print $1+2}'`;hs2Go=`cat {{ install_dir }}/hive/bin/hive | grep "hms.yaml" `;if [[  -z "$hs2Go"   ]];then sed -i " "$restult"r  {{ install_dir }}/hive/bin/mjmx.txt" {{ install_dir }}/hive/bin/hive ;fi

kylin的jmx监控

编辑kylin/bin/kylin.sh文件:

 -javaagent:/opt/bdp/monitor/jmx_prometheus_javaagent-0.16.1.jar=9099:/opt/bdp/monitor/hadoop/kylin.yaml \

kylin.yaml内容

rules:
    - pattern: '.*'

loki的监控

自带的端口号
修改配置文件loki.yaml：http_listen_port: 3100

promtail的监控

自带的端口号
修改配置文件promtail.yaml：http_listen_port: 9080

doris的监控

自带的端口号
修改配置文件fe.conf： http_port=8030

presto-server的jmx监控

在presto-server/etc/jvm.txt文件最底部追加如下内容：

-javaagent:{{install_dir}}/monitor/jmx_prometheus_javaagent-0.16.1.jar={{ prometheus_port }}:{{ install_dir }}/monitor/hadoop/presto-server.yaml

presto-server.yaml内容：

rules:
    - pattern: "com.facebook.presto.hive<>(.+AllTime.+): (.*)"
      name: "presto hive_namenodestats_$1"
      help: "Presto Hive: NameNodeStats $1"
      type: GAUGE
    - pattern: "com.facebook.presto.hive<>(.+TotalCount.*): (.*)"
      name: "presto hive_namenodestats_$1"
      help: "Presto Hive: NameNodeStats $1"
      type: COUNTER
    - pattern: "com.facebook.presto.hive.metastore.thrift<>(.+AllTime.+): (.*)"
      name: "presto hivemetastorestats_$1"
      help: "Presto Hive: Metastore Stats $1"
      type: GAUGE
    - pattern: "com.facebook.presto.hive.metastore.thrift<>(.+TotalCount.*): (.*)"
      name: "presto hivemetastorestats_$1"
      help: "Presto Hive: Metastore Stats $1"
      type: COUNTER
    - pattern: "com.facebook.presto.execution<>(.+TotalCount.*): (.*)"
      name: "presto taskmanager_$1"
      help: "Presto: TaskManager Stats $1"
      type: COUNTER
    - pattern: "com.facebook.presto.failureDetector<>ActiveCount: (.*)"
      name: "presto heartbeatdetector_activecount"
      help: "Presto: HeartBeat Detector ActiveCount"
      type: GAUGE
    - pattern: "com.facebook.presto.execution<>(.+AllTime.+): (.*)"
      name: "presto querymanager_$1"
      help: "Presto: QueryManager $1"
      type: GAUGE
    - pattern: "com.facebook.presto.execution<>(.+TotalCount.*): (.*)"
      name: "presto querymanager_$1"
      help: "Presto: QueryManager $1"
      type: COUNTER
    - pattern: "com.facebook.presto.execution<>(.*): (.*)"
      name: "presto taskexecutor_$1"
      help: "Presto: TaskExecutor Stats $1"
      type: COUNTER
    - pattern: "com.facebook.presto.memory<>(.+): (.*)"
      name: "presto clustermemorymanager_$1"
      help: "Presto: ClusterMemoryManager $1"
      type: GAUGE
    - pattern: "com.facebook.presto.memory<>(.+): (.*)"
      name: "presto clustermemorypool_$1_$2"
      help: "Presto ClusterMemoryPool: $1 $2"
      type: GAUGE

下面shell请忽略（个人笔记）：

  shell: result=`cat {{install_dir}}/presto-server/etc/jvm.config | grep "jmx_prometheus_javaagent"`;if [[ -z "$result"  ]];then echo -e \-javaagent:{{install_dir}}/monitor/jmx_prometheus_javaagent-0.16.1.jar={{ prometheus_port }}:{{ install_dir }}/monitor/hadoop/presto-server.yaml >> {{install_dir}}/presto-server/etc/jvm.txt; fi