Relabeling(重定义标签),是在拉取(scraping)阶段前,修改target和它的labels;
在每个scrape_configs
可以定义多个重定义标签的步骤;
默认的, target的job
标签设置为配置文件里的job_name
的值;
__address__
设置为配置里的targets
的值;
而instance
标签的值,是重定义标签操作之后__address__
的值
__scheme__和__metrics_path__
标签的值,也是从配置里取值的;
__param_
标签的值,是传给URL的查询参数
的值;
以__meta_
开头的标签,也可以用于重定义标签;
重定义标签完成后,__
开头的标签会被删除;
重定义标签阶段,如果要临时存储值用于下一阶段的处理,使用__tmp
开头的标签名,这种标签不会被Prometheus使用;
# 从已有的标签选择值的源标签组;多个标签,使用separator分隔;
# regex匹配源标签里的值
# 动作为replace, keep, 和 drop
[ source_labels: '[' [, ...] ']' ]
# 多个源标签的分隔符;
[ separator: | default = ; ]
# replace动作必需,regex匹配到的值,要替换的目标标签;
[ target_label: ]
# 正则匹配源标签的值
[ regex: | default = (.*) ]
# 源标签值取hash的模块;
[ modulus: ]
# 要替换的匹配分组号
[ replacement: | default = $1 ]
# 基于正则匹配的动作
[ action: | default = replace ]
正则匹配是到结尾的, 可以使用.*
匹配更多内容
replace: 正则匹配源标签的值用来替换目标标签;如果有replacement,使用replacement替换目标标签;
keep: 如果正则没有匹配到源标签,删除targets
drop: 正则匹配到源标签,删除targets
hashmod: 设置目标标签值为源标签值的hash值
labelmap: 正则匹配所有标签名; 将匹配的标签的值复制到由replacement提供的标签名
labeldrop: 正则匹配所有标签名;匹配则移除标签;
labelkeep: 正则匹配所有标签名;不匹配的标签会被移除;
使用默认的prometheus target和监控指标
默认配置文件
cat prometheus.yml
------
global:
scrape_interval: 15s
evaluation_interval: 15s
alerting:
alertmanagers:
- static_configs:
- targets:
rule_files:
scrape_configs:
- job_name: 'prometheus'
static_configs:
- targets: ['localhost:9090']
======
查看target
curl 'http://localhost:9090/api/v1/targets?state=active'
------
{
"status": "success",
"data": {
"activeTargets": [{
"discoveredLabels": { // 这块只能作为源标签
"__address__": "localhost:9090",
"__metrics_path__": "/metrics",
"__scheme__": "http",
"job": "prometheus"
},
"labels": {
"instance": "localhost:9090",
"job": "prometheus"
},
"scrapePool": "prometheus",
"scrapeUrl": "http://localhost:9090/metrics",
"lastError": "",
"lastScrape": "2020-04-25T08:42:19.254191755-04:00",
"lastScrapeDuration": 0.012091634,
"health": "up"
}],
"droppedTargets": []
}
}
======
relabel_config主要是处理target的labels字段
# 检查配置
./promtool check config prometheus.yml
# 重载配置;需要以--web.enable-lifecycle启动promethues
curl -X POST http://localhost:9090/-/reload
# 查看使用的配置
curl http://localhost:9090/api/v1/status/config
添加2个自定义标签,用于后续的演示
vi prometheus.yml
------
scrape_configs:
- job_name: 'prometheus'
static_configs:
- targets: ['localhost:9090']
labels:
userLabel1: value1
userLabel2: value2
======
重载配置后查询
curl 'http://localhost:9090/api/v1/targets?state=active'
------
{
"status": "success",
"data": {
"activeTargets": [{
"discoveredLabels": {
"__address__": "localhost:9090",
"__metrics_path__": "/metrics",
"__scheme__": "http",
"job": "prometheus",
"userLabel1": "value1", //新增
"userLabel2": "value2" //新增
},
"labels": {
"instance": "localhost:9090",
"job": "prometheus",
"userLabel1": "value1", //新增
"userLabel2": "value2" //新增
},
...
}],
"droppedTargets": []
}
}
用源标签的值,替换目标标签的值
vi prometheus.yml
------
scrape_configs:
...
relabel_configs:
- source_labels: [userLabel1]
target_label: userLabel2
#默认action 是 'replace'
======
重载配置后查询
curl 'http://localhost:9090/api/v1/targets?state=active'
------
"discoveredLabels": {
"__address__": "localhost:9090",
"__metrics_path__": "/metrics",
"__scheme__": "http",
"job": "prometheus",
"userLabel1": "value1", //不变
"userLabel2": "value2" //不变
},
"labels": {
"instance": "localhost:9090",
"job": "prometheus",
"userLabel1": "value1",
"userLabel2": "value1" //用userLabel1的值替换了userLabel2
},
======
relabel_configs是在拉取(scraping)前,修改target和它的labels
用userLabel1的部分值替换userLabel2
vi prometheus.yml
------
scrape_configs:
...
relabel_configs:
- source_labels: [userLabel1]
regex: 'value([0-9]+)'
target_label: userLabel2
replacement: '$1'
action: replace
======
重载配置后查询
curl 'http://localhost:9090/api/v1/targets?state=active'
------
"labels": {
"instance": "localhost:9090",
"job": "prometheus",
"userLabel1": "value1",
"userLabel2": "1"
},
======
vi prometheus.yml
------
scrape_configs:
...
relabel_configs:
- regex: userLabel1
action: labeldrop
======
重载配置后查询
curl 'http://localhost:9090/api/v1/targets?state=active'
------
{
"status": "success",
"data": {
"activeTargets": [{
"discoveredLabels": { //这部分只能做为源标签
"__address__": "localhost:9090",
"__metrics_path__": "/metrics",
"__scheme__": "http",
"job": "prometheus",
"userLabel1": "value1",
"userLabel2": "value2"
},
"labels": {
"instance": "localhost:9090",
"job": "prometheus",
"userLabel2": "value2" //删除了userLabel1
},
...
}
}
======
vi prometheus.yml
------
scrape_configs:
...
relabel_configs:
- source_labels: [userLabel1]
action: drop
======
重载配置后查询
curl 'http://localhost:9090/api/v1/targets?state=active'
------
{
"status": "success",
"data": {
"activeTargets": [],
"droppedTargets": []
}
}
======
vi prometheus.yml
------
scrape_configs:
...
relabel_configs:
- regex: user(.*)1
action: labelmap
======
重载配置后查询
curl 'http://localhost:9090/api/v1/targets?state=active'
------
"labels": {
"Label": "value1", //新生成的标签
"instance": "localhost:9090",
"job": "prometheus",
"userLabel1": "value1",
"userLabel2": "value2"
},
======
参考链接
官方配置示例
官方文档