k8s学习笔记6-搭建监控kube-prometheus
k8s学习笔记6-搭建监控kube-prometheus
- 一.介绍
- 二.部署
-
- 1.源文件下载
- 2.镜像文件下载
- 3.部署
- 三.验证
- 四.登录grafana，并导入模板
-
- 1.主机详情的dashboard
- 2.pod,service,namespace,节点的dashboard
- 五.参考链接
一.介绍
应用部署到k8s上去了之后，我们想监控应用的pod，service以及节点的情况，目前k8s上提供的方案是kube-prometheus，这个方案整合了prometheus监控，报警，grafana仪表显示，感觉很好用
二.部署
1.源文件下载
#文件下载
git clone https://github.com/prometheus-operator/kube-prometheus.git
#进入到源文件夹,并且将相关部署文件拷贝到main文件夹下
cd kube-prometheus/manifests/
mkdir main
mv *.* main
2.镜像文件下载
1.由于网络原因，很多镜像无法下载，先查看当前有那些镜像需要下载
root@k8s-master1:~/kube-prometheus/manifests/main# grep image: ./*.*
./alertmanager-alertmanager.yaml: image: quay.io/prometheus/alertmanager:v0.24.0
./blackboxExporter-deployment.yaml: image: quay.io/prometheus/blackbox-exporter:v0.21.1
./blackboxExporter-deployment.yaml: image: jimmidyson/configmap-reload:v0.5.0
./blackboxExporter-deployment.yaml: image: quay.io/brancz/kube-rbac-proxy:v0.13.0
./grafana-deployment.yaml: image: grafana/grafana:9.0.4
./kubeStateMetrics-deployment.yaml: image: k8s.gcr.io/kube-state-metrics/kube-state-metrics:v2.5.0
./kubeStateMetrics-deployment.yaml: image: quay.io/brancz/kube-rbac-proxy:v0.13.0
./kubeStateMetrics-deployment.yaml: image: quay.io/brancz/kube-rbac-proxy:v0.13.0
./nodeExporter-daemonset.yaml: image: quay.io/prometheus/node-exporter:v1.3.1
./nodeExporter-daemonset.yaml: image: quay.io/brancz/kube-rbac-proxy:v0.13.0
./prometheus-prometheus.yaml: image: quay.io/prometheus/prometheus:v2.37.0
./prometheusAdapter-deployment.yaml: image: k8s.gcr.io/prometheus-adapter/prometheus-adapter:v0.9.1
./prometheusOperator-deployment.yaml: image: quay.io/prometheus-operator/prometheus-operator:v0.58.0
./prometheusOperator-deployment.yaml: image: quay.io/brancz/kube-rbac-proxy:v0.13.0
root@k8s-master1:~/kube-prometheus/manifests/main#
2.在docker hub上将这些镜像下载，tag，推送到harbor，修改文件（只举一个列子）
#下载镜像
docker pull quay.io/prometheus/alertmanager:v0.24.0
#tag镜像
docker tag quay.io/prometheus/alertmanager:v0.24.0 registry.harbor.com/library/alertmanager:v0.24.0
#将镜像push到已经搭建的harbor仓库
docker push registry.harbor.com/library/alertmanager:v0.24.0
#修改对应的部署文件，将镜像地址替换
sed -i s#quay.io/prometheus/alertmanager:v0.24.0#registry.harbor.com/library/alertmanager:v0.24.0#g alertmanager-alertmanager.yaml
3.部署
1.先部署setup里面的文件
root@k8s-master1:~/kube-prometheus/manifests/setup# kubectl create -f .
2.修改grafana-service.yaml文件，方便我们后续登录到grafana浏览器
piVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: grafana
app.kubernetes.io/name: grafana
app.kubernetes.io/part-of: kube-prometheus
app.kubernetes.io/version: 9.0.4
name: grafana
namespace: monitoring
spec:
type: NodePort #修改成nodeport模式
ports:
- name: http
port: 3000
targetPort: http
nodePort: 30180 #将端口设置成30180
selector:
app.kubernetes.io/component: grafana
app.kubernetes.io/name: grafana
app.kubernetes.io/part-of: kube-prometheus
3.部署main文件夹下的文件
root@k8s-master1:~/kube-prometheus/manifests/main# kubectl apply -f .
特别说明：
替换镜像的时候，对应的参数，也需要做对应的更改，不然会出现问题
root@k8s-master1:/root/kube-prometheus/manifests/main# grep v0.58.0 *.*
prometheusOperator-deployment.yaml:# - --prometheus-config-reloader=quay.io/prometheus-operator/prometheus-config-reloader:v0.58.0
prometheusOperator-deployment.yaml: - --prometheus-config-reloader=registry.harbor.com/library/prometheus-config-reloader:v0.58.0
prometheusOperator-deployment.yaml: image: registry.harbor.com/library/prometheus-operator:v0.58.0
root@k8s-master1:/root/kube-prometheus/manifests/main#
三.验证
验证是否部署成功
root@k8s-master1:~/kube-prometheus/manifests/main# kubectl get all -n monitoring
NAME READY STATUS RESTARTS AGE
pod/alertmanager-main-0 2/2 Running 0 4h21m
pod/alertmanager-main-1 2/2 Running 0 4h21m
pod/alertmanager-main-2 2/2 Running 0 4h21m
pod/blackbox-exporter-5fb779998c-8hgnz 3/3 Running 0 4h21m
pod/grafana-56f59b877b-7rhd7 1/1 Running 0 4h21m
pod/kube-state-metrics-c8bffc659-bjnw9 3/3 Running 0 4h21m
pod/node-exporter-55ghq 2/2 Running 0 4h21m
pod/node-exporter-mbg6s 2/2 Running 0 4h21m
pod/node-exporter-nnvqw 2/2 Running 0 4h21m
pod/prometheus-adapter-68f86c4dc6-2zpkk 1/1 Running 0 4h21m
pod/prometheus-adapter-68f86c4dc6-tj56z 1/1 Running 0 4h21m
pod/prometheus-k8s-0 2/2 Running 0 4h21m
pod/prometheus-k8s-1 2/2 Running 0 4h21m
pod/prometheus-operator-58974d75dd-24g42 2/2 Running 0 4h21m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/alertmanager-main ClusterIP 10.105.163.36 <none> 9093/TCP,8080/TCP 4h21m
service/alertmanager-operated ClusterIP None <none> 9093/TCP,9094/TCP,9094/UDP 4h21m
service/blackbox-exporter ClusterIP 10.106.6.24 <none> 9115/TCP,19115/TCP 4h21m
service/grafana NodePort 10.108.36.154 <none> 3000:30180/TCP 4h21m
service/kube-state-metrics ClusterIP None <none> 8443/TCP,9443/TCP 4h21m
service/node-exporter ClusterIP None <none> 9100/TCP 4h21m
service/prometheus-adapter ClusterIP 10.105.230.63 <none> 443/TCP 4h21m
service/prometheus-k8s NodePort 10.109.111.46 <none> 9090:30181/TCP,8080:32154/TCP 4h21m
service/prometheus-operated ClusterIP None <none> 9090/TCP 4h21m
service/prometheus-operator ClusterIP None <none> 8443/TCP 4h21m
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/node-exporter 3 3 3 3 3 kubernetes.io/os=linux 4h21m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/blackbox-exporter 1/1 1 1 4h21m
deployment.apps/grafana 1/1 1 1 4h21m
deployment.apps/kube-state-metrics 1/1 1 1 4h21m
deployment.apps/prometheus-adapter 2/2 2 2 4h21m
deployment.apps/prometheus-operator 1/1 1 1 4h21m
NAME DESIRED CURRENT READY AGE
replicaset.apps/blackbox-exporter-5fb779998c 1 1 1 4h21m
replicaset.apps/grafana-56f59b877b 1 1 1 4h21m
replicaset.apps/kube-state-metrics-c8bffc659 1 1 1 4h21m
replicaset.apps/prometheus-adapter-68f86c4dc6 2 2 2 4h21m
replicaset.apps/prometheus-operator-58974d75dd 1 1 1 4h21m
NAME READY AGE
statefulset.apps/alertmanager-main 3/3 4h21m
statefulset.apps/prometheus-k8s 2/2 4h21m
root@k8s-master1:~/kube-prometheus/manifests/main#
四.登录grafana，并导入模板
1.登录grafana，默认帐号是admin，默认密码也是admin，登录后需要修改密码
1.主机详情的dashboard
a.导入主机详情的dashboard-1
b.导入主机详情的dashboard-2，点击import
c.进入到主机的dashboard界面，上面有很多地方，可以根据自己的需求去设置
界面1：
界面2
上面两个图，仅仅是主机k8s-master1主机的详细监控数据，可以根据自己的需求进行切换切换
2.pod,service,namespace,节点的dashboard
a.导入数据数据模板13105
b.进入到导入界面，选择import进行导入，也可以改名啥的
c.点击导入后，就会进入到dashboard界面
界面1：
目前界面暂时没有数据，可以暂时看一下，我们的测试集群
如果根据需求，有需要更换不同的dashboard，可以登录网址https://grafana.com/grafana/dashboards进行选择
五.参考链接
1.kubernetes通过kube-prometheus安装高可用的监控告警（官方中文文档）
https://www.orchome.com/8193
2.grafana的dashboard界面
https://grafana.com/grafana/dashboards
3.Prometheus监控（kube-prometheus）
https://blog.csdn.net/qq_16240085/article/details/117460715