Kubernetes-v1.17.4部署kube-prometheus-v0.5.0

0.环境准备

通过 Kubernetes 搭建 kube-prometheus时，Prometheus、Grafana、node_exporter、Alertmanager等组件会自动安装，版本如下：

IP	Docker	Rancher	Kubernetes	kube-prometheus	Prometheus(30090) (Auto Install)	Grafana(30000) (Auto Install)	node_exporter (Auto Install)	alertmanager(30093) (Auto Install)
10.180.249.245 manager.rancher	v19.03.8	v2.3.6	v1.17.4 master	v0.5.0	v2.15.2	v6.6.0	v0.18.1	v0.20.0
10.180.249.246 master.rancher	v19.03.8	-	v1.17.4 worker	-	v2.15.2	v6.6.0	v0.18.1	v0.20.0
10.180.249.247 worker.rancher	v19.03.8	-	v1.17.4 worker	-	v2.15.2	v6.6.0	v0.18.1	v0.20.0

Centos7.4-Rancher2.3.6-Docker19.03环境Ansible部署
Docker-v19.03.8-Rancher-v2.3.6部署Kubernetes-v1.17.4集群

kube-prometheus-v0.5.0下载：
https://github.com/coreos/kube-prometheus/releases

1.解压部署

在manager.rancher节点执行
tar -zxvf kube-prometheus-0.5.0.tar.gz -C /opt/

cd /opt/kube-prometheus-0.5.0
kubectl create -f manifests/setup
until kubectl get servicemonitors --all-namespaces ; do date; sleep 1; echo “”; done
kubectl create -f manifests/

上面步骤可以合并一个步骤(合并步骤会存在一次执行不完全成功的情况，建议使用上面分步执行)
kubectl create -f manifests/setup -f manifests/

如果重复执行上述步骤，需要把上面步骤中创建删除
kubectl delete --ignore-not-found=true -f manifests/setup -f manifests/

2.等待部署完成

查看命名空间 monitoring 下 pod 部署情况
kubectl get pod -n monitoring
三个集群节点相同

3.更改访问模式

更改访问模式时，需要保证第二步部署完成。

3.1 Prometheus

修改配置文件
kubectl edit svc/prometheus-k8s -n monitoring
nodePort: 30090
ClusterIP => NodePort

3.2 Grafana

修改配置文件
kubectl edit svc/grafana -n monitoring
nodePort: 30000
ClusterIP => NodePort
使用admin/admin登录，修改密码


Grafana配置Prometheus

3.3 Alertmanager

修改配置文件
kubectl edit svc/alertmanager-main -n monitoring
nodePort: 30093
ClusterIP => NodePort

4.Prometheus问题

展开Status菜单，查看targets，可以看到只有图中两个监控任务没有对应的目标，这和serviceMonitor资源对象有关

cd /opt/kube-prometheus-0.5.0/manifests
vim prometheus-kubeControllerManagerService.yaml

apiVersion: v1
kind: Service
metadata:
  namespace: kube-system
  name: kube-controller-manager
  labels:
    k8s-app: kube-controller-manager
spec:
  selector:
    component: kube-controller-manager
  type: ClusterIP
  clusterIP: None
  ports:
  - name: http-metrics
    port: 10252
    targetPort: 10252
    protocol: TCP
---
apiVersion: v1
kind: Endpoints
metadata:
  labels:
    k8s-app: kube-controller-manager
  name: kube-controller-manager
  namespace: kube-system
subsets:
- addresses:
  - ip: 10.180.249.245
  ports:
  - name: http-metrics
    port: 10252
    protocol: TCP

kubectl apply -f prometheus-kubeControllerManagerService.yaml

vim prometheus-kubeSchedulerService.yaml

apiVersion: v1
kind: Service
metadata:
  namespace: kube-system
  name: kube-scheduler
  labels:
    k8s-app: kube-scheduler
spec:
  type: ClusterIP
  clusterIP: None
  ports:
  - name: port
    port: 10251
    protocol: TCP
---
apiVersion: v1
kind: Endpoints
metadata:
  labels:
    k8s-app: kube-scheduler
  name: kube-scheduler
  namespace: kube-system
subsets:
- addresses:
  - ip: 10.180.249.245
  ports:
  - name: http-metrics
    port: 10251
    protocol: TCP

kubectl apply -f prometheus-kubeSchedulerService.yaml

参考：
https://github.com/coreos/kube-prometheus