离线环境安装docker,k8s,prometheus-operator 之prometheus-operator安装

篇幅五. prometheus-operator安装

1. 下载包

1.1.1.1(可连公网服务器)

git clone -b v0.10.0 https://github.com/prometheus-operator/kube-prometheus.git
scp -r kube-prometheus [email protected]:/home/kubernetes/k8s/app

2. 配置修改

ll kube-prometheus| awk -F' ' '{print $9}'

build.sh
CHANGELOG.md
code-of-conduct.md
CONTRIBUTING.md
DCO
developer-workspace
docs
example.jsonnet
examples
experimental
go.mod
go.sum
jsonnet
jsonnetfile.json
jsonnetfile.lock.json
kustomization.yaml
LICENSE
Makefile
manifests
README.md
RELEASE.md
scripts
sync-to-internal-registry.jsonnet
tests

我们需要修改的是 manifests 下所有 .yaml 里的 image 字段
把公网地址换成内网私有仓库地址

./alertmanager-alertmanager.yaml:  image: 1.1.1.1:5000/alertmanager:v0.23.0
./blackboxExporter-deployment.yaml:        image: 1.1.1.1:5000/blackbox-exporter:v0.19.0
./blackboxExporter-deployment.yaml:        image: 1.1.1.1:5000/configmap-reload:v0.5.0
./blackboxExporter-deployment.yaml:        image: 1.1.1.1:5000/kube-rbac-proxy:v0.11.0
./grafana-deployment.yaml:        image: 1.1.1.1:5000/grafana:8.3.3
./kubeStateMetrics-deployment.yaml:        image: 1.1.1.1:5000/kube-state-metrics:2.3.0
./kubeStateMetrics-deployment.yaml:        image: 1.1.1.1:5000/kube-rbac-proxy:v0.11.0
./kubeStateMetrics-deployment.yaml:        image: 1.1.1.1:5000/kube-rbac-proxy:v0.11.0
./nodeExporter-daemonset.yaml:        image: 1.1.1.1:5000/node-exporter:v1.3.1
./nodeExporter-daemonset.yaml:        image: 1.1.1.1:5000/kube-rbac-proxy:v0.11.0
./prometheusAdapter-deployment.yaml:        image: 1.1.1.1:5000/prometheus-adapter:v0.9.1
./prometheusOperator-deployment.yaml:        - --prometheus-config-reloader=1.1.1.1:5000/prometheus-config-reloader:v0.53.1
./prometheusOperator-deployment.yaml:        image: 1.1.1.1:5000/prometheus-operator:v0.53.1
./prometheusOperator-deployment.yaml:        image: 1.1.1.1:5000/kube-rbac-proxy:v0.11.0
./prometheus-prometheus.yaml:  image: 1.1.1.1:5000/prometheus:v2.32.1

3. 安装

cd kube-prometheus
kubectl apply --server-side -f manifests/setup
kubectl create -f manifests/

此步骤执行完大概率有部分pod 启动不了，原因是 manifests 下多了一些脚本生成的 yaml文件，
此些yaml文件里image 默认还是公网地址，根据未启动pod 找到对应yaml文件修改正确即可
修改正确后再次 apply

kubectl apply -f manifests/

4. 修改prometheus以及alertmanage的Service为NodePort

4. 1 cat manifests/prometheus-service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: prometheus
    app.kubernetes.io/instance: k8s
    app.kubernetes.io/name: prometheus
    app.kubernetes.io/part-of: kube-prometheus
    app.kubernetes.io/version: 2.32.1
  name: prometheus-k8s
  namespace: monitoring
spec:
  type: NodePort
  ports:
  - name: web
    port: 9090
    targetPort: web
    nodePort: 9090
  - name: reloader-web
    port: 8080
    targetPort: reloader-web
  selector:
    app.kubernetes.io/component: prometheus
    app.kubernetes.io/instance: k8s
    app.kubernetes.io/name: prometheus
    app.kubernetes.io/part-of: kube-prometheus
  sessionAffinity: ClientIP

4. 2 cat manifests/alertmanager-service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: alert-router
    app.kubernetes.io/instance: main
    app.kubernetes.io/name: alertmanager
    app.kubernetes.io/part-of: kube-prometheus
    app.kubernetes.io/version: 0.23.0
  name: alertmanager-main
  namespace: monitoring
spec:
  type: NodePort
  ports:
  - name: web
    port: 9093
    targetPort: web
    nodePort: 9093
  - name: reloader-web
    port: 8080
    targetPort: reloader-web
  selector:
    app.kubernetes.io/component: alert-router
    app.kubernetes.io/instance: main
    app.kubernetes.io/name: alertmanager
    app.kubernetes.io/part-of: kube-prometheus
  sessionAffinity: ClientIP

4. 3 再次apply

kubectl apply -f manifests/prometheus-service.yaml
kubectl apply -f manifests/alertmanager-service.yaml

这样web端即可访问 prometheus 以及 alertmanager

注: 此处映射 9090 9093端口也是需要开启的，k8s默认好像只能用30000以上端口
修改配置文件/etc/kubernetes/manifests/kube-apiserver.yaml

#添加  - --service-node-port-range=1-65535 此行
    - --service-cluster-ip-range=10.96.0.0/12
    - --service-node-port-range=1-65535
    - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt

重启 kubectl

systemctl daemon-reload
systemctl restart kubelet