在k8s 1.26.6上部署ES集群
一、k8s集群架构：
IP 角色，左边是ip，右边是hostname master1 是192.168.1.3 的hostname
192.168.1.3 master1
192.168.1.4 master2
192.168.1.5 master3
192.168.1.6 node1
192.168.1.7 node2
二、部署ES集群
1、配置storageclass，用于动态创建pvc，并自动绑定pv
cat sc.yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
执行一下：
kubectl apply -f sc.yaml
2、创建名称空间
kubectl create ns elasticsearch
3、创建PV
cat pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: local-storage-pv-1
namespace: elasticsearch
labels:
name: local-storage-pv-1
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: local-storage
local:
path: /data/es
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- master1
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: local-storage-pv-2
namespace: elasticsearch
labels:
name: local-storage-pv-2
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: local-storage
local:
path: /data/es
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- master2
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: local-storage-pv-3
namespace: elasticsearch
labels:
name: local-storage-pv-3
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: local-storage
local:
path: /data/es
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- master3
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: local-storage-pv-4
namespace: elasticsearch
labels:
name: local-storage-pv-4
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: local-storage
local:
path: /data/es
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- node1
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: local-storage-pv-5
namespace: elasticsearch
labels:
name: local-storage-pv-5
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: local-storage
local:
path: /data/es
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- node2
一共是5个PV，每个都通过nodeSelectorTerms跟k8s节点绑定。
执行一下：
kubectl apply -f pv.yaml
3、创建StatefulSet，ES属于数据库类型的应用，此类应用适合StatefulSet类型
cat sts.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: es7-cluster
namespace: elasticsearch
spec:
serviceName: elasticsearch7
replicas: 5
selector:
matchLabels:
app: elasticsearch7
template:
metadata:
labels:
app: elasticsearch7
spec:
containers:
- name: elasticsearch7
image: elasticsearch:7.16.2
resources:
limits:
cpu: 1000m
requests:
cpu: 100m
ports:
- containerPort: 9200
name: rest
protocol: TCP
- containerPort: 9300
name: inter-node
protocol: TCP
volumeMounts:
- name: data
mountPath: /usr/share/elasticsearch/data
env:
- name: cluster.name
value: k8s-logs
- name: node.name
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: discovery.zen.minimum_master_nodes
value: "2"
- name: discovery.seed_hosts
value: "es7-cluster-0.elasticsearch7,es7-cluster-1.elasticsearch7,es7-cluster-2.elasticsearch7"
- name: cluster.initial_master_nodes
value: "es7-cluster-0,es7-cluster-1,es7-cluster-2"
- name: ES_JAVA_OPTS
value: "-Xms1g -Xmx1g"
initContainers:
- name: fix-permissions
image: busybox
command: ["sh", "-c", "chown -R 1000:1000 /usr/share/elasticsearch/data"]
securityContext:
privileged: true
volumeMounts:
- name: data
mountPath: /usr/share/elasticsearch/data
- name: increase-vm-max-map
image: busybox
command: ["sysctl", "-w", "vm.max_map_count=262144"]
securityContext:
privileged: true
- name: increase-fd-ulimit
image: busybox
command: ["sh", "-c", "ulimit -n 65536"]
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "local-storage"
resources:
requests:
storage: 1Gi
该ES集群通过volumeClaimTemplates来关联storageClass，并自动绑定相应的PV。
执行一下：
kubectl apply -f sts.yaml
1
4、创建NodePort类型的Service来蒋ES集群暴漏出去
cat svc.yaml
apiVersion: v1
kind: Service
metadata:
name: elasticsearch7
namespace: elasticsearch
spec:
selector:
app: elasticsearch7
type: NodePort
ports:
- port: 9200
nodePort: 30002
targetPort: 9200
执行一下：
kubectl apply -f svc.yaml
以上就创建完了，我们来看一下上面创建的资源状态：
（1）storageclass
kubectl get scNAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
local-storage kubernetes.io/no-provisioner Delete WaitForFirstConsumer false 21h
（2）PV
kubectl get pvNAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
local-storage-pv-1 1Gi RWO Retain Bound elasticsearch/data-es7-cluster-1 local-storage 21h
local-storage-pv-2 1Gi RWO Retain Bound elasticsearch/data-es7-cluster-2 local-storage 21h
local-storage-pv-3 1Gi RWO Retain Bound elasticsearch/data-es7-cluster-0 local-storage 21h
local-storage-pv-4 1Gi RWO Retain Bound elasticsearch/data-es7-cluster-4 local-storage 19m
local-storage-pv-5 1Gi RWO Retain Bound elasticsearch/data-es7-cluster-3 local-storage 19mkubectl get pvc -n elasticsearchNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
data-es7-cluster-0 Bound local-storage-pv-3 1Gi RWO local-storage 21h
data-es7-cluster-1 Bound local-storage-pv-1 1Gi RWO local-storage 21h
data-es7-cluster-2 Bound local-storage-pv-2 1Gi RWO local-storage 21h
data-es7-cluster-3 Bound local-storage-pv-5 1Gi RWO local-storage 20m
data-es7-cluster-4 Bound local-storage-pv-4 1Gi RWO local-storage 19m
（3）StatefulSet
kubectl get statefulset -n elasticsearch
NAME READY AGE
es7-cluster 5/5 21h
kubectl get pod -n elasticsearch
NAME READY STATUS RESTARTS AGE
es7-cluster-0 1/1 Running 1 21h
es7-cluster-1 1/1 Running 1 21h
es7-cluster-2 1/1 Running 1 21h
es7-cluster-3 1/1 Running 0 19m
es7-cluster-4 1/1 Running 0 18m
（4）Service
kubectl get svc -n elasticsearch
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
elasticsearch7 NodePort 10.97.196.176 9200:30002/TCP 21h
（5）通过restful接口查看ES集群状态
curl 192.168.1.3:30002/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
10.244.136.5 60 39 1 0.19 0.31 0.36 cdfhilmrstw - es7-cluster-0
10.244.104.1 45 41 1 0.98 0.44 0.25 cdfhilmrstw - es7-cluster-3
10.244.180.5 40 40 1 0.28 0.32 0.32 cdfhilmrstw - es7-cluster-2
10.244.166.141 63 41 1 0.06 0.14 0.14 cdfhilmrstw - es7-cluster-4
10.244.137.68 44 41 1 0.29 0.45 0.59 cdfhilmrstw * es7-cluster-1
通过网页访问也是正常的：
5、部署kibana
kibana我们使用Deployment类型来部署，而它的Service我们使用NodePort，完整的代码如下：
vi kibana.yaml
apiVersion: v1
kind: Service
metadata:
name: kibana
namespace: elasticsearch
labels:
app: kibana
spec:
ports:
- port: 5601
targetPort: 5601
nodePort: 30001
type: NodePort
selector:
app: kibana
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: kibana
namespace: elasticsearch
labels:
app: kibana
spec:
selector:
matchLabels:
app: kibana
template:
metadata:
labels:
app: kibana
spec:
nodeSelector:
node: node2
containers:
- name: kibana
image: kibana:7.16.2
resources:
limits:
cpu: 1000m
requests:
cpu: 1000m
env:
- name: ELASTICSEARCH_HOSTS
value: http://elasticsearch7:9200
- name: SERVER_PUBLICBASEURL
value: "0.0.0.0:5601"
- name: I18N.LOCALE
value: zh-CN
ports:
- containerPort: 5601
上面的Deployment部分，我们通过nodeSelector指定调度到了node2节点，前提是给node2节点加了个node=node2这样一个标签。
执行一下：
kubectl apply -f kibana.yaml
再查看一下状态：
kubectl get all -n elasticsearch
NAME READY STATUS RESTARTS AGE
pod/es7-cluster-0 1/1 Running 0 47m
pod/es7-cluster-1 1/1 Running 0 46m
pod/es7-cluster-2 1/1 Running 0 46m
pod/es7-cluster-3 1/1 Running 0 43m
pod/es7-cluster-4 1/1 Running 0 42m
pod/kibana-768595479f-mhw9q 1/1 Running 0 13m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/elasticsearch7 NodePort 10.108.115.19 9200:30002/TCP 47m
service/kibana NodePort 10.111.66.207 5601:30001/TCP 13m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/kibana 1/1 1 1 13m
NAME DESIRED CURRENT READY AGE
replicaset.apps/kibana-768595479f 1 1 1 13m
NAME READY AGE
statefulset.apps/es7-cluster 5/5 47m
看到kibana的Service和Pod都在运行中了，通过页面验证一下kibana能不能访问：