ECK(Elastic Cloud on Kubernetes)搭建及filebeat部署

ECK搭建

ECK提供了operator之后，搭建非常的容易。

下载crds.yaml和operator.yaml，然后执行

$ kubectl apply -f crds.yaml
$ kubectl apply -f operator.yaml

执行之后，会在kubernetes中创建一个statefulset，可以通过以下命令查看日志

kubectl -n  elastic-system logs -f statefulset.apps/elastic-operator

创建PV，用于Elasticsearch存储

在所有的服务器（elasticsearch-data运行的节点）上，创建一个local类型的pv

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-eyecool-lthpc
spec:
  capacity:
    storage: 10Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  storageClassName: local-storage
  local:
    path: /k8s-pv
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - lthpc

PV是全局对象，可以通过以下命令查看

$ kubectl get pv
NAME                    CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                                STORAGECLASS    REASON   AGE
pv-eyecool-eyecool097   10Gi       RWO            Delete           Bound    logging/elasticsearch-data-es-eyecool-es-default-2   local-storage            157m
pv-eyecool-gpu          10Gi       RWO            Delete           Bound    logging/elasticsearch-data-es-eyecool-es-default-1   local-storage            157m
pv-eyecool-lthpc        10Gi       RWO            Delete           Bound    logging/elasticsearch-data-es-eyecool-es-default-0   local-storage            157m

接下来创建Elasticsearch对象，并使用本地路径存储日志

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: es-eyecool
  namespace: logging
spec:
  version: 7.15.2
  http:
    tls:
      selfSignedCertificate:
        # 关闭https，使用http进行访问，否则需要提供证书，具体操作参考官方文档
        disabled: true
  nodeSets:
  - name: default
    count: 3
    config:
      node.store.allow_mmap: false
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data
      spec:
        accessModes: [ "ReadWriteOnce" ]
        # 选定local-storage为storage class，自动绑定上一步创建的pv
        storageClassName: "local-storage"
        resources:
          requests:
            storage: 1Gi

自动绑定的pvc如下：

$ kubectl -n logging get pvc
NAME                                         STATUS   VOLUME                  CAPACITY   ACCESS MODES   STORAGECLASS    AGE
elasticsearch-data-es-eyecool-es-default-0   Bound    pv-eyecool-lthpc        10Gi       RWO            local-storage   157m
elasticsearch-data-es-eyecool-es-default-1   Bound    pv-eyecool-gpu          10Gi       RWO            local-storage   157m
elasticsearch-data-es-eyecool-es-default-2   Bound    pv-eyecool-eyecool097   10Gi       RWO            local-storage   157m

创建kibana对象，并连接到创建的elasticsearch对象

apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: es-kibana
  namespace: logging
spec:
  version: 7.15.2
  count: 1
  elasticsearchRef:
    # 上一步创建的Elasticsearch集群
    name: es-eyecool
  http:
    tls:
      selfSignedCertificate:
        # 关闭https，使用http对外提供服务，否则需要配置证书【请参考官方文档】
        disabled: true

通过ingress暴露kibana服务

ingress.yaml的定义如下：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-kibana
  namespace: logging
spec:
  ingressClassName: nginx
  rules:
  - host: kibana.eyecool.cn
    http:
      paths:
      - backend:
          service:
            name: es-kibana-kb-http
            port:
              number: 5601
        path: /
        pathType: Prefix

通过以下命令进行创建

$ kubectl create -f ingress.yaml

注意：ingress nginx需要单独安装，请参考kubernetes ingress - (jianshu.com)

查看kibana的密码

$ kubectl -n logging  get secret es-eyecool-es-elastic-user -o=jsonpath='{.data.elastic}' | base64 --decode; echo

使用域名kibana.eyecool.cn进行访问即可