helm部署Filebeat + ELK
系统架构图：
1) 多个Filebeat在各个Node进行日志采集，然后上传至Logstash
2) 多个Logstash节点并行（负载均衡，不作为集群），对日志记录进行过滤处理，然后上传至Elasticsearch集群
3) 多个Elasticsearch构成集群服务，提供日志的索引和存储能力
4) Kibana负责对Elasticsearch中的日志数据进行检索、分析
1. Elasticsearch部署
官方chart地址：https://github.com/elastic/helm-charts/tree/master/elasticsearch
创建logs命名空间
kubectl create ns logs
添加elastic helm charts 仓库
helm repo add elastic https://helm.elastic.co
安装
helm install --name elasticsearch elastic/elasticsearch --namespace logs
参数说明
image: "docker.elastic.co/elasticsearch/elasticsearch"
imageTag: "7.2.0"
imagePullPolicy: "IfNotPresent"
podAnnotations: {}
esJavaOpts: "-Xmx1g -Xms1g"
resources:
requests:
cpu: "100m"
memory: "2Gi"
limits:
cpu: "1000m"
memory: "2Gi"
volumeClaimTemplate:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "nfs-client"
resources:
requests:
storage: 50Gi
2. Filebeat部署
官方chart地址：https://github.com/elastic/helm-charts/tree/master/filebeat
Add the elastic helm charts repo
helm repo add elastic https://helm.elastic.co
Install it
helm install --name filebeat elastic/filebeat --namespace logs
参数说明：
image: "docker.elastic.co/beats/filebeat"
imageTag: "7.2.0"
imagePullPolicy: "IfNotPresent"
resources:
requests:
cpu: "100m"
memory: "100Mi"
limits:
cpu: "1000m"
memory: "200Mi"
那么问题来了，filebeat默认收集宿主机上docker的日志路径：/var/lib/docker/containers。如果我们修改了docker的安装路径要怎么收集呢，很简单修改chart里的DaemonSet文件里边的hostPath参数：
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers #改为docker安装路径
对java程序的报错异常log实现多行合并，用multiline定义正则来匹配。
filebeatConfig:
filebeat.yml: |
filebeat.inputs:
- type: docker
containers.ids:
- '*'
multiline.pattern: '^[0-9]'
multiline.negate: true
multiline.match: after
processors:
- add_kubernetes_metadata:
in_cluster: true
output.elasticsearch:
hosts: '${ELASTICSEARCH_HOSTS:elasticsearch-master:9200}'
3. Kibana部署
官方chart地址：https://github.com/elastic/helm-charts/tree/master/kibana
Add the elastic helm charts repo
helm repo add elastic https://helm.elastic.co
Install it
helm install --name kibana elastic/kibana --namespace logs
参数说明：
elasticsearchHosts: "http://elasticsearch-master:9200"
replicas: 1
image: "docker.elastic.co/kibana/kibana"
imageTag: "7.2.0"
imagePullPolicy: "IfNotPresent"
resources:
requests:
cpu: "100m"
memory: "500m"
limits:
cpu: "1000m"
memory: "1Gi"
4. Logstash部署
官方chart地址：https://github.com/helm/charts/tree/master/stable/logstash
安装
$ helm install --name logstash stable/logstash --namespace logs
参数说明：
image: repository: docker.elastic.co/logstash/logstash-oss tag: 7.2.0 pullPolicy: IfNotPresent persistence: enabled: true storageClass: "nfs-client" accessMode: ReadWriteOnce size: 2Gi
5. Elastalert部署
官方chart地址：https://github.com/helm/charts/tree/master/stable/elastalert
安装
helm install -n elastalert ./elastalert --namespace logs
效果图：
