EFK介绍
EFK，全称Elasticsearch Fluentd Kibana ，是kubernetes中比较常用的日志收集方案，也是官方比较推荐的方案。
通过EFK，可以把集群的所有日志收集到Elasticsearch中，然后可以对日志做分析。一般用于故障排查，数据分析等。。。
数据流示意图
官方项目
https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch小技巧，如果只希望下载github项目的某一个目录，可以使用svn，这里就只下载fluentd-elasticsearch目录，
例如需要下载的子目录为：
https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch
将/tree/master/换成trunk,然后使用svn下载即可
svn co https://github.com/kubernetes/kubernetes/trunk/cluster/addons/fluentd-elasticsearch这里因为是学习，一步步安装，感兴趣的可以看官方项目
部署Elasticsearch
存储服务是基础，需要先部署，其他两个服务运行的时候需要连接es。
1、编写efk-es-statefulset.yaml
---
apiVersion: v1
kind: Namespace
metadata:
name: efk
---
kind: Service
apiVersion: v1
metadata:
name: elasticsearch-logging
namespace: efk
labels:
app: elasticsearch-logging
spec:
selector:
app: elasticsearch-logging
clusterIP: None
ports:
- port: 9200
name: rest
- port: 9300
name: inter
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: elasticsearch-logging
namespace: efk
spec:
serviceName: elasticsearch-logging
replicas: 3
selector:
matchLabels:
app: elasticsearch-logging
template:
metadata:
labels:
app: elasticsearch-logging
spec:
initContainers:
- name: increase-vm-max-map
image: busybox
command: ["sysctl", "-w", "vm.max_map_count=262144"]
securityContext:
privileged: true
- name: increase-fd-ulimit
image: busybox
command: ["sh", "-c", "ulimit -n 65536"]
securityContext:
privileged: true
containers:
- name: elasticsearch-logging
image: docker.elastic.co/elasticsearch/elasticsearch:7.9.1
ports:
- name: rest
containerPort: 9200
- name: inter
containerPort: 9300
resources:
limits:
cpu: 1000m
requests:
cpu: 1000m
volumeMounts:
- name: elasticsearch-logging
mountPath: /usr/share/elasticsearch/data
env:
- name: cluster.name
value: k8s-logs
- name: node.name
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: cluster.initial_master_nodes
value: "elasticsearch-logging-0,elasticsearch-logging-1,elasticsearch-logging-2"
- name: discovery.zen.minimum_master_nodes
value: "2"
- name: discovery.seed_hosts
value: "elasticsearch-logging"
- name: ES_JAVA_OPTS
value: "-Xms512m -Xmx512m"
- name: network.host
value: "0.0.0.0"
volumes:
- name: elasticsearch-logging
emptyDir: {}2、执行部署命令
这里需要注意，如果长时间下载不下来镜像，可以自行先将镜像下载，要不然可能会一直不成功
本文把服务都部署到命名空间:efk
[root@k8s-master001 EFK]# kubectl apply -f efk-es-statefulset.yaml
[root@k8s-node001 EFK]# kubectl get po -n efk
NAME READY STATUS RESTARTS AGE
elasticsearch-logging-0 1/1 Running 0 10m
elasticsearch-logging-1 1/1 Running 0 10m
elasticsearch-logging-2 1/1 Running 0 9m42s3、验证es是否正常运行
sh-4.2# curl http://localhost:9200/_cluster/state?pretty
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0{
"cluster_name" : "k8s-logs",
"cluster_uuid" : "OLzzi6sbSZG11bqBFM9z5Q",
"version" : 38,
"state_uuid" : "uFa1_QKgRAK_NJ33SArGDw",
"master_node" : "XiShXS0DSGmx0Dxp1r9vEw",
"blocks" : { },
"nodes" : {
"XN-vHccLRkaEgr9Q1cctNA" : {
"name" : "elasticsearch-logging-2",
"ephemeral_id" : "WBEY2tGNRzmc3cBDJAEP9Q",
"transport_address" : "100.108.163.2:9300",
"attributes" : {
"ml.machine_memory" : "16630661120",
"ml.max_open_jobs" : "20",
"xpack.installed" : "true",
"transform.node" : "true"
}
},
.................以上, elasticsearch就部署好了，接下来部署kibana
部署kibana
1、编写efk-kibana.yaml
apiVersion: v1
kind: Service
metadata:
name: kibana-logging
namespace: efk
labels:
app: kibana-logging
spec:
ports:
- port: 5601
type: NodePort
selector:
app: kibana-logging
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: kibana-logging
namespace: efk
labels:
app: kibana-logging
spec:
selector:
matchLabels:
app: kibana-logging
template:
metadata:
labels:
app: kibana-logging
spec:
containers:
- name: kibana-logging
image: docker.elastic.co/kibana/kibana:7.9.1
resources:
limits:
cpu: 1000m
requests:
cpu: 1000m
env:
- name: ELASTICSEARCH_HOSTS
value: http://elasticsearch-logging:9200
ports:
- containerPort: 56012、执行部署命令
[root@k8s-node001 EFK]# kubectl apply -f efk-kibana.yaml
service/kibana-logging created
deployment.apps/kibana-logging created
[root@k8s-node001 EFK]# kubectl get po,svc -n efk
NAME READY STATUS RESTARTS AGE
kibana-logging-6b5f984c44-7ljjn 1/1 Running 0 8m16s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/elasticsearch-logging ClusterIP None 9200/TCP,9300/TCP 28m
service/kibana-logging NodePort 10.105.208.90 5601:32352/TCP 13m 3、验证kibana
服务以及通过NodePort暴露，通过IP+32352，可以访问到kibana web界面，如图所示
下一步，我们来部署日志收集客户端Fluentd
部署Fluentd
1、使用configmap创建fluentd配置文件
配置比较长，可以查看链接，这里就不贴出来了
https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/fluentd-elasticsearch/fluentd-es-configmap.yaml2、执行部署
[root@k8s-node001 EFK]# kubectl appply -f fluentd-es-configmap.yaml3、创建fluentd-es-ds.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: fluentd-es
namespace: efk
labels:
k8s-app: fluentd-es
addonmanager.kubernetes.io/mode: Reconcile
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: fluentd-es
labels:
k8s-app: fluentd-es
addonmanager.kubernetes.io/mode: Reconcile
rules:
- apiGroups:
- ""
resources:
- "namespaces"
- "pods"
verbs:
- "get"
- "watch"
- "list"
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: fluentd-es
labels:
k8s-app: fluentd-es
addonmanager.kubernetes.io/mode: Reconcile
subjects:
- kind: ServiceAccount
name: fluentd-es
namespace: efk
apiGroup: ""
roleRef:
kind: ClusterRole
name: fluentd-es
apiGroup: ""
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: fluentd-es-v3.0.2
namespace: efk
labels:
k8s-app: fluentd-es
version: v3.0.2
addonmanager.kubernetes.io/mode: Reconcile
spec:
selector:
matchLabels:
k8s-app: fluentd-es
version: v3.0.2
template:
metadata:
labels:
k8s-app: fluentd-es
version: v3.0.2
spec:
securityContext:
seccompProfile:
type: RuntimeDefault
priorityClassName: system-node-critical
serviceAccountName: fluentd-es
containers:
- name: fluentd-es
image: registry.cn-qingdao.aliyuncs.com/up2cloud/fluentd:v3.0.2
env:
- name: FLUENTD_ARGS
value: --no-supervisor -q
resources:
limits:
memory: 500Mi
requests:
cpu: 100m
memory: 200Mi
volumeMounts:
- name: varlog
mountPath: /var/log
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: config-volume
mountPath: /etc/fluent/config.d
ports:
- containerPort: 24231
name: prometheus
protocol: TCP
livenessProbe:
tcpSocket:
port: prometheus
initialDelaySeconds: 5
timeoutSeconds: 10
readinessProbe:
tcpSocket:
port: prometheus
initialDelaySeconds: 5
timeoutSeconds: 10
terminationGracePeriodSeconds: 30
volumes:
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: config-volume
configMap:
name: fluentd-es-config-v0.2.04、执行部署
[root@k8s-node001 EFK]# kubectl apply -f fluentd-es-ds.yaml5、查看部署结果
[root@k8s-node001 EFK]# kubectl get po -n efk
NAME READY STATUS RESTARTS AGE
elasticsearch-logging-0 1/1 Running 0 3h34m
elasticsearch-logging-1 1/1 Running 0 3h33m
elasticsearch-logging-2 1/1 Running 0 3h33m
fluentd-es-v3.0.2-24lbr 1/1 Running 0 26m
fluentd-es-v3.0.2-5qcsv 1/1 Running 0 26m
fluentd-es-v3.0.2-gnp58 1/1 Running 0 26m
fluentd-es-v3.0.2-gtx4s 1/1 Running 0 26m
fluentd-es-v3.0.2-mxz9t 1/1 Running 0 26m
kibana-logging-6b5f984c44-7ljjn 1/1 Running 0 3h19m从输出信息可以看到，整套日志收集系统已经全部正常运行，现在就可以使用kibana查看收集到的日志了
至此日志收集系统搭建完毕，EFK更多用途后面会陆续介绍，也可以自行前往官网查看。
