k8s elk
#方案选择
Log-pilot +elasticsearch+kibana
Log-pilot
阿里开源日志收集agent,封装了filebeathe flutent,支持多后端例如logstash,redis
属于日志采集代理,deamonset 形式部署在k8s每个node节点。
Log-pilot 部署简单配置即可。
可以收集stdout和容器内日志。
Log-pilot k8s部署步骤
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: log-pilot
labels:
app: log-pilot
spec:
updateStrategy:
type: RollingUpdate
template:
metadata:
labels:
app: log-pilot
spec:
containers:
- name: log-pilot
image: registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.7-filebeat
resources:
limits:
memory: 500Mi
requests:
cpu: 200m
memory: 200Mi
env:
- name: “NODE_NAME”
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: “LOGGING_OUTPUT”
value: “elasticsearch”
- name: “ELASTICSEARCH_HOST”
value: " ELASTICSEARCH_HOST "
- name: “ELASTICSEARCH_PORT”
value: “31200”
volumeMounts:
- name: sock
mountPath: /var/run/docker.sock
- name: root
mountPath: /host
readOnly: true
- name: varlib
mountPath: /var/lib/filebeat
- name: varlog
mountPath: /var/log/filebeat
- name: localtime
mountPath: /etc/localtime
readOnly: true
livenessProbe:
failureThreshold: 3
exec:
command:
- /pilot/healthz
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 2
securityContext:
capabilities:
add:
- SYS_ADMIN
terminationGracePeriodSeconds: 30
volumes:
- name: sock
hostPath:
path: /var/run/docker.sock
- name: root
hostPath:
path: /
- name: varlib
hostPath:
path: /var/lib/filebeat
type: DirectoryOrCreate
- name: varlog
hostPath:
path: /var/log/filebeat
type: DirectoryOrCreate
- name: localtime
hostPath:
path: /etc/localtime
注意点:
被收集容器配置:主要配置env
apiVersion: v1
kind: Pod
metadata:
name: tomcat
spec:
containers:
elasticsearch部署步骤
k8s elasticsearch部署
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: base-elasticsearch6-deployment
spec:
replicas: 1
template: {metadata: {labels: {name: base-elasticsearch6}}, spec: {initContainers: [{name: init-sysctl, image: ‘busybox:1.27.2’, command: [sysctl, ‘-w’, vm.max_map_count=262144], securityContext: {privileged: true}}], containers: [{name: web-base-elasticsearch6, image: ‘docker.elastic.co/elasticsearch/elasticsearch:6.4.0’, imagePullPolicy: IfNotPresent, resources: {limits: {cpu: 1000m}, requests: {cpu: 100m}}, env: [{name: ES_JAVA_OPTS, value: ‘-Xms512m -Xmx512m’}], ports: [{containerPort: 9200}, {containerPort: 9300}], volumeMounts: [{name: elasticsearch, mountPath: /mnt/search}, {name: elasticsearch-yml, mountPath: /usr/share/elasticsearch/config/elasticsearch.yml, subPath: elasticsearch.yml}]}], volumes: [{name: elasticsearch, hostPath: {path: /mnt/search/student}}, {name: elasticsearch-yml, configMap: {name: esconfig}}]}}
Elastic svc
apiVersion: v1
kind: Service
metadata:
name: base-elasticsearch6
labels:
name: base-elasticsearch6
spec:
type: NodePort
ports:
Elastic head 展示elastic 数据
kind: List
apiVersion: v1
items:
直接加入elasticsearch 就可以展示
kibana部署步骤
kind: List
apiVersion: v1
items: