kubernetes收集Pod日志

k8s日志收集方案

• 三种方案优缺点对比

方式	优点	缺点
方案一:Node上部署一个日志收集程序	每个Node仅需部署一个日志收集程序消耗资源少，对应用无侵入	应用程序日志需要写到标准输出和标准错误输出，不支持多行日志
方案二:Pod中附加专用日志收集的容器	低耦合	每个Pod启动一个收集代理，增加资源消耗，且增加运维维护成本
方案三:应用程序直接推送日志	无需额外收集工具	侵入应用，增加应用复杂度

以第二种方案收集NGINX日志为例

搭建Kibana和Elasticsearch

kind: List
apiVersion: v1
items:
- apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: kb-single
  spec:
    selector:
          matchLabels:
            app: kb-single
    replicas: 1
    template:
      metadata:
        labels:
          app: kb-single
      spec:
        containers:
        - image: docker.elastic.co/kibana/kibana:6.4.0
          name: kb
          imagePullPolicy: IfNotPresent
          env:
          - name: ELASTICSEARCH_URL
            value: "http://es-single:9200"
          ports:
          - name: http
            containerPort: 5601
- apiVersion: v1
  kind: Service
  metadata:
    name: kb-single-svc
  spec:
    type: NodePort
    ports:
    - name: http
      port: 5601
      targetPort: 5601 
      nodePort: 32601
    selector:
      app: kb-single            
- apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: es-single
  spec:
    selector:
          matchLabels:
            app: es-single
    replicas: 1
    template:
      metadata:
        labels:
          app: es-single
      spec:
        containers:
        - image: docker.elastic.co/elasticsearch/elasticsearch:6.4.0
          imagePullPolicy: IfNotPresent
          name: es
          env:
          - name: network.host
            value: "_site_"
          - name: node.name
            value: "${HOSTNAME}"
          - name: discovery.zen.ping.unicast.hosts
            value: "${ES_SINGLE_NODEPORT_SERVICE_HOST}"
          - name: cluster.name
            value: "test-single"
          - name: ES_JAVA_OPTS
            value: "-Xms128m -Xmx128m"
          volumeMounts:
          - name: es-single-data
            mountPath: /usr/share/elasticsearch/data
        volumes:
          - name: es-single-data
            emptyDir: {}
- apiVersion: v1
  kind: Service
  metadata: 
    name: es-single-nodeport
  spec:
    type: NodePort
    ports:
    - name: http
      port: 9200
      targetPort: 9200
      nodePort: 31200
    - name: tcp
      port: 9300
      targetPort: 9300
      nodePort: 31300
    selector:
      app: es-single
- apiVersion: v1
  kind: Service
  metadata:
    name: es-single
  spec:
    clusterIP: None
    ports:
    - name: http
      port: 9200
    - name: tcp
      port: 9300
    selector:
      app: es-single

访问任意节点IP+32601

收集NGINX日志

• 创建nginx的configmap

apiVersion: v1
kind: ConfigMap
metadata:
    name: nginx-config
data:
    default.conf: |
        server {
          listen       80;
          server_name  localhost;
          root   /usr/share/nginx/html;
          access_log  /var/log/nginx/host_access.log;
          error_log  /var/log/nginx/host_error.log debug;
          location / {
            root   /usr/share/nginx/html;
            index  index.html index.htm;
          }
          error_page   500 502 503 504  /50x.html;
          location = /50x.html {
            root   /usr/share/nginx/html;
          }
          }

• 创建filebeat-nginx的configmap

kind: List
apiVersion: v1
items:
- apiVersion: v1
  kind: ConfigMap
  metadata:
    name: filebeat-nginx-config
  data:
    filebeat.yml: |
      processors:
        - add_cloud_metadata:
      filebeat.modules:
      - module: system
      filebeat.inputs:
      - type: log
        paths:
          - /var/log/nginx/host_access.log
        symlinks: true
        tags: ["nginx-access-log"]
      output.elasticsearch:
        hosts: ['es-single:9200']
        indices:
          - index: "nginx-access-log-%{+yyyy.MM.dd}"
            when.contains:
              tags: "nginx-access-log"

• 创建nginx

apiVersion: v1
kind: Service
metadata:
    name: nginx-service
spec:
    type: NodePort
    selector:
        app: nginx
    ports:
        - protocol: TCP
          port: 80
          targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - name: nginx-config
          mountPath: "/etc/nginx/conf.d"
        - name: nginx-logs
          mountPath: "/var/log/nginx"
        
      - name: filebeat
        image: docker.elastic.co/beats/filebeat:6.4.0
        imagePullPolicy: IfNotPresent
        args: ["-c","/etc/filebeat.yml","-e"]
        volumeMounts:
          - name: filebeat-config
            mountPath: "/etc/filebeat.yml"
            subPath: filebeat.yml
          - name: nginx-logs
            mountPath: "/var/log/nginx"
      volumes:
        - name: nginx-config
          configMap:
            name: nginx-config
        - name: filebeat-config
          configMap: 
            name: filebeat-nginx-config
        - name: nginx-logs
          emptyDir: {}

• 添加kibana索引