[k8s]融合docker记日志的思路实践

参考:
https://k8smeetup.github.io/docs/concepts/cluster-administration/logging/

k8s日志思路

• 先搞清楚docker的日志记录方法
  docker默认将容器的stdout stderr都记录到/var/lib/contianer/*.log里了,同时docker logs c1,可以看到日志.
  思路1:

RUN ln -sf /dev/stdout /var/log/nginx/access.log \
    && ln -sf /dev/stderr /var/log/nginx/error.log  #这仅是一种输出到sterr的方式,还有tail -f,cat等

将需要记录的日志,绑定到stdout, 如nginx默认镜像,2个日志文件,分别绑定到stdout,stderr.

不过这样容器混淆.错误和正确日志都混杂在一起了.

思路2:
将docker需要共享的日志通过VOLUM申明出去,找个tail -f的日志volume-from来共享这个日志目录,然后tail -f方式输出到stderr.

• 思路2:k8s里的想同方法处理的影子

一个pod产生2个日志文件,将他们输出到stdout–缺点有点乱

参考: https://k8smeetup.github.io/docs/concepts/cluster-administration/logging/

apiVersion: v1
kind: Pod
metadata:
  name: counter
spec:
  containers:
  - name: count
    image: busybox
    args:
    - /bin/sh
    - -c
    - >
      i=0;
      while true;
      do
        echo "$i: $(date)" >> /var/log/1.log;
        echo "$(date) INFO $i" >> /var/log/2.log;
        i=$((i+1));
        sleep 1;
      done
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  volumes:
  - name: varlog
    emptyDir: {}

• 思路2:k8s里将他们分开,即捆绑一个 tail -f功能的容器,共享volume

一个pod产生2个日志文件,分别将他们输出到stdout

参考: https://k8smeetup.github.io/docs/concepts/cluster-administration/logging/

这个和docker点类似,pod容纳多个容器,共享一个volume.

apiVersion: v1
kind: Pod
metadata:
  name: counter
spec:
  containers:
  - name: count
    image: busybox
    args:
    - /bin/sh
    - -c
    - >
      i=0;
      while true;
      do
        echo "$i: $(date)" >> /var/log/1.log;
        echo "$(date) INFO $i" >> /var/log/2.log;
        i=$((i+1));
        sleep 1;
      done
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  - name: count-log-1
    image: busybox
    args: [/bin/sh, -c, 'tail -n+1 -f /var/log/1.log']
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  - name: count-log-2
    image: busybox
    args: [/bin/sh, -c, 'tail -n+1 -f /var/log/2.log']
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  volumes:
  - name: varlog
    emptyDir: {}

• 还有一种方式,fluentd+app容器共享volume,fluentd读取日志发到日志服务器上去.同时fluentd的配置在configmap里管理即可.缺点: cm变动得重启.

k8s里 fluentd+app管理日志

apiVersion: v1
kind: Pod
metadata:
  name: counter
spec:
  containers:
  - name: count
    image: busybox
    args:
    - /bin/sh
    - -c
    - >
      i=0;
      while true;
      do
        echo "$i: $(date)" >> /var/log/1.log;
        echo "$(date) INFO $i" >> /var/log/2.log;
        i=$((i+1));
        sleep 1;
      done
    volumeMounts:
    - name: varlog
      mountPath: /var/log
  - name: count-agent
    image: gcr.io/google_containers/fluentd-gcp:1.30
    env:
    - name: FLUENTD_ARGS
      value: -c /etc/fluentd-config/fluentd.conf
    volumeMounts:
    - name: varlog
      mountPath: /var/log
    - name: config-volume
      mountPath: /etc/fluentd-config
  volumes:
  - name: varlog
    emptyDir: {}
  - name: config-volume
    configMap:
      name: fluentd-config

apiVersion: v1
data:
  fluentd.conf: |
    <source>
      type tail
      format none
      path /var/log/1.log
      pos_file /var/log/1.log.pos
      tag count.format1
    </source>

    <source>
      type tail
      format none
      path /var/log/2.log
      pos_file /var/log/2.log.pos
      tag count.format2
    </source>

    <match **>
      type google_cloud
    </match>
kind: ConfigMap
metadata:
  name: fluentd-config

　还有一种思路: fluentd启动监听个端口, docker将日志驱动指定为fluentd,并打tag,发给fluentd,fluentd将日志写到一个文件里.供filebeat来读取

- 启动fluentd

docker run -d -p 24224:24224 -p 24224:24224/udp -v /data:/fluentd/log fluent/fluentd

• 启动app容器

docker run -d \
    --log-driver=fluentd \
    --log-opt fluentd-address=localhost:24224 \
    --log-opt tag="log-test-container-A" \
    busybox sh -c 'while true; do echo "This is a log message from container A"; sleep 10; done;'

• filebeat的配置

filebeat.prospectors:
- type: log
  paths:
    - /data/*.log 
output.logstash:
  hosts: ["192.168.x.x:5043"]