Loki搭建日志收集系统

Loki

什么是Loki

Loki是受Prometheus启发的水平可扩展，高度可用的多租户日志聚合系统。他被设计为非常经济高效且易于操作。它不索引日志内容，而是为每个日志流设置一组标签

Loki文档网址：https://grafana.com/docs/loki/latest/

下载网址：https://github.com/grafana/loki/releases

安装loki

• 获取软件包
  
• 解压软件包

unzip loki-linux-amd64.zip

[root@localhost loki]# ll
-rwxr-xr-x 1 root root 88166400 May 20  2020 loki-linux-amd64
-rw-r--r-- 1 root root 26432293 Feb 22 11:26 loki-linux-amd64.zip

• 编写配置文件

vim loki.yaml
---
auth_enabled: false

server:
  http_listen_port: 3100

ingester:
  lifecycler:
    address: 192.168.0.181
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
    final_sleep: 0s
  chunk_idle_period: 5m
  chunk_retain_period: 30s
  max_transfer_retries: 0

schema_config:
  configs:
    - from: 2020-07-01
      store: boltdb
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 168h

storage_config:
  boltdb:
    directory: /loki/index

  filesystem:
    directory: /loki/chunks

limits_config:
  enforce_metric_name: false
  reject_old_samples: true
  reject_old_samples_max_age: 168h
  ingestion_rate_mb: 15

chunk_store_config:
  max_look_back_period: 0s

table_manager:
  retention_deletes_enabled: false
  retention_period: 0s

• 启动loki

[root@localhost loki]# nohup ./loki-linux-amd64 -config.file=./loki.yaml >./loki.log 2>&1 &
[1] 108017
[root@localhost loki]# ps -fe|grep loki
root     108017  79258  9 16:00 pts/0    00:00:00 ./loki-linux-amd64 -config.file=./loki.yaml

安装promtail

promtail是负责日志收集，类似elk中的logstash、filebeat等，如果收集docker容器日志的话可以用docker插件来收集容器日志

• 获取软件包并解压
  
• 配置pormtail配置文件

vim promtail.yaml
---
server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: ./positions.yaml

clients:
  - url: http://192.168.0.181:3100/loki/api/v1/push

scrape_configs:
 - job_name: zzz
   static_configs:
   - labels:
      app: zzz
      host: 192.168.0.181
      env: prod
      __path__: /var/log/messages

• 启动promtail

nohup ./promtail-linux-amd64 -config.file=./promtail.yaml > ./promtail.log 2>&1 &
[root@localhost loki]# ps -fe|grep promtail
root     113788  79258  1 17:00 pts/0    00:00:00 ./promtail-linux-amd64 -config.file=./promtail.yaml

配置grafana展示日志

收集docker容器日志

• 确认docker版本

[root@localhost local]# docker --version
Docker version 18.09.7, build 2d0083d

需要更改docker的log-driver为loki 13版本的会更改失败

• 安装插件

docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions
[root@localhost local]# docker plugin list
ID                  NAME                DESCRIPTION           ENABLED
5df430512cf6        loki:latest         Loki Logging Driver   true

• 启动容器测试

docker run -itd -p 80:80 --log-driver=loki --log-opt loki-url="http://192.168.0.181:3100/loki/api/v1/push" --log-opt max-size=50m --log-opt max-file=10 nginx

通过curl命令访问nginx，然后再grafana中查看