Erlang使用Elasticsearch、Logstash、Kibana搭建日志平台

1.简介

将Erlang运行时产生的Log，通过ELK(Elasticsearch、Logstash、Kibana)进行查看分析。

 

2.安装

2.1 Erlang相关:

Erlang 20+

https://github.com/lambdaclass/lager_logstash_backend  这是扩展的功能库

如果你使用Rebar3，将其依赖添加到配置文件。

如果没有使用Rebar3 ，需要自己手动下载 其它的依赖库，例如<lager> <jiffy>等依赖库，编写makefile文件。

2.2 ELK安装:

brew install elasticsearch

brew install kibana

brew install logstash

启动ES & Kibana

$ brew services start elasticsearch

$ brew services start kibana

启动后可以在本地浏览器访问

ES: http://localhost:9200

Kibana: http://localhost:5601

 

3.运行

logstash启动需要指定config，自己手动创建一个logstash.conf ，我放置在etc目录下。

启动：logstash -f /etc/logstash.conf

input {
    udp  {
        codec => "json"
        host => "127.0.0.1"
        port  => 9125
        type  => "erlang"
    }
}

filter {
  
}

output {
   elasticsearch {
        hosts => ["127.0.0.1:9200"]
        index => "logstash_test"     
     } 
     stdout { codec => rubydebug }
}

 

Erlang这边只需要启动：

 ok = lager:start()。

相应的配置文件：erl -config xxx

[


  %% Lager Bein %%%
  {kernel, [{start_timer, true}]},

  {sasl, [
    {utc_log, true},
    {sasl_error_logger, {file, "log/sasl-error.log"}},
    {errlog_type, error},
    {error_logger_mf_dir, "log/sasl"},      % Log directory
    {error_logger_mf_maxbytes, 104857600},   % 100 MB max file size
    {error_logger_mf_maxfiles, 5}           % 5 files max
  ]},

  {lager, [
    {log_root, "./log"},
      %% What handlers to install with what arguments
    {handlers, [
      {lager_console_backend, info},
      {lager_file_backend, [
        {file, "log/error.log"}, {level, error}, {size, 10485760}, {date, "$D0"}, {count, 5}]},
      {lager_file_backend, [
        {file, "log/console.log"}, {level, info}, {size, 10485760}, {date, "$D0"}, {count, 5}]},
      {lager_logstash_backend, [
        {level,        error},
        {logstash_host, "127.0.0.1"},
        {logstash_port, 9125},
        {node_role,    "erlang"},
        {node_version, "0.0.1"},
        {metadata, [
          {account_token,  [{encoding, string}]},
          {client_os,      [{encoding, string}]},
          {client_version, [{encoding, string}]}
        ]}
      ]}
    ]},

    %% Whether to write a crash log, and where. Undefined means no crash logger.
    {crash_log, "log/crash.log"}
  ]}
  %% Lager End %%%
].

4.最终效果