ELK Stack 7.3.0构建多系统多用户安全认证日志平台（五）

本篇内容主要讲解以下两个方面：

• 日志过滤清洗组件Logstash安装使用
• Elasticsearch 索引文件可视化管理工具Kibana安装使用

操作系统环境，centos7

一、Logstash组件安装使用

1、下载Logstash安装包

curl -L -O https://artifacts.elastic.co/downloads/logstash/logstash-7.3.0.tar.gz

2、解压

tar -zxvf logstash-7.3.0.tar.gz

3、配置文件logstash.conf修改

input {
 redis {
  host => "192.168.137.55"
  port => 6379
  data_type => "list"
  key => "filebeatinput"
 }
 redis {
  host => "192.168.137.55"
  port => 6379
  data_type => "list"
  key => "filebeatinputapp2"
 }
}

output {
 if[fields][log_type] == 'filebeatapp1'{
  elasticsearch {
   hosts => ["http://192.168.137.55:9200","http://192.168.137.56:9200","http://192.168.137.57:9200"]
   index => "app1index-filebeatapp1-%{+YYYY.MM.dd}"
   #user => "elastic"
   #password => "elastic123"
  }
 }
 if[fields][log_type] == 'filebeatapp2'{
  elasticsearch {
   hosts => ["http://192.168.137.55:9200","http://192.168.137.56:9200","http://192.168.137.57:9200"]
   index => "app2index-filebeatapp2-%{+YYYY.MM.dd}"
   #user => "elastic"
   #password => "elastic123"
   }
  }
 }

4、切换至elkstack用户启动Logstash，可以先通过控制台标准输入、输出方式进行验证logstash是否正常启动

bin/logstash -e 'input { stdin {}} output{stdout {codec=>rubydebug}}'
输入inputmsgtest
{
 "host" => "nodek8sworker",
 "@timestamp" => 2019-08-24T09:28:34.130Z,
 "message" => "inputmsgtest",
 "@version" => "1"
}
看到以上输出，logstash启动成功。

5、通过logstash配置文件启动

bin/logstash -f config/logstash-redis-es.conf

[2019-08-24T17:34:22,987][INFO ][logstash.inputs.redis ] Registering Redis {:identity=>"redis://@192.168.137.55:6379/0 list:filebeatinput"}
[2019-08-24T17:34:23,007][INFO ][logstash.inputs.redis ] Registering Redis {:identity=>"redis://@192.168.137.55:6379/0 list:filebeatinputapp2"}
[2019-08-24T17:34:23,254][INFO ][logstash.javapipeline ] Pipeline started {"pipeline.id"=>"main"}
[2019-08-24T17:34:25,253][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-08-24T17:34:31,162][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
启动成功

二、可视化管理平台Kibana安装使用

1、下载kibana安装包

wget https://artifacts.elastic.co/downloads/kibana/kibana-7.3.0-linux-x86_64.tar.gz

2、解压

tar -zxvf kibana-7.3.0-linux-x86_64.tar.gz

3、切换至elkstack用户，如果还没创建用户，可以参考第二章的内容进行用户创建及授权，启动kibana

cd /kibana-7.3.0-linux-x86_64
su elkstack
bin/kibana

log [09:53:36.443] [warning][task_manager] Task vis_telemetry "oss_telemetry-vis_telemetry" failed in attempt to run: [version_conflict_engine_exception] [oss_telemetry-vis_telemetry]: version conflict, required seqNo [23], primary term [9]. current document has seqNo [24] and primary term [9], with { index_uuid="EJuiBUIVRF6Caik8e_81Rg" & shard="0" & index=".kibana_task_manager" }
log [09:53:37.023] [info][listening] Server running at http://0.0.0.0:5601
log [09:53:37.459] [info][server][Kibana][http] http server running
log [09:53:37.934] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready

浏览器访问：http://192.168.137.55:5601

image.png

根据索引文件创建index pattern，创建完之后就可以进行搜索。

image.png

本篇内容就讲到这里，大家在操作过程中如果遇到问题，欢迎留言一起沟通。