ELK高级搜索一入门
ELK介绍
ELK是什么？
ELK是一个免费开源的日志分析架构技术栈总称，官网https://www.elastic.co/cn。包含三大基础组件，分别是Elasticsearch、Logstash、Kibana。Logstash用于数据抽取。Elasticsearch是搜索分析。Kibana是数据展现。
ELK架构
组件介绍
Elasticsearch
是使用java开发，基于Lucene、分布式、通过Restful方式进行交互的近实时搜索平台框架。它的特点有：分布式，零配置，自动发现，索引自动分片，索引副本机制，restful风格接口，多数据源，自动搜索负载等。下载地址：Download Elastic Enterprise Search | Elastic
- 启动ES
yangyanping@ZBMac-WP2HJYDWY bin % ./elasticsearch
[2022-05-29T19:57:27,899][INFO ][o.e.n.Node ] [ZBMac-WP2HJYDWY] version[7.10.1], pid[26672], build[default/tar/1c34507e66d7db1211f66f3513706fdf548736aa/2020-12-05T01:00:33.671820Z], OS[Mac OS X/10.16/x86_64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15.0.1/15.0.1+9]
[2022-05-29T19:57:27,905][INFO ][o.e.n.Node ] [ZBMac-WP2HJYDWY] JVM home [/Users/yangyanping/Downloads/server/es7/jdk.app/Contents/Home], using bundled JDK [true]
- 检查ES是否启动成功
GET http://localhost:9200/?Pretty
{
"name" : "ZBMAC-15aba3b68",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "ZyNrs0u7SmSAMHDLaDl3WA",
"version" : {
"number" : "7.10.1",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "1c34507e66d7db1211f66f3513706fdf548736aa",
"build_date" : "2020-12-05T01:00:33.671820Z",
"build_snapshot" : false,
"lucene_version" : "8.7.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}- 查询集群状态请求
GET http://localhost:9200/_cluster/health?pretty
{
"cluster_name" : "elasticsearch",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 6,
"active_shards" : 6,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}Logstash
Logstash 基于java开发，是一个数据抽取转化工具。一般工作方式为c/s架构，client端安装在需要收集信息的主机上，server端负责将收到的各节点日志进行过滤、修改等操作，并发往elasticsearch或其他组件上去。
Kibana
Kibana 基于nodejs，也是一个开源和免费的可视化工具。Kibana可以为 Logstash 和 ElasticSearch 提供的日志分析友好的 Web 界面，可以汇总、分析和搜索重要数据日志。下载地址：Download Kibana Free | Get Started Now | Elastic
- 启动 Kibana
yangyanping@ZBMac-WP2HJYDWY bin % sudo ./kibana --allow-root
[2022-04-13T14:47:41.163+08:00][INFO ][plugins-service] Plugin "metricsEntities" is disabled.
[2022-04-13T14:47:41.270+08:00][INFO ][http.server.Preboot] http server running at http://localhost:5601
[2022-04-13T14:47:41.376+08:00][INFO ][plugins-system.preboot] Setting up [1] plugins: [interactiveSetup]
[2022-04-13T14:47:41.381+08:00][INFO ][preboot] "interactiveSetup" plugin is holding setup: Validating Elasticsearch connection configuration…
[2022-04-13T14:47:41.431+08:00][INFO ][root] Holding setup until preboot stage is completed.
i Kibana has not been configured.
Go to http://localhost:5601/?code=919355 to get started.- Kibana的访问
通过地址：http://localhost:5601/app/dev_tools#/console可以在控制台中发送请求到elastic。
如：发送get请求，查看集群状态GET _cluster/health
- 快速查看集群中有哪些索引 ，发送请求：GET /_cat/indices?v
- 创建索引
elasticsearch-head
node安装
下载地址：Download | Node.js
yangyanping@ZBMac-WP2HJYDWY ~ % brew install node
Running `brew update --preinstall`...
==> Auto-updated Homebrew!
Updated 2 taps (homebrew/core and homebrew/cask).
==> Updated Formulae
Updated 73 formulae.
==> Updated Casks
Updated 8 casks.
..........elasticsearch-head安装
下载地址：GitHub - mobz/elasticsearch-head: A web front end for an elastic search cluster
yangyanping@ZBMac-WP2HJYDWY elasticsearch-head-master % npm install
npm WARN EBADENGINE Unsupported engine {
npm WARN EBADENGINE package: '[email protected]',
npm WARN EBADENGINE required: { node: '0.10 || 0.12 || 4 || 5 || 6' },
npm WARN EBADENGINE current: { node: 'v18.2.0', npm: '8.9.0' }
.........
yangyanping@ZBMac-WP2HJYDWY elasticsearch-head-master % npm run startelasticsearch 配置修改和重启
打开ES的配置文件elasticsearch.yml输入下面的两句
http.cors.enabled: true
http.cors.allow-origin: "*"
vi elasticsearch.yml
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"插件运行情况如下截图
访问地址：http://localhost:9100/
参考：
ELK集中化日志解决方案