Elasticsearch环境安装

Elasticsearch环境安装

1、Elasticsearch

Elasticsearch是一个基于Lucene的搜索服务器。它提供了一个分布式多用户能力的全文搜索引擎，基于RESTful web接口。Elasticsearch是用Java语言开发的，并作为Apache许可条款下的开放源码发布，是一种流行的企业级搜索引擎。Elasticsearch用于云计算中，能够达到实时搜索，稳定，可靠，快速，安装使用方便。官方客户端在Java、.NET（C#）、PHP、Python、Apache Groovy、Ruby和许多其他语言中都是可用的。根据DB-Engines的排名显示，Elasticsearch是最受欢迎的企业搜索引擎，其次是Apache Solr，也是基于Lucene。

官网

2、环境安装

2.1 环境准备

2.1.1 JDK安装（略）

2.1.2 用户与组创建

groupadd elasticsearch
useradd -g elasticsearch elasticsearch
passwd elasticsearch

2.1.3 上传安装包（略）

2.2 安装

2.2.1 解压

tar -zxf elasticsearch-6.4.3.tar.gz

2.2.2 修改配置文件（config目录）

elasticsearch.yml

network.host: 192.168.31.150

http.port: 9200

2.2.3 启动

./elasticsearch

2.2.4 报错处理

2.2.4.1 错误一

ERROR: [2] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2020-04-30T23:10:54,681][INFO ][o.e.n.Node ] [2WZPJku] stopping …
[2020-04-30T23:10:54,754][INFO ][o.e.n.Node ] [2WZPJku] stopped
[2020-04-30T23:10:54,754][INFO ][o.e.n.Node ] [2WZPJku] closing …
[2020-04-30T23:10:54,787][INFO ][o.e.n.Node ] [2WZPJku] closed

注意：以下操作都需要用root用户

一：每个进程最大同时打开文件数太小，可通过下面2个命令查看当前数量

ulimit -Hn
ulimit -Sn

[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]

修改/etc/security/limits.conf文件，增加配置，用户退出后重新登录生效

* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096

二：max_map_count文件包含限制一个进程可以拥有的VMA(虚拟内存区域)的数量。

[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

修改/etc/sysctl.conf文件，增加配置vm.max_map_count=262144

vi /etc/sysctl.conf
sysctl -p

还有可能出现最大线程个数太低。查看命令

ulimit -Hu
ulimit -Su

max number of threads [3818] for user [es] is too low, increase to at least [4096]

修改配置文件/etc/security/limits.conf（和问题1是一个文件），增加配置

*               soft    nproc           4096
*               hard    nproc           4096

2.2.5 启动成功

启动成功直接访问http://192.168.31.150:9200即可

启动日志

[2020-04-30T23:22:49,329][INFO ][o.e.n.Node ] [] initializing …
[2020-04-30T23:22:49,455][INFO ][o.e.e.NodeEnvironment ] [2WZPJku] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [13.6gb], net total_space [16.9gb], types [rootfs]
[2020-04-30T23:22:49,455][INFO ][o.e.e.NodeEnvironment ] [2WZPJku] heap size [1015.6mb], compressed ordinary object pointers [true]
[2020-04-30T23:22:49,456][INFO ][o.e.n.Node ] [2WZPJku] node name derived from node ID [2WZPJkuAQmWBfDQ60ceWTw]; set [node.name] to override
[2020-04-30T23:22:49,456][INFO ][o.e.n.Node ] [2WZPJku] version[6.4.3], pid[1589], build[default/tar/fe40335/2018-10-30T23:17:19.084789Z], OS[Linux/3.10.0-862.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot™ 64-Bit Server VM/1.8.0_241/25.241-b07]
[2020-04-30T23:22:49,457][INFO ][o.e.n.Node ] [2WZPJku] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.RuMmCsgz, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/home/elasticsearch/elasticsearch-6.4.3, -Des.path.conf=/home/elasticsearch/elasticsearch-6.4.3/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
[2020-04-30T23:22:52,460][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [aggs-matrix-stats]
[2020-04-30T23:22:52,460][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [analysis-common]
[2020-04-30T23:22:52,460][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [ingest-common]
[2020-04-30T23:22:52,460][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [lang-expression]
[2020-04-30T23:22:52,460][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [lang-mustache]
[2020-04-30T23:22:52,461][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [lang-painless]
[2020-04-30T23:22:52,461][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [mapper-extras]
[2020-04-30T23:22:52,461][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [parent-join]
[2020-04-30T23:22:52,461][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [percolator]
[2020-04-30T23:22:52,461][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [rank-eval]
[2020-04-30T23:22:52,461][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [reindex]
[2020-04-30T23:22:52,461][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [repository-url]
[2020-04-30T23:22:52,461][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [transport-netty4]
[2020-04-30T23:22:52,461][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [tribe]
[2020-04-30T23:22:52,461][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [x-pack-core]
[2020-04-30T23:22:52,461][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [x-pack-deprecation]
[2020-04-30T23:22:52,461][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [x-pack-graph]
[2020-04-30T23:22:52,462][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [x-pack-logstash]
[2020-04-30T23:22:52,462][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [x-pack-ml]
[2020-04-30T23:22:52,462][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [x-pack-monitoring]
[2020-04-30T23:22:52,462][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [x-pack-rollup]
[2020-04-30T23:22:52,462][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [x-pack-security]
[2020-04-30T23:22:52,462][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [x-pack-sql]
[2020-04-30T23:22:52,462][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [x-pack-upgrade]
[2020-04-30T23:22:52,462][INFO ][o.e.p.PluginsService ] [2WZPJku] loaded module [x-pack-watcher]
[2020-04-30T23:22:52,462][INFO ][o.e.p.PluginsService ] [2WZPJku] no plugins loaded
[2020-04-30T23:22:59,582][INFO ][o.e.x.s.a.s.FileRolesStore] [2WZPJku] parsed [0] roles from file [/home/elasticsearch/elasticsearch-6.4.3/config/roles.yml]
[2020-04-30T23:23:00,525][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/1637] [Main.cc@109] controller (64 bit): Version 6.4.3 (Build 7a0781676dd492) Copyright © 2018 Elasticsearch BV
[2020-04-30T23:23:01,199][DEBUG][o.e.a.ActionModule ] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2020-04-30T23:23:01,541][INFO ][o.e.d.DiscoveryModule ] [2WZPJku] using discovery type [zen]
[2020-04-30T23:23:03,005][INFO ][o.e.n.Node ] [2WZPJku] initialized
[2020-04-30T23:23:03,005][INFO ][o.e.n.Node ] [2WZPJku] starting …
[2020-04-30T23:23:03,330][INFO ][o.e.t.TransportService ] [2WZPJku] publish_address {192.168.31.150:9300}, bound_addresses {192.168.31.150:9300}
[2020-04-30T23:23:03,361][INFO ][o.e.b.BootstrapChecks ] [2WZPJku] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2020-04-30T23:23:06,427][INFO ][o.e.c.s.MasterService ] [2WZPJku] zen-disco-elected-as-master ([0] nodes joined)[, ], reason: new_master {2WZPJku}{2WZPJkuAQmWBfDQ60ceWTw}{h2gsBdRJTVuxlmz4-nCRsg}{192.168.31.150}{192.168.31.150:9300}{ml.machine_memory=1910050816, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[2020-04-30T23:23:06,432][INFO ][o.e.c.s.ClusterApplierService] [2WZPJku] new_master {2WZPJku}{2WZPJkuAQmWBfDQ60ceWTw}{h2gsBdRJTVuxlmz4-nCRsg}{192.168.31.150}{192.168.31.150:9300}{ml.machine_memory=1910050816, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {2WZPJku}{2WZPJkuAQmWBfDQ60ceWTw}{h2gsBdRJTVuxlmz4-nCRsg}{192.168.31.150}{192.168.31.150:9300}{ml.machine_memory=1910050816, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)[, ]]])
[2020-04-30T23:23:06,520][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [2WZPJku] Failed to clear cache for realms [[]]
[2020-04-30T23:23:06,524][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [2WZPJku] publish_address {192.168.31.150:9200}, bound_addresses {192.168.31.150:9200}
[2020-04-30T23:23:06,524][INFO ][o.e.n.Node ] [2WZPJku] started
[2020-04-30T23:23:06,647][INFO ][o.e.g.GatewayService ] [2WZPJku] recovered [0] indices into cluster_state
[2020-04-30T23:23:07,095][INFO ][o.e.c.m.MetaDataIndexTemplateService] [2WZPJku] adding template [.watches] for index patterns [.watches*]
[2020-04-30T23:23:07,228][INFO ][o.e.c.m.MetaDataIndexTemplateService] [2WZPJku] adding template [.watch-history-9] for index patterns [.watcher-history-9*]
[2020-04-30T23:23:07,300][INFO ][o.e.c.m.MetaDataIndexTemplateService] [2WZPJku] adding template [.triggered_watches] for index patterns [.triggered_watches*]
[2020-04-30T23:23:07,396][INFO ][o.e.c.m.MetaDataIndexTemplateService] [2WZPJku] adding template [.monitoring-logstash] for index patterns [.monitoring-logstash-6-]
[2020-04-30T23:23:07,522][INFO ][o.e.c.m.MetaDataIndexTemplateService] [2WZPJku] adding template [.monitoring-es] for index patterns [.monitoring-es-6-]
[2020-04-30T23:23:07,602][INFO ][o.e.c.m.MetaDataIndexTemplateService] [2WZPJku] adding template [.monitoring-beats] for index patterns [.monitoring-beats-6-]
[2020-04-30T23:23:07,644][INFO ][o.e.c.m.MetaDataIndexTemplateService] [2WZPJku] adding template [.monitoring-alerts] for index patterns [.monitoring-alerts-6]
[2020-04-30T23:23:07,701][INFO ][o.e.c.m.MetaDataIndexTemplateService] [2WZPJku] adding template [.monitoring-kibana] for index patterns [.monitoring-kibana-6-]
[2020-04-30T23:23:07,909][INFO ][o.e.l.LicenseService ] [2WZPJku] license [44908d07-b4e3-4444-b0f4-c959e626fd8b] mode [basic] - valid