大数据环境搭建系列【六】Elasticsearch集群搭建

前言

本文记录搭建ES集群的过程，系统使用centos7

集群规划

本文通过三个节点搭建ES集群，集群规划如下

hostname	IP
ES1	192.168.88.60
ES2	192.168.88.61
ES3	192.168.88.62

JDK环境

安装ES需要jdk环境，首先每台机器都需要准备安装jdk，这里不做详细描述了
jdk版本和es的兼容性

关闭防火墙、hosts配置

（略）

ES安装

• 下载
  下载地址
  选择linux x86 版本，我下载的版本号是8.2.3
• 安装
  解压到指定安装目录，我这里安装在opt下

tar -zxvf elasticsearch-8.2.3-linux-x86_64.tar.gz -C /opt/

ES配置

ES的配置文件在config目录下，以下介绍几个重点配置

• elasticsearch.yml es的核心配置
  首先新建ES的数据目录及日志存储路径

mkdir -p /home/es/data
mkdir -p /home/es/logs

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: ES-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /home/es/data
#
# Path to log files:
#
path.logs: /home/es/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
network.host: ES1
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["ES1", "ES2","ES3"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["ES-1", "ES-2","ES-3"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# --------------------------------- Readiness ----------------------------------
#
# Enable an unauthenticated TCP readiness endpoint on localhost
#
#readiness.port: 9399
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false

• jvm.options es的jvm参数配置
  主要配置一下jvm heap的大小

################################################################
## IMPORTANT: JVM heap size
################################################################
##
## The heap size is automatically configured by Elasticsearch
## based on the available memory in your system and the roles
## each node is configured to fulfill. If specifying heap is
## required, it should be done through a file in jvm.options.d,
## which should be named with .options suffix, and the min and
## max should be set to the same value. For example, to set the
## heap to 4 GB, create a new file in the jvm.options.d
## directory containing these lines:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/8.2/heap-size.html
## for more information
##
################################################################

创建ES用户修改权限

首先要说明的是elasticsearch 不能使用root启动，所以启动前或者安装前创建一个elasticsearch用户，然后把文件所有者都改为该用户

useradd esuser
chown -R esuser:esuser /opt/elasticsearch-8.2.3
chown -R esuser:esuser /home/es

启动集群

三台机器都安装配置好以后都执行以下启动操作

su esuser
/opt/elasticsearch-8.2.3/bin/elasticsearch

启动异常解决

• max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
  字面意思是最大虚拟内存空间为65530，太低了，需要增加到至少262144。那就增加到大于这个值就可以了，至于大多少就根据自己服务器的情况看了，具体就是配置sysctl.conf文件

vim /etc/sysctl.conf 

#文件末尾追加
vm.max_map_count=655360

加载配置

sysctl -p

• max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
  elasticsearch进程的最大文件描述符[4096]太低，增加到至少[65535]，修改以下文件进行配置

vim /etc/security/limits.conf

#末尾追加
esuser hard nofile 65536
esuser soft nofile 65536

esuser 表示运行 elasticsearch 的用户，hard 与 soft 表示限制的类型，nofile 表示 max number of open file descriptors，65536 表示设置的大小。

查看集群的状态

http://192.168.88.60:9200/_cat/health?v

页面返回以上内容status为green表示集群目前在最佳状态