skywalking推荐采用es作为数据存储服务，且需要做一些特殊的配置，例如线程池的配置。所以单独整理一es部署文档。
1.官网下载es安装包
elasticsearch-6.4.2.zip
解压放在指定目录
image.png
2.创建用户
es不能用root角色的用户启动，所以需要创建普通用户
groups命令查询当前用户所属的角色
[root@i-ua1ewr6h bin]# groups
root
创建用户，并赋予权限
groups命令查询当前用户所属的角色
adduser esuser
chown -R esuser /data/elasticsearch/
下面许多配置都是针对该用户的配置
注意：先不要切换到esuser用户，先用高的权限用户配置下面配置。
3.配置ES节点
修改/config/elasticsearch.yml
elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
# 集群中的所有节点的集群名称必须一样，只有集群名称一样才能组成一个逻辑集群
cluster.name: skywalking-es-cluster
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
# 节点名称，每个节点名称要不一样
node.name: node-3
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1
# 放开网络权限
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
# 集群依赖发现的配置。选择指定host，互相依赖发现的方式
discovery.zen.ping.unicast.hosts: ["192.168.36.2:9300", "192.168.36.3:9300","192.168.36.4:9300"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes:
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
http.cors.enabled: true
#允许跨域
http.cors.allow-origin: "*"
# 线程的size取（cpu核数+1）
thread_pool:
search:
size: 5
queue_size: 10000
index:
size: 5
queue_size: 10000
bulk:
size: 5
queue_size: 10000
write:
size: 5
queue_size: 10000
# When you face query error at trace page, remember to check this.
#index:
# max_result_window: 5000000
discovery.zen.ping.unicast.hosts:集群依赖发现的配置，根据实际服务器地址进行配置
node.name: 每个节点要不一样，例如node-1,node-2,node-3
修改config/jvm.options
原来的Xms4和Xmx默认为1g，需要根据实际配置进行修改，可以选择服务器内存的40%-50%的大小，例如8G的配置为：
jvm.options
-Xms4g
-Xmx4g
注意：
参考官方教程：https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
参考网友不同的意见：https://www.cnblogs.com/yuanjiangw/p/9706830.html
(1)通常不要超过服务器物理内存的50%(如果服务器还部署了其他耗内存较多的应用，要另外考虑，应该设置的值比50%小的多)
(2)xms和xmx要相等
xms和xmx不相同，es启动将报错：
initial heap size [1073741824] not equal to maximum heap size [3221225472]; this can cause resize pauses and prevents mlockall from locking the entire heap
xms：JVM启动时整个堆(包括年轻代，年老代)的初始化大小
xmx：最大内存
xms与xmx如何设置：https://www.zhihu.com/question/361490914
验证：等启动完后看一下：
image.png
配置系统参数
准备：安装vim(可选)
yum install vim
在limits.conf（在文件最后添加）
[root@i-ayemhl9b bin]# vim /etc/security/limits.conf
esuser hard nofile 65536
esuser soft nofile 65536
esuser soft memlock unlimited
esuser hard memlock unlimited
以上配置解决问题
max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
memory locking requested for elasticsearch process but memory is not locked
vim /etc/sysctl.conf
添加 “vm.max_map_count”设置: vm.max_map_count=655360
并执行：sysctl -p
[root@i-ayemhl9b bin]# vim /etc/sysctl.conf
vm.max_map_count=655360
[root@i-ayemhl9b data]# sysctl -p
vm.max_map_count = 655360
以上配置解决问题：
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
3.ulimit -n和-u可以查看linux的最大进程数和最大文件打开数
参考：https://www.thegeekdiary.com/how-to-set-nproc-hard-and-soft-values-in-centos-rhel-567/
3.1、vim /etc/security/limits.d/20-nproc.conf文件(CentOS 7、RHEL 7修改20-nproc.conf，CentOS 5、CentOS 6、RHEL 5、RHEL 6修改90-nproc.conf)尾部添加
* soft nofile 204800
* hard nofile 204800
3.2 vim /etc/security/limits.d/def.conf文件尾添加
* soft nofile 204800
* hard nofile 204800
这两个文件的设置将会覆盖前面的设置。重启后生效
以上配置解决问题：max number of threads [3895] for user [esuser] is too low, increase to at least [4096]
执行： visudo
加 esuser ALL=(ALL) ALL
示例：
## Next comes the main part: which users can run what software on
## which machines (the sudoers file can be shared between multiple
## systems).
## Syntax:
##
## user MACHINE=COMMANDS
##
## The COMMANDS section may have other options added to it.
##
## Allow root to run any commands anywhere
root ALL=(ALL) ALL
esuser ALL=(ALL) ALL
以上配置解决某些情况下无法读写的问题
启动
1.安装jdk
jdk 放在/data/jdk/jdk1.8.0_111
# 赋予权限
chown -R esuser /data/jdk/
#切换用户
sudo -su esuser
# 进入用户根目录
cd ~
# 编辑.bash_profile
vi .bash_profile
# 加上
PATH=$PATH:$HOME/.local/bin:$HOME/bin
export JAVA_HOME=/data/jdk/jdk1.8.0_111
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH
# 刷新配置
source .bash_profile
# 赋予文件操作权限
chmod 777 -R /data/jdk/
# 测试
java -verison
2.切换到esuser，启动es
[root@i-iberg11b elasticsearch-6.4.2]$ sudo -su esuser
[esuser@i-iberg11b elasticsearch-6.4.2]$ cd /data/elasticsearch/
[esuser@i-iberg11b elasticsearch-6.4.2]$ chmod 777 -R *
解决没权限的问题：
[root@i-iberg11b elasticsearch-6.4.2]$ [2019-12-07T11:10:41,143][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-1] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: org.elasticsearch.bootstrap.BootstrapException: [java.io](http://java.io/).IOException: Cannot run program "/data/elasticsearch-6.4.2/modules/x-pack-ml/platform/linux-x86_64/bin/controller": error=13, Permission denied
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:140) ~[elasticsearch-6.4.2.jar:6.4.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:127) ~[elasticsearch-6.4.2.jar:6.4.2]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.4.2.jar:6.4.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.4.2.jar:6.4.2]
3.执行./bin/elasticsearch
备注：./bin/elasticsearch -d可以后台运行
[esuser@i-ua1ewr6h config]$ ./elasticsearch
[2020-02-11T22:44:25,233][INFO ][o.e.n.Node ] [node-3] initializing ...
[2020-02-11T22:44:25,298][INFO ][o.e.e.NodeEnvironment ] [node-3] using [1] data paths, mounts [[/data (/dev/mapper/mbcloud-mbcloud--lv)]], net usable_space [92.7gb], net total_space [98.3gb], types [ext4]
[2020-02-11T22:44:25,298][INFO ][o.e.e.NodeEnvironment ] [node-3] heap size [3.9gb], compressed ordinary object pointers [true]
[2020-02-11T22:44:25,300][INFO ][o.e.n.Node ] [node-3] node name [node-3], node ID [DCVmwmcqRXaqHMW9NvV6Gg]
[2020-02-11T22:44:25,300][INFO ][o.e.n.Node ] [node-3] version[6.4.2], pid[9731], build[default/zip/04711c2/2018-09-26T13:34:09.098244Z], OS[Linux/3.10.0-957.12.2.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_111/25.111-b14]
[2020-02-11T22:44:25,301][INFO ][o.e.n.Node ] [node-3] JVM arguments [-Xms4g, -Xmx4g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.nY7IagFY, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/data/elasticsearch/elasticsearch-6.4.2, -Des.path.conf=/data/elasticsearch/elasticsearch-6.4.2/config, -Des.distribution.flavor=default, -Des.distribution.type=zip]
[2020-02-11T22:44:27,349][INFO ][o.e.p.PluginsService ] [node-3] loaded module [aggs-matrix-stats]
[2020-02-11T22:44:27,349][INFO ][o.e.p.PluginsService ] [node-3] loaded module [analysis-common]
[2020-02-11T22:44:27,349][INFO ][o.e.p.PluginsService ] [node-3] loaded module [ingest-common]
[2020-02-11T22:44:27,349][INFO ][o.e.p.PluginsService ] [node-3] loaded module [lang-expression]
[2020-02-11T22:44:27,349][INFO ][o.e.p.PluginsService ] [node-3] loaded module [lang-mustache]
[2020-02-11T22:44:27,350][INFO ][o.e.p.PluginsService ] [node-3] loaded module [lang-painless]
[2020-02-11T22:44:27,350][INFO ][o.e.p.PluginsService ] [node-3] loaded module [mapper-extras]
[2020-02-11T22:44:27,350][INFO ][o.e.p.PluginsService ] [node-3] loaded module [parent-join]
[2020-02-11T22:44:27,350][INFO ][o.e.p.PluginsService ] [node-3] loaded module [percolator]
[2020-02-11T22:44:27,350][INFO ][o.e.p.PluginsService ] [node-3] loaded module [rank-eval]
[2020-02-11T22:44:27,350][INFO ][o.e.p.PluginsService ] [node-3] loaded module [reindex]
[2020-02-11T22:44:27,350][INFO ][o.e.p.PluginsService ] [node-3] loaded module [repository-url]
[2020-02-11T22:44:27,351][INFO ][o.e.p.PluginsService ] [node-3] loaded module [transport-netty4]
[2020-02-11T22:44:27,351][INFO ][o.e.p.PluginsService ] [node-3] loaded module [tribe]
[2020-02-11T22:44:27,351][INFO ][o.e.p.PluginsService ] [node-3] loaded module [x-pack-core]
[2020-02-11T22:44:27,351][INFO ][o.e.p.PluginsService ] [node-3] loaded module [x-pack-deprecation]
[2020-02-11T22:44:27,351][INFO ][o.e.p.PluginsService ] [node-3] loaded module [x-pack-graph]
[2020-02-11T22:44:27,351][INFO ][o.e.p.PluginsService ] [node-3] loaded module [x-pack-logstash]
[2020-02-11T22:44:27,351][INFO ][o.e.p.PluginsService ] [node-3] loaded module [x-pack-ml]
[2020-02-11T22:44:27,351][INFO ][o.e.p.PluginsService ] [node-3] loaded module [x-pack-monitoring]
[2020-02-11T22:44:27,352][INFO ][o.e.p.PluginsService ] [node-3] loaded module [x-pack-rollup]
[2020-02-11T22:44:27,352][INFO ][o.e.p.PluginsService ] [node-3] loaded module [x-pack-security]
[2020-02-11T22:44:27,352][INFO ][o.e.p.PluginsService ] [node-3] loaded module [x-pack-sql]
[2020-02-11T22:44:27,352][INFO ][o.e.p.PluginsService ] [node-3] loaded module [x-pack-upgrade]
[2020-02-11T22:44:27,352][INFO ][o.e.p.PluginsService ] [node-3] loaded module [x-pack-watcher]
[2020-02-11T22:44:27,352][INFO ][o.e.p.PluginsService ] [node-3] no plugins loaded
[2020-02-11T22:44:31,615][INFO ][o.e.x.s.a.s.FileRolesStore] [node-3] parsed [0] roles from file [/data/elasticsearch/elasticsearch-6.4.2/config/roles.yml]
[2020-02-11T22:44:32,189][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/9800] [Main.cc@109] controller (64 bit): Version 6.4.2 (Build 660eefe6f2ea55) Copyright (c) 2018 Elasticsearch BV
[2020-02-11T22:44:32,669][DEBUG][o.e.a.ActionModule ] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2020-02-11T22:44:32,897][INFO ][o.e.d.DiscoveryModule ] [node-3] using discovery type [zen]
[2020-02-11T22:44:33,798][INFO ][o.e.n.Node ] [node-3] initialized
[2020-02-11T22:44:33,798][INFO ][o.e.n.Node ] [node-3] starting ...
[2020-02-11T22:44:33,963][INFO ][o.e.t.TransportService ] [node-3] publish_address {192.168.36.2:9300}, bound_addresses {[::]:9300}
[2020-02-11T22:44:33,985][INFO ][o.e.b.BootstrapChecks ] [node-3] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2020-02-11T22:44:37,081][INFO ][o.e.c.s.MasterService ] [node-3] zen-disco-elected-as-master ([0] nodes joined)[, ], reason: new_master {node-3}{DCVmwmcqRXaqHMW9NvV6Gg}{5p5OsE9TRRCRysivJShu7Q}{192.168.36.2}{192.168.36.2:9300}{ml.machine_memory=8201076736, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[2020-02-11T22:44:37,087][INFO ][o.e.c.s.ClusterApplierService] [node-3] new_master {node-3}{DCVmwmcqRXaqHMW9NvV6Gg}{5p5OsE9TRRCRysivJShu7Q}{192.168.36.2}{192.168.36.2:9300}{ml.machine_memory=8201076736, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {node-3}{DCVmwmcqRXaqHMW9NvV6Gg}{5p5OsE9TRRCRysivJShu7Q}{192.168.36.2}{192.168.36.2:9300}{ml.machine_memory=8201076736, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)[, ]]])
[2020-02-11T22:44:37,126][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node-3] publish_address {192.168.36.2:9200}, bound_addresses {[::]:9200}
[2020-02-11T22:44:37,126][INFO ][o.e.n.Node ] [node-3] started
[2020-02-11T22:44:37,381][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [node-3] Failed to clear cache for realms [[]]
[2020-02-11T22:44:37,434][INFO ][o.e.l.LicenseService ] [node-3] license [7a22c4d9-f268-427b-83e7-26736d217d9b] mode [basic] - valid
[2020-02-11T22:44:37,446][INFO ][o.e.g.GatewayService ] [node-3] recovered [0] indices into cluster_state
验证：
验证日志
[root@i-ua1ewr6h ~]# curl http://localhost:9200
{
"name" : "node-3",
"cluster_name" : "skywalking-es-cluster",
"cluster_uuid" : "0hD4Co71Seq62ZyMxSaN6A",
"version" : {
"number" : "6.4.2",
"build_flavor" : "default",
"build_type" : "zip",
"build_hash" : "04711c2",
"build_date" : "2018-09-26T13:34:09.098244Z",
"build_snapshot" : false,
"lucene_version" : "7.4.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
[root@i-ua1ewr6h ~]#
查看节点状态：
http://localhost:9200/_nodes/stats?pretty