ELK - CentOS7 安装 elasticsearch 7.17.5


# 新建用户组:groupadd elk

# 添加用户:elk
# adduser elk  或者:useradd elk
# 设置密码(不能太简单,length>=7,不能包含用户名)
# passwd elk   
举例:JS123456

# 将用户添加到指定用户组:
#useradd -G {group-name} {username}
usermod -G elk elk

# 查看验证用户组是否分配成功:
#groups {username}
groups elk

# 用root用户给october赋予指定目录的权限

命令格式:chown {username}:{group-name} /{dir}
chown -R elk:elk /opt/soft/elk/elasticsearch-7.17.5


tar -zxvf  elasticsearch-7.17.5-linux-x86_64.tar.gz
mv elasticsearch-7.17.5-linux-x86_64 /opt/soft/elk/elasticsearch-7.17.5


# mkdir -p esdata
# mkdir -p eslogs


# chown -R elk:elk /opt/soft/elk/esdata/
# chown -R elk:elk /opt/soft/elk/eslogs/


【配置服务器】
# ulimit -Hn 
# ulimit -Sn

# 修改配置
# vim /etc/security/limits.conf

设置:
soft    nofile          65536
hard    nofile          65536
* soft nofile 65536

* hard nofile 65536

* soft nproc 32000

* hard nproc 32000

* hard memlock unlimited

* soft memlock unlimited

# 修改配置
# vim /etc/sysctl.conf

增加配置:
vm.max_map_count=262144

elasticsearch最小需要max_map_count为262144,而虚拟机默认只有65536。
加大vm.max_map_count(进程可以拥有的VMA(虚拟内存区域)的数量,虚拟内存区域是一个连续的虚拟地址空间区域)
# vim /etc/sysctl.conf
在sysctl.conf中添加
vm.max_map_count=262144
# sysctl -p  #重启生效
--------------------------------sysctl.conf配置完毕-------------------------
配置原因:
1、资源问题:每个打开的文件都需要消耗内存来管理,而内存是有限的。
2、安全问题:如果不限制的话,有不怀好心的人启动一个进程来无限的创建和打开新的文件,会让服务器崩溃。
3、bootstrap.memory_lock: false;bootstrap.system_call_filter: false配置必须

说明:限制资源使用的配置文件是 /etc/security/limits.conf,和/etc/security/limits.d/目录,
      /etc/security/limits.d/里面配置会覆盖/etc/security/limits.conf的配置
      
# vim /etc/security/limits.conf
elk soft nproc 65536  - 单个用户可用的最大进程数量(超过会警告);
elk hard nproc 65536  - 单个用户可用的最大进程数量(超过会报错);
elk soft nofile 4096  - 可打开的文件描述符的最大数(超过会警告);
elk hard nofile 4096  - 可打开的文件描述符的最大数(超过会报错);
# vim /etc/security/limits.d/20-nproc.conf
----------------------------------------------------------------------------------------------------------------------------------------

# 修改完执行:
# sysctl -p  -- 查看配置

# 防火前&端口配置

查看状态:
firewall-cmd --state
systemctl status firewalld.service

添加端口:
firewall-cmd --get-active-zones
firewall-cmd --list-port  -- 查看已经开放端口
firewall-cmd --zone=public --add-port=9200/tcp --permanent
firewall-cmd --zone=public --add-port=9300/tcp --permanent
firewall-cmd --reload 重启防火墙

firewall-cmd --query-port=9200/tcp
firewall-cmd --query-port=9300/tcp


# 其他:如云部署的需要开放对应服务商的安全组端口策略

【插件安装】

1.ik分词器插件安装

# cd /opt/soft/elk/elasticsearch-7.17.5/plugins
mkdir -p analysis-ik
# wget https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.17.5/elasticsearch-analysis-ik-7.17.5.zip

# 解压插件
# unzip -d ./analysis-ik/ elasticsearch-analysis-ik-7.17.5.zip 


【配置elasticsearch服务端】

1.修改 elasticsearch.yml配置

# cd /opt/soft/elk/elasticsearch-7.17.5/config

# 设置:[集群名称、节点配置、路径配置、内存配置、网络配置、服务发现、安全配置]
cluster.name: search-application
node.name: node-1
path.data: /opt/soft/elk/esdata
path.logs: /opt/soft/elk/eslogs
# 设置为true锁住内存,当服务混合部署了多个组件及服务时,应开启此操作,允许es占用足够多的内存
bootstrap.memory_lock: true
# 设置绑定的ip,设置为0.0.0.0以后就可以让任何计算机节点访问到了
network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts: ["host1", "host2"]
# 设置在集群中的所有节点名称,当然你也可以采用默认的也行,如果是单机,放入当前节点即可
cluster.initial_master_nodes: ["node-1", "node-2"]
# 支持跨域,跨域配置是为了kibana,logstash等组件连接
http.cors.enabled: true
http.cors.allow-origin: "*"
# 关闭地图更新
ingest.geoip.downloader.enabled: false
# es优化,是否支持过滤掉系统调用
bootstrap.system_call_filter: false

2.修改

# 堆栈配置
-Xms500m
-Xmx500m

3.修改elasticsearch-env.bat文件

设置JDK:ES_JAVA_HOME   -- ES7.x推荐使用自带的JDK

【服务启动】

# su elk
# ./elasticsearch -d  - 加-d为后台启动

注意:一定要切换为专用的用户,否则会报错如下:

[root@VM-0-11-centos bin]# ./elasticsearch -d
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
Future versions of Elasticsearch will require Java 11; your Java version from [/usr/local/java/jdk1.8.0_202/jre] does not meet this requirement. Consider switching to a distribution of Elasticsearch with a bundled JDK. If you are already using a distribution with a bundled JDK, ensure the JAVA_HOME environment variable is not set.
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
Future versions of Elasticsearch will require Java 11; your Java version from [/usr/local/java/jdk1.8.0_202/jre] does not meet this requirement. Consider switching to a distribution of Elasticsearch with a bundled JDK. If you are already using a distribution with a bundled JDK, ensure the JAVA_HOME environment variable is not set.
[root@VM-0-11-centos bin]# uncaught exception in thread [main]
java.lang.RuntimeException: can not run elasticsearch as root
        at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:107)
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:183)
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:434)
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:169)
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:160)
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77)
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112)
        at org.elasticsearch.cli.Command.main(Command.java:77)
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:125)
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80)
For complete error details, refer to the log at /opt/soft/elk/eslogs/search-application.log
2022-08-08 05:27:51,939144 UTC [7615] INFO  Main.cc@111 Parent process died - ML controller exiting

# su - elk

【错误解决】

错误一
[elk@VM-0-11-centos ~]$ Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /opt/soft/elk/elasticsearch-7.17.5/config/elasticsearch.keystore
Likely root cause: java.nio.file.AccessDeniedException: /opt/soft/elk/elasticsearch-7.17.5/config/elasticsearch.keystore

查看elasticsearch.keystore文件
[elk@VM-0-11-centos elasticsearch-7.17.5]$ cd config/
[elk@VM-0-11-centos config]$ ll
total 48
-rw-rw---- 1 root root   199 Aug  8 13:27 elasticsearch.keystore
-rw-rw---- 1 elk  elk   1042 Jun 24 05:55 elasticsearch-plugins.example.yml
-rw-rw---- 1 elk  elk   3699 Aug  8 13:24 elasticsearch.yml
-rw-rw---- 1 elk  elk   3277 Aug  8 13:26 jvm.options
drwxr-x--- 2 elk  elk   4096 Jun 24 05:59 jvm.options.d
-rw-rw---- 1 elk  elk  19304 Jun 24 05:58 log4j2.properties
-rw-rw---- 1 elk  elk    473 Jun 24 05:58 role_mapping.yml
-rw-rw---- 1 elk  elk    197 Jun 24 05:58 roles.yml
-rw-rw---- 1 elk  elk      0 Jun 24 05:58 users
-rw-rw---- 1 elk  elk      0 Jun 24 05:58 users_roles


解决:
su root
chown -R elk:elk /opt/soft/elk/elasticsearch-7.17.5/
chown -R elk:elk /opt/soft/elk/elasticsearch-7.17.5/config

# chmod +x bin/elasticsearch

错误二
/opt/soft/elk/eslogs/search-application_index_indexing_slowlog.json (权限不够)

chown -R elk:elk /opt/soft/elk/esdata/*
chown -R elk:elk /opt/soft/elk/eslogs/*


$ ps -ef|grep elasticsearch|grep -v grep 


错误三:: not all primary shards of [.geoip_databases] index are active


解决:

因为他启动时会去更新地图的一些数据库,这里直接禁掉即可,用到时再说,配置文件conf/elasticsearch.yml,增加配置

# 修改:elasticsearch.yml
# ingest.geoip.downloader.enabled: false


错误:
Java HotSpot(TM) 64-Bit Server VM warning: Cannot open file logs/gc.log due to Permission denied


【访问服务】

ip:9200
 

你可能感兴趣的:(centos7,安装,elasticsearch,elk,大数据)