Elasticsearch系列-基础篇(二):环境搭建

环境准备

  • Centos 7.2.1511
  • Oracle JDK1.8.0_181
  • Elasticsearch 6.3.1

软件下载

JDK历史版本下载:

进入jdk下载页面:
https://www.oracle.com/technetwork/java/javase/downloads/index.html
拉到最底下 Java Archive档案馆,点击download会列出jdk的历史版本,

jdk下载.png

或者直接进入到历史版本列表页:
https://www.oracle.com/technetwork/java/javase/archive-139210.html
点击需要的版本进入详细的小版本列表,选择需要的小版本下载即可。

image.png

PS:下载需要Oracle账户,网上搜即可,提供一个可用的账号,用户名:[email protected] 密码:OracleTest1234

JDK安装:

查看现有JDK版本:

[root@host test]# java -version
openjdk version "1.8.0_201"
OpenJDK Runtime Environment (build 1.8.0_201-b09)
OpenJDK 64-Bit Server VM (build 25.201-b09, mixed mode)

替换掉open jdk为oracle jdk,安装oracle jdk

rpm -ivh jdk-8u181-linux-x64.rpm

使用linux软件版本管理命令update-alternatives查看现有java版本:

[root@hostname test]# update-alternatives --display java
java - status is auto.
 link currently points to /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/jre/bin/java
/usr/java/jdk1.8.0_181-amd64/jre/bin/java - priority 180181
 slave ControlPanel: /usr/java/jdk1.8.0_181-amd64/jre/bin/ControlPanel
 slave javaws: /usr/java/jdk1.8.0_181-amd64/jre/bin/javaws
 ...
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/jre/bin/java - family java-1.8.0-openjdk.x86_64 priority 1800212
 slave ControlPanel: (null)
 slave javaws: (null)
 slave jcontrol: (null)
 slave jjs: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/jre/bin/jjs
 ...
/usr/java/jre1.8.0_181-amd64/bin/java - priority 180181
 slave ControlPanel: /usr/java/jre1.8.0_181-amd64/bin/ControlPanel
 slave javaws: /usr/java/jre1.8.0_181-amd64/bin/javaws
 slave jcontrol: /usr/java/jre1.8.0_181-amd64/bin/jcontrol
 ...
Current `best' version is /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/jre/bin/java.

输出的内容省略了一部分,可以看到,系统中存在OpenJDK 和 Oracle的JDK,下面选择Oracle JDK为默认

[root@hostname test]# update-alternatives --config java

There are 3 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
   1           /usr/java/jdk1.8.0_181-amd64/jre/bin/java
*+ 2           java-1.8.0-openjdk.x86_64 (/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/jre/bin/java)
   3           /usr/java/jre1.8.0_181-amd64/bin/java

Enter to keep the current selection[+], or type selection number: 1
[root@hostname test]# java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

选择1之后,查看java版本,已经是oracle的jdk了

[root@hostname test]# java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

Elasticsearch 安装

下载Elasticsearch 6.3.1:

curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz

解压缩至/usr/local/elasticserch下:

tar -xvf elasticsearch-6.3.1.tar.gz && mv elasticsearch-6.3.1 /usr/local/elasticsearch

创建数据及日志存储目录,统一放到/data/elasticsearch下:

mkdir /data/elasticsearch/data
mkdir /data/elasticsearch/log

配置/usr/local/elasticsearch/config/elasticsearch.yml,修改配置如下:

# 集群名称 (所有节点必须一致)
cluster.name: datacube-test-cluster
# 节点名称
node.name: node-1
# 数据存放目录
path.data: /data/elasticsearch/data
# 日志存放目录
path.logs: /data/elasticsearch/logs
# 监听地址
network.host: 0.0.0.0
# 监听端口(默认9200)
http.port: 9200
# 配置单播地址
discovery.zen.ping.unicast.hosts: ["192.168.0.1", "192.168.0.2", "192.168.0.3", "192.168.0.4", "192.168.0.5"]
# 设置选举master需要的最少节点数(推荐设置为:可成为master节点数/2 +1)
# 如果没有这种设置,遭受网络故障的集群就有可能将集群分成两个独立的集群 - 俗称脑裂 - 这将导致数据丢失
discovery.zen.minimum_master_nodes: 3

集群各节点采用相同配置,其中集群名称须一致,节点名称取各节点hostname即可

启动配置:

elasticsearch默认是禁止使用root用户启动的,需要创建其他用户, 创建elastic用户:

useradd elastic

赋予es相关文件夹权限:

chown -R elastic:elastic /data/elasticsearch/ && chown -R elastic:elastic /usr/local/elasticsearch/

切换至elastic用户,启动ES:

[root@hostname test]# su - elastic
[elastic@hostname test]# /usr/local/elasticsearch/bin/elasticsearch -d

启动过程中会报以下错误:

[1]: max file descriptors [65535] for elasticsearch process is too low, increase to at least [65536]
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

切换至root账户,配置如下内容:
1、修改/etc/security/limits.conf,修改以下内容:

* soft nofile 65536
* hard nofile 65536

2、修改/etc/sysctl.conf,添加配置:

vm.max_map_count=655360

最后执行sysctl -p使配置生效,切换至elastic用户,启动各节点es:

/usr/local/elasticsearch/bin/elasticsearch -d

查看集群健康状态,5个节点已正常启动:

[root@hostname test]# curl 'http://localhost:9200/_cluster/health?pretty'
{
  "cluster_name" : "datacube-test-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 5,
  "number_of_data_nodes" : 5,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

X-PACK破解

1、启用测试版本:因为elasticsearch在6.3版本之后x-pack是默认安装好的,所以不再需要用户自己去安装,es启动后直接启用即可:

curl -H "Content-Type:application/json" -XPOST  http://localhost:9200/_xpack/license/start_trial?acknowledge=true&pretty

返回如下信息表示启用测试版成功:

{
    "acknowledged":true,
    "trial_was_started":true,
    "type":"trial"
}

2、进入/tmp目录创建LicenseVerifier.java、XPackBuild.java文件

package org.elasticsearch.license; 
import java.nio.*; import java.util.*; 
import java.security.*; 
import org.elasticsearch.common.xcontent.*; 
import org.apache.lucene.util.*; 
import org.elasticsearch.common.io.*; 
import java.io.*; 
 
public class LicenseVerifier { 
    public static boolean verifyLicense(final License license, final byte[] encryptedPublicKeyData) {
        return true; 
    } 
    
    public static boolean verifyLicense(final License license)     { 
        return true; 
    } 
}
package org.elasticsearch.xpack.core;
import org.elasticsearch.common.io.*;
import java.net.*;
import org.elasticsearch.common.*;
import java.nio.file.*;
import java.io.*; 
import java.util.jar.*; 
public class XPackBuild { 
    public static final XPackBuild CURRENT;
    private String shortHash; 
    private String date; 
    @SuppressForbidden(reason = "looks up path of xpack.jar directly") static Path getElasticsearchCodebase() { 
        final URL url = XPackBuild.class.getProtectionDomain().getCodeSource().getLocation();
        try { return PathUtils.get(url.toURI()); }
        catch (URISyntaxException bogus) { 
            throw new RuntimeException(bogus); } 
        } 
        
    XPackBuild(final String shortHash, final String date) {
            this.shortHash = shortHash; 
            this.date = date; 
            } 
            
    public String shortHash() {
        return this.shortHash;
        } 
    public String date(){ 
        return this.date; 
        }
        
    static { 
        final Path path = getElasticsearchCodebase();
        String shortHash = null; 
        String date = null;
        Label_0157: { shortHash = "Unknown"; date = "Unknown"; 
    } 
    
    CURRENT = new XPackBuild(shortHash, date); 
    }
}

3、分别编译这两个文件生成LicenseVerifier.class和XPackBuild.class两个文件

javac -cp "/usr/local/elasticsearch/lib/elasticsearch-6.3.1.jar:/usr/local/elasticsearch/lib/lucene-core-7.3.1.jar:/usr/local/elasticsearch/modules/x-pack/x-pack-core/x-pack-core-6.3.1.jar" /tmp/LicenseVerifier.java
javac -cp "/usr/local/elasticsearch/lib/elasticsearch-6.3.1.jar:/usr/local/elasticsearch/lib/lucene-core-7.3.1.jar:/usr/local/elasticsearch/modules/x-pack/x-pack-core/x-pack-core-6.3.1.jar:/usr/local/elasticsearch/lib/elasticsearch-core-6.3.1.jar"  XPackBuild.java

4、覆盖ES安装包下的x-pack-core-6.3.1.jar

cp /usr/local/elasticsearch/modules/x-pack/x-pack-core/x-pack-core-6.3.1.jar /tmp
# 解压x-pack-core-6.3.1.jar
jar -xf x-pack-core-6.3.1.jar
# 将编译后的文件覆盖原文件
mv LicenseVerifier.class org/elasticsearch/license/
mv XPackBuild.class org/elasticsearch/xpack/core/
# 删除/tmp下的无用文件
rm -rf x-pack-core-6.3.1.jar LicenseVerifier.java XPackBuild.java
# 重新打包
jar -cvf x-pack-core-6.3.1.jar *
# 覆盖es安装目录下的jar
cp x-pack-core-6.3.1.jar /usr/local/elasticsearch/modules/x-pack/x-pack-core/

5、修改elasticsearch.yml,打开x-pach安全验证

#添加如下代码
xpack.security.enabled: true

6、使用es命令生成用户名、密码:

# 自动生成密码(二选一)
/usr/local/elasticsearch/bin/elasticsearch-setup-passwords auto
# 手动生成密码(二选一)
/usr/local/elasticsearch/bin/elasticsearch-setup-passwords interactive

自动生成密码会将logstash、beat、kibana、es的用户名密码全部生成

7、配置SSL

/usr/local/elasticsearch/bin/elasticsearch-certgen

执行上述命令会出现如下信息,按要求填写:

******************************************************************************
Note: The 'elasticsearch-certgen' tool has been deprecated in favour of the
      'elasticsearch-certutil' tool. This command will be removed in a future
      release.
******************************************************************************

This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL in the Elastic stack. Depending on the command
line option specified, you may be prompted for the following:

* The path to the output file
    * The output file is a zip file containing the signed certificates and
      private keys for each instance. If a Certificate Authority was generated,
      the certificate and private key will also be included in the output file.
* Information about each instance
    * An instance is any piece of the Elastic Stack that requires a SSL certificate.
      Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beats
      may all require a certificate and private key.
    * The minimum required value for each instance is a name. This can simply be the
      hostname, which will be used as the Common Name of the certificate. A full
      distinguished name may also be used.
    * A filename value may be required for each instance. This is necessary when the
      name would result in an invalid file or directory name. The name provided here
      is used as the directory name (within the zip) and the prefix for the key and
      certificate files. The filename is required if you are prompted and the name
      is not displayed in the prompt.
    * IP addresses and DNS names are optional. Multiple values can be specified as a
      comma separated string. If no IP addresses or DNS names are provided, you may
      disable hostname verification in your SSL configuration.
* Certificate Authority private key password
    * The password may be left empty if desired.

Let's get started...

Please enter the desired output file [certificate-bundle.zip]: cert.zip  (需要输入生成的压缩包名称)
Enter instance name: my-application(需要输入实例名)
Enter name for directories and files [p4mES]: elasticsearch(需要输入文件夹名)
Enter IP Addresses for instance (comma-separated if more than one) []: 127.0.0.1  (实例ip,多个ip用逗号隔开,输入集群所有节点的IP)
Enter DNS names for instance (comma-separated if more than one) []: node-1(节点名,多个节点用逗号隔开,输入所有的节点名称)
Would you like to specify another instance? Press 'y' to continue entering instance information: 
Certificates written to /usr/local/elasticsearch/bin/cert.zip(这个是生成的文件存放地址,不用填写,在哪个目录下执行elasticsearch-certgen就会生成在哪个目录,不要在/usr/local/elasticsearch/bin下执行elasticsearch-certgen,否则你上面需要填写文件夹时不能写elasticsearch,因为解压时和elasticsearch命令冲突)

This file should be properly secured as it contains the private keys for all
instances and the certificate authority.

After unzipping the file, there will be a directory for each instance containing
the certificate and private key. Copy the certificate, key, and CA certificate
to the configuration directory of the Elastic product that they will be used for
and follow the SSL configuration instructions in the product guide.

For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.

把生成的zip包解压,将里面的ca.crt、ca.key、elasticsearch.crt、elasticsearch.key放到/usr/local/elasticsearch/config目录下,然后修改elasticsearch.yml添加以下内容:

vim /usr/local/elasticsearch/config/elasticsearch.yml
# 添加如下变量
xpack.security.transport.ssl.enabled: true
xpack.ssl.key: elasticsearch.key
xpack.ssl.certificate: elasticsearch.crt
xpack.ssl.certificate_authorities: ca.crt

8、校验是否禁用ipv6

vim /etc/sysctl.conf
# 添加如下变量
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
# 保存退出
# 使修改生效
sysctl -p

9、破解,上传license.json文件:

{
    "license": {
        "uid": "9gfhf46-5g78-4f1e-b5a4-afet359bc3a3",
        "type": "platinum",
        "issue_date_in_millis": 1534723200000,
        "expiry_date_in_millis": 2544271999999,
        "max_nodes": 100,
        "issued_to": "www.plaza4me.com",
        "issuer": "Web Form",
        "signature": "AAAAAwAAAA3lQFlr4GED3cGRsdfgrDDFEWGN0hjZDBGYnVyRXpCOsdfasdfsgEfghgdg3423MVZwUzRxVk1PSmkxagfsdf3242UWh3bHZVUTllbXNPbzBUemtnbWpBbmlWRmRZb25KNFlBR2x0TXc2K2p1Y1VtMG1UQU9TRGZVSGRwaEJGUjE3bXd3LzRqZ05iLzRteWFNekdxRGpIYlFwYkJiNUs0U1hTVlJKNVlXekMrSlVUdFIvV0FNeWdOYnlESDc3MWhlY3hSQmdKSjJ2ZTcvYlBFOHhPQlV3ZHdDQ0tHcG5uOElCaDJ4K1hob29xSG85N0kvTWV3THhlQk9NL01VMFRjNDZpZEVXeUtUMXIyMlIveFpJUkk2WUdveEZaME9XWitGUi9WNTZVQW1FMG1DenhZU0ZmeXlZakVEMjZFT2NvOWxpZGlqVmlHNC8rWVVUYzMwRGVySHpIdURzKzFiRDl4TmM1TUp2VTBOUlJZUlAyV0ZVL2kvVk10L0NsbXNFYVZwT3NSU082dFNNa2prQ0ZsclZ4NTltbU1CVE5lR09Bck93V2J1Y3c9PQAAAQCGcZtOlZwj0Rnl2MUjERG94a+xcifpVAurIA+z4rroxaqaewpb2MJLZVJt1ZCGeKB0KIWRAm2pkPjM2JigjaPIUBhpW4/yUzbdRtRuQB4loEKd7/p9EbHDh5GzeI8qfkMh3j7QaAlz4Bk+eett+ZNqNXHEdkr+Re9psdnqfUESz1uROhMoYWbn/Bdd0AJLKzhRnEOE972xdnAar8bCP1DIDljI9IOnYhEc6O6CboKCMJY4AWOvJY83bud4FO25hrKf6bMy0F2oO2yUkVV0UiFMX19JbhcC+WIAgxMk/KG7e/MqR8bJ1jNu2usMlgkvV97BxiPogTujFnTQxoHdpNdR",
        "start_date_in_millis": 1534723200000
    }
}

把license.json上传到服务器并使用curl提交:

curl -XPUT -u elastic 'http://127.0.0.1:9200/_xpack/license' -H "Content-Type: application/json" -d @license.json

10、将生成的elastic密码配置到kibana中

cd /usr/local/kibana/config
vim kibana.yml
# 使用生成密码步骤生成的kibana用户名密码
elasticsearch.username: kibana
elasticsearch.password: XXXXXXXXXXX

启动完成后访问kibana
在登陆成功后的主页面Management->LicenseManagement可以看到使用期限为2025年。

参考链接:https://blog.csdn.net/qq_25475209/article/details/81906701

你可能感兴趣的:(Elasticsearch系列-基础篇(二):环境搭建)