Elasticsearch多master配置及x-pack破解

x-pack是ELK的一个插件,集监控、权限、告警等于一体。支持集群级别、节点级别、索引级别的监控;支持索引、字段级别的权限控制。x-pack目前并不开源，试用期一个月。本着自用方便，尝试破解。
多master下，只需要增加master节点，修改下discovery.zen.minimum_master_nodes参数即可。

结点规划

节点	角色
node1	master-1、x-pack
node2	data-1、x-pack
node3	data-2、x-pack
node4	client-1、kibana、x-pack(client-1)、x-pack(kibana)

搭建elasticsearch集群

增加用户

useradd elk
passwd elk

下载elasticsearch安装包

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.4.tar.gz

解压分发安装包

[elk@node1 ~]$ tar -zxvf elasticsearch-6.2.4.tar.gz
[elk@node1 ~]$ scp -r elasticsearch-6.2.4 elk@node2:~
[elk@node1 ~]$ scp -r elasticsearch-6.2.4 elk@node3:~
[elk@node1 ~]$ scp -r elasticsearch-6.2.4 elk@node4:~

配置master-1

多master配置，只需要增加机器简单修改即可。

[elk@node1 ~]$ vim elasticsearch-6.2.4/config/elasticsearch.yml
    #集群名称
    cluster.name: my-elk

    #节点名称
    node.name: master-1

    #是否是master节点,master节点存元数据
    node.master: true
    #是否是data数据节点,data数据节点存数据
    node.data: false
    #是否是ingest节点,ingest节点可以在数据真正进入index前,通过配置pipline拦截器对数据ETL
    node.ingest: false

    #数据目录,可挂载多个盘
    path.data: /home/elk/elasticsearch-6.2.4/es-data/data
    #日志目录
    path.logs: /home/elk/elasticsearch-6.2.4/es-data/logs/

    #http host和port
    network.host: node1
    http.port: 9200

    #组成该集群的节点
    discovery.zen.ping.unicast.hosts: [node1, node2,node3,node4]

    #防止脑裂配置，注意在多master时，这个值应该等于 Math.floor(master候选节点数/2)+1
    #意思是master候选节点的数目最少达到多少个，才去选举master
    #没有这个配置,在多master时容易造成脑裂,出现多个集群
    #这里只有一个master就设置成1即可
    discovery.zen.minimum_master_nodes: 1

配置data-1

[elk@node2 ~]$ vim elasticsearch-6.2.4/config/elasticsearch.yml
    cluster.name: my-elk

    node.name: data-1

    node.master: false
    node.data: true
    node.ingest: true

    path.data: /home/elk/elasticsearch-6.2.4/es-data/data
    path.logs: /home/elk/elasticsearch-6.2.4/es-data/logs/

    network.host: node2
    http.port: 9200

    discovery.zen.ping.unicast.hosts: ["node1", "node2","node3","node4"]

    discovery.zen.minimum_master_nodes: 1

配置data-2

[elk@node3 ~]$ vim elasticsearch-6.2.4/config/elasticsearch.yml
    cluster.name: my-elk

    node.name: data-2

    node.master: false
    node.data: true
    node.ingest: true

    path.data: /home/elk/elasticsearch-6.2.4/es-data/data
    path.logs: /home/elk/elasticsearch-6.2.4/es-data/logs/

    network.host: node3
    http.port: 9200

    discovery.zen.ping.unicast.hosts: ["node1", "node2","node3","node4"]

    discovery.zen.minimum_master_nodes: 1

配置client-1

[elk@node4 ~]$ vim elasticsearch-6.2.4/config/elasticsearch.yml
    cluster.name: my-elk

    node.name: client-1

    #都设置成false后，此节点就成为了client，起到路由请求和结果聚合的作用。生成环境下应该配置成大内存。
    #client有两种类型 client-coordinate 和 client-tribe
    #client-coordinate 请求路由到elasticsearch节点
    #client-tribe 请求路由到不同elasticsearch集群，需要增加其他配置
    #这里默认是client-coordinate
    node.master: false
    node.data: false
    node.ingest: false

    path.data: /home/elk/elasticsearch-6.2.4/es-data/data
    path.logs: /home/elk/elasticsearch-6.2.4/es-data/logs/

    network.host: node4
    http.port: 9200

    discovery.zen.ping.unicast.hosts: ["node1", "node2","node3","node4"]

    discovery.zen.minimum_master_nodes: 1

配置kibana

下载解压

[elk@node4 ~]$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.4.tar.gz
[elk@node4 ~]$ tar -zxvf kibana-6.2.4-linux-x86_64.tar.gz

配置

[elk@node4 ~]$ vim kibana-6.2.4-linux-x86_64/config/kibana.yml
    server.port: 5601
    server.host: "node4"
    elasticsearch.url: "http://node4:9200"

安装x-pack插件

下载x-pack并分发到各节点

wget https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-6.2.4.zip

es安装x-pack

[elk@node1 ~]$ elasticsearch-6.2.4/bin/elasticsearch-plugin install file:///home/elk/x-pack-6.2.4.zip
[elk@node2 ~]$ elasticsearch-6.2.4/bin/elasticsearch-plugin install file:///home/elk/x-pack-6.2.4.zip
[elk@node3 ~]$ elasticsearch-6.2.4/bin/elasticsearch-plugin install file:///home/elk/x-pack-6.2.4.zip
[elk@node4 ~]$ elasticsearch-6.2.4/bin/elasticsearch-plugin install file:///home/elk/x-pack-6.2.4.zip

kibana安装x-pack

[elk@node4 ~]$ kibana-6.2.4-linux-x86_64/bin/kibana-plugin install file:///home/elk/x-pack-6.2.4.zip

启动elasticsearch

[elk@node1 ~]$ elasticsearch-6.2.4/bin/elasticsearch
[elk@node2 ~]$ elasticsearch-6.2.4/bin/elasticsearch
[elk@node3 ~]$ elasticsearch-6.2.4/bin/elasticsearch
[elk@node4 ~]$ elasticsearch-6.2.4/bin/elasticsearch

启动elasticsearch时异常解决

参考网上的帖子即可
注意：异常解决后,需要重新登录，才可生效。

设置kibana通过x-pack连接client-1的用户名密码

[elk@node4 ~]$ elasticsearch-6.2.4/bin/x-pack/setup-passwords interactive
#kibana.yml中增加用户名密码
[elk@node4 ~]$ vim kibana-6.2.4-linux-x86_64/config/kibana.yml
    elasticsearch.username: "elastic"
    #这里就是上边设置的密码
    elasticsearch.password: "123456"

启动kibana查看Monitoring

[elk@node4 ~]$ kibana-6.2.4-linux-x86_64/bin/kibana
#用上边设置的用户名密码登录
http://node4:5601/
#但试用期只有一个月，自用不放便。

破解x-pack

下载运行反编译软件

[wangpei@localhost ~/gitHub]$ git clone https://github.com/deathmarine/Luyten.git
[wangpei@localhost ~/gitHub/Luyten]$ mvn clean install
[wangpei@localhost ~/gitHub/Luyten]$ java -jar target/luyten-0.5.3.jar

反编译类

反编译elasticsearch-6.2.4/plugins/x-pack/x-pack-core/x-pack-core-6.2.4.jar中的两个类

反编译LicenseVerifier.class

找到org.elasticsearch.license.LicenseVerifier.class，反编译结果拷贝至IDE，建同样的包和类LicenseVerifier.java,内容替换如下(即替换掉License验证部分)

package org.elasticsearch.license;

public class LicenseVerifier
{
    public static boolean verifyLicense(final License license, final byte[] encryptedPublicKeyData) {
        return true;
    }

    public static boolean verifyLicense(final License license) {
        return true;
    }
}

反编译XPackBuild.class

找到org.elasticsearch.xpack.core.XPackBuild.class，反编译结果拷贝至IDE，建同样的包和类XPackBuild.java,内容替换如下

package org.elasticsearch.xpack.core;

import org.elasticsearch.common.io.*;
import java.net.*;
import org.elasticsearch.common.*;
import java.nio.file.*;
import java.io.*;
import java.util.jar.*;

public class XPackBuild
{
    public static final XPackBuild CURRENT;
    private String shortHash;
    private String date;

    @SuppressForbidden(reason = "looks up path of xpack.jar directly")
    static Path getElasticsearchCodebase() {
        final URL url = XPackBuild.class.getProtectionDomain().getCodeSource().getLocation();
        try {
            return PathUtils.get(url.toURI());
        }
        catch (URISyntaxException bogus) {
            throw new RuntimeException(bogus);
        }
    }

    XPackBuild(final String shortHash, final String date) {
        this.shortHash = shortHash;
        this.date = date;
    }

    public String shortHash() {
        return this.shortHash;
    }

    public String date() {
        return this.date;
    }

    static {
        final Path path = getElasticsearchCodebase();
        String shortHash = null;
        String date = null;
        Label_0157: {
            shortHash = "Unknown";
            date = "Unknown";
        }
        CURRENT = new XPackBuild(shortHash, date);
    }
}

编译并替换

#编译用到的依赖来自于elasticsearch-6.2.4/lib目录

#编译LicenseVerifier.java
javac -cp "elasticsearch-6.2.4.jar:lucene-core-7.2.1.jar:x-pack-core-6.2.4.jar:elasticsearch-core-6.2.4.jar" LicenseVerifier.java

#编译XPackBuild.java
javac -cp "elasticsearch-6.2.4.jar:lucene-core-7.2.1.jar:x-pack-core-6.2.4.jar:elasticsearch-core-6.2.4.jar" XPackBuild.java

#替换LicenseVerifier.class
mkdir -p org/elasticsearch/license/
cp LicenseVerifier.class org/elasticsearch/license
jar uf x-pack-core-6.2.4.jar org/elasticsearch/license/LicenseVerifier.class

#替换XPackBuild.class
mkdir -p org/elasticsearch/xpack/core
cp XPackBuild.class org/elasticsearch/xpack/core
jar uf x-pack-core-6.2.4.jar org/elasticsearch/xpack/core/XPackBuild.class

替换破解后的jar包

用上边得到的破解包x-pack-core-6.2.4.jar替换所有elasticsearch节点中，elasticsearch-6.2.4/plugins/x-pack/x-pack-core目录下的x-pack-core-6.2.4.jar

上传授权文件

#(1)需要在所有elasticsearch节点elasticsearch-6.2.4/config/elasticsearch.yml增加配置项，用于上传授权文件
xpack.security.enabled: false

#(2)申请licene授权文件并在邮箱中下载
https://license.elastic.co/registration

#(3)修改type 为platinum 铂金 有效期到2050年 "expiry_date_in_millis":2524579200999

#(4)启动4台elasticsearch

#(5)4个节点替换licene
curl -XPUT -u elastic:changeme 'http://node1:9200/_xpack/license' -H "Content-Type: application/json" -d @license.json

制作SSL

#(1)master节点生成证书
[elk@node1 elasticsearch-6.2.4]$ pwd
    /home/elk/elasticsearch-6.2.4
[elk@node1 elasticsearch-6.2.4]$ bin/x-pack/certutil ca

#这里需要设置密码,后边会用到
[elk@node1 elasticsearch-6.2.4]$ bin/x-pack/certutil cert --ca elastic-stack-ca.p12

[elk@node1 elasticsearch-6.2.4]$ mkdir config/certs
[elk@node1 elasticsearch-6.2.4]$ cp elastic-certificates.p12 config/certs

#(2)证书拷贝至所有elasticsearch节点
[elk@node1 elasticsearch-6.2.4]$ scp -r config/certs/ elk@node2:~/elasticsearch-6.2.4/config/
[elk@node1 elasticsearch-6.2.4]$ scp -r config/certs/ elk@node3:~/elasticsearch-6.2.4/config/
[elk@node1 elasticsearch-6.2.4]$ scp -r config/certs/ elk@node4:~/elasticsearch-6.2.4/config/

#(3)所有elasticsearch节点启用SSL
elasticsearch.yml中增加配置
#xpack.security.enabled: false
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12

#(4)所有elasticsearch节点和kibana节点将密码添加至elasticsearch-keystore
#注意:这里输入的密码是生成证书时设置的密码
[elk@node1 elasticsearch-6.2.4]$ bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
[elk@node1 elasticsearch-6.2.4]$ bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

重启所有elasticsearch节点和kibana节点

查看有效期

登录后可以看到，有效期到2050年 Your Platinum license will expire on January 1, 2050.

查看数据

#(1)创建索引
PUT user_web_info

#(2)设置mapping
PUT user_web_info/_mapping/user
{
  "properties": {
    "uuid":{"type":"long"},
    "name":{"type":"text","fields":{"keyword":{"type": "keyword"}}},
    "nickname":{"type":"text"},
    "age":{"type":"integer"},
    "dt":{"type":"date","format": "yyyy-MM-dd"}
  }
}

#(3)插入数据
POST user_web_info/user/_bulk
{ "create": {"_id": "1" }}
{"uuid":1,"name":"jack chen","nickname":"apple pear","age":20,"dt":"2016-06-25"}
{ "create": {"_id": "2" }}
{"uuid":2,"name":"jack ma","nickname":"apple pear pear","age":22,"dt":"2016-08-23"}
{ "create": {"_id": "3" }}
{"uuid":3,"name":"lucy","nickname":"apple pear apple","age":23,"dt":"2016-08-25"}

#(4)创建Index Patterns
Management=>Index Patterns=>user_web_info

#(5)Discover页浏览数据