Elasticsearch/Enterprise Search/Kibana安装记录

参考文章：https://www.elastic.co/guide/en/elasticsearch/reference/8.8/deb.html
服务器系统：Ubuntu 20.04.4 LTS (GNU/Linux 5.4.0-153-generic x86_64)
elasticsearch版本：8.8.2(软件包发行版)

Elasticsearch的安装

导入 elasticsearch PGP密钥

我们使用 Elasticsearch 签名密钥（PGP 密钥D88E42B4，可从https://pgp.mit.edu获取）和指纹对所有包进行签名 ：

4609 5ACC 8548 582C 1A26 99A9 D27D 666C D88E 42B4

下载并安装公共签名密钥：

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

安装

使用APT安装

在使用apt安装前，先需要安装apt-transport-https

sudo apt-get install apt-transport-https

将存储库定义保存到/etc/apt/sources.list.d/elastic-8.x.list

echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list

使用以下命令安装 Elasticsearch

sudo apt-get update && sudo apt-get install elasticsearch

手动下载安装

首先，先到elasticsearch查找你所需要的下载的版本链接。

然后使用下面语句下载

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.8.2-amd64.deb
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.8.2-amd64.deb.sha512

比较下载的Debian软件包的 SHA 和发布的SHA是否一致

root@qhdata-dev:/home/qhdata/elasticsearch# shasum -a 512 -c elasticsearch-8.8.2-amd64.deb.sha512 
elasticsearch-8.8.2-amd64.deb: OK
root@qhdata-dev:/home/qhdata/elasticsearch# 

安装

sudo dpkg -i elasticsearch-8.8.2-amd64.deb

启动elasticsearch安全功能

安装 Elasticsearch 时，默认启用并配置安全功能。安装 Elasticsearch 时，会自动进行以下安全配置：

• elastic启用身份验证和授权，并为内置超级用户 生成密码。
• 为传输层和 HTTP 层生成 TLS 的证书和密钥，并使用这些密钥和证书启用和配置 TLS。

密码、证书和密钥将输出到您的终端。例如：

root@qhdata-dev:/home/qhdata/elasticsearch# sudo dpkg -i elasticsearch-8.8.2-amd64.deb
Selecting previously unselected package elasticsearch.
(Reading database ... 153850 files and directories currently installed.)
Preparing to unpack elasticsearch-8.8.2-amd64.deb ...
Unpacking elasticsearch (8.8.2) ...
Setting up elasticsearch (8.8.2) ...
--------------------------- Security autoconfiguration information ------------------------------

Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.

The generated password for the elastic built-in superuser is : 

If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token '
after creating an enrollment token on your existing cluster.

You can complete the following actions at any time:

Reset the password of the elastic built-in superuser with 
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.

Generate an enrollment token for Kibana instances with 
 '/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.

Generate an enrollment token for Elasticsearch nodes with 
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.

-------------------------------------------------------------------------------------------------
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
 sudo systemctl start elasticsearch.service
root@qhdata-dev:/home/qhdata/elasticsearch# 

将Security autoconfiguration information的信息保存在security_autoconfiguration_information.cfg中，方便后续查看。

重新配置节点以加入现有集群

安装 Elasticsearch 时，安装过程默认配置单节点集群。如果您希望节点加入现有集群，请在首次启动新节点之前在现有节点上生成注册令牌。

在现有集群中的任何节点上，生成节点注册令牌：

/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node

复制注册令牌，该令牌将输出到您的终端。
在新的 Elasticsearch 节点上，将注册令牌作为参数传递给该 elasticsearch-reconfigure-node工具：

/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token 

Elasticsearch 现在已配置为加入现有集群。
使用启动新节点systemd

启用系统索引的自动创建功能

一些商业功能会自动在 Elasticsearch 中创建索引。默认情况下，Elasticsearch 配置为允许自动创建索引，无需执行任何其他步骤。但是，如果您在 Elasticsearch 中禁用了自动索引创建，则必须进行配置 action.auto_create_index以elasticsearch.yml允许商业功能创建以下索引：

action.auto_create_index: .monitoring*,.watches,.triggered_watches,.watcher-history*,.ml*

运行Elasticsearch(在systemd下)

要将 Elasticsearch 配置为在系统启动时自动启动，请运行以下命令：

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service

Elasticsearch 可以按如下方式启动和停止：

sudo systemctl start elasticsearch.service  
sudo systemctl stop elasticsearch.service

这些命令不提供有关 Elasticsearch 是否成功启动的反馈。相反，此信息将写入位于/var/log/elasticsearch/.
如果您的 Elasticsearch 密钥库受密码保护，则需要 systemd使用本地文件和 systemd 环境变量提供密钥库密码。该本地文件在存在时应受到保护，并且一旦 Elasticsearch 启动并运行即可安全删除。

echo "keystore_password" > /path/to/my_pwd_file.tmp
chmod 600 /path/to/my_pwd_file.tmp
sudo systemctl set-environment ES_KEYSTORE_PASSPHRASE_FILE=/path/to/my_pwd_file.tmp
sudo systemctl start elasticsearch.service

检查Elasticsearch是否正在运行

您可以通过向localhost:9200发送 HTTPS 请求来测试您的 Elasticsearch 节点是否正在运行：

curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic https://localhost:9200 

然后输入安装期间生成的超级用户elastic的密码。返回如下。

root@qhdata-dev:/home/qhdata/elasticsearch# curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic https://localhost:9200
Enter host password for user 'elastic':
{
  "name" : "qhdata-dev",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "un55kUpqQ9iFGEfp5UUQ5g",
  "version" : {
    "number" : "8.8.2",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "98e1271edf932a480e4262a471281f1ee295ce6b",
    "build_date" : "2023-06-26T05:16:16.196344851Z",
    "build_snapshot" : false,
    "lucene_version" : "9.6.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}
root@qhdata-dev:/home/qhdata/elasticsearch# 

Elasticsearch配置

外网访问

修改/etc/elasticsearch/elasticsearch.yml

然后重启服务，在外网页中输入https://:9200，进入网页后会提示输入用户跟密码，我这里使用的是超级用户elastic与对应的密码。然后即可登陆插卡该elasticsearch的信息。

第三方包安装

elasticsearch-analysis-ik中文分词

官方网址：https://github.com/medcl/elasticsearch-analysis-ik
首先，在https://github.com/medcl/elasticsearch-analysis-ik/releases中找到对应的版本。

然后使用elasticsearch-plugin安装（从版本v5.5.1开始支持）

cd /usr/share/elasticsearch/bin
./elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v8.8.2/elasticsearch-analysis-ik-8.8.2.zip

安装完后再plugins中可以查看到对应的包

root@qhdata-dev:/usr/share/elasticsearch/bin# cd /usr/share/elasticsearch/plugins
root@qhdata-dev:/usr/share/elasticsearch/plugins# ls -r
analysis-ik
root@qhdata-dev:/usr/share/elasticsearch/plugins# 

重启elasticsearch

Enterprise Search的安装

参考文章：https://www.elastic.co/cn/downloads/enterprise-search
服务器系统：Ubuntu 20.04.4 LTS (GNU/Linux 5.4.0-153-generic x86_64)
enterprise search版本：8.8.2(软件包发行版)

安装

首先，先到enterprise-search查找你所需要的下载的版本链接。

然后使用下面语句下载

wget https://artifacts.elastic.co/downloads/enterprise-search/enterprise-search-8.8.2.deb
wget https://artifacts.elastic.co/downloads/enterprise-search/enterprise-search-8.8.2.deb.sha512

比较下载的Debian软件包的 SHA 和发布的SHA是否一致

root@qhdata-dev:/home/qhdata/elasticsearch# shasum -a 512 -c enterprise-search-8.8.2.deb.sha512
enterprise-search-8.8.2.deb: OK
root@qhdata-dev:/home/qhdata/elasticsearch# 

安装

sudo dpkg -i enterprise-search-8.8.2.deb

启动elasticsearch并查找http证书

当您第一次启动 Elasticsearch 时，会自动进行以下安全配置：

• elastic启用身份验证和授权，并为内置超级用户 生成密码。
• 为传输层和 HTTP 层生成 TLS 的证书和密钥，并使用这些密钥和证书启用和配置 TLS。

密码、证书和密钥将输出到您的终端。
http_ca.crt在单独的终端中，找到Elasticsearch 创建的文件的绝对路径名。然后使用find指令查找。

root@qhdata-dev:/etc/elasticsearch# find $(pwd) -name 'http_ca.crt'
/etc/elasticsearch/certs/http_ca.crt
root@qhdata-dev:/etc/elasticsearch# 

然后记录下证书的路径

enterprise search配置

修改/usr/share/enterprise-search/config/enterprise-search.yml

然后使用bin/enterprise-search执行，第一次由于没有设置secret_management.encryption_keys会提示报错，同时会产生对应的key。

root@qhdata-dev:/usr/share/enterprise-search# bin/enterprise-search
Found java executable in PATH
Java version detected: 11.0.19 (major version: 11)
Enterprise Search is starting...
ERROR: ld.so: object 'libjemalloc.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.

[2023-07-20T08:00:59.383+00:00][337822][4004][app-server][INFO]: Elastic Enterprise Search version=8.8.2, JRuby version=9.3.3.0, Ruby version=2.6.8, Rails version=6.0.6.1
[2023-07-20T08:01:00.862+00:00][337822][4004][app-server][ERROR]: 
--------------------------------------------------------------------------------

Invalid config file (/usr/share/enterprise-search/config/enterprise-search.yml):
The setting '#/secret_management/encryption_keys' is not valid
No secret management encryption keys were provided.
Your secrets cannot be stored unencrypted.
You can use the following generated encryption key in your config file to store new encrypted secrets:

secret_management.encryption_keys: [53aaf2997a5d38b53296aab7d310a102d7514322e39b6f8a4d94bcea717fa558]


--------------------------------------------------------------------------------


root@qhdata-dev:/usr/share/enterprise-search# root@qhdata-dev:/usr/share/enterprise-search# 

这时候再修改即可。

运行enterprise search(在systemd下)

参考文章：https://discuss.elastic.co/t/how-to-keep-enterprisesearch-running-all-the-time/274448/14
首先，先创建enterprise-search.service文件

sudo vim /etc/systemd/system/enterprise-search.service

[Unit]
Description = enterprise search

[Service]
ExecStart = /usr/share/enterprise-search/bin/enterprise-search
Restart = always
Type = simple

[Install]
WantedBy = multi-user.target

然后运行以下命令：

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable enterprise-search.service

enterprise search可以按如下方式启动和停止：

sudo systemctl start enterprise-search.service
sudo systemctl stop enterprise-search.service

这些命令不提供有关 enterprise-search是否成功启动的反馈。可以通过journalctl -u enterprise-search.service访问日志信息

Kibana的安装

参考文章：https://www.elastic.co/guide/en/kibana/8.8/deb.html#deb-repo
服务器系统：Ubuntu 20.04.4 LTS (GNU/Linux 5.4.0-153-generic x86_64)
kibana版本：8.8.2(软件包发行版)

安装

导入 Elastic PGP密钥

我们使用 Elastic 签名密钥（PGP 密钥D88E42B4，可从https://pgp.mit.edu获取）和指纹对所有包进行签名 ：

4609 5ACC 8548 582C 1A26 99A9 D27D 666C D88E 42B4

下载并安装公共签名密钥：

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

安装

使用APT安装

在使用apt安装前，先需要安装apt-transport-https

sudo apt-get install apt-transport-https

将存储库定义保存到/etc/apt/sources.list.d/elastic-8.x.list

echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list

使用以下命令安装 Kibana

sudo apt-get update && sudo apt-get install kibana

手动下载安装

首先，先到Kibana查找你所需要的下载的版本链接。

然后使用下面语句下载

wget https://artifacts.elastic.co/downloads/kibana/kibana-8.8.2-amd64.deb
wget https://artifacts.elastic.co/downloads/kibana/kibana-8.8.2-amd64.deb.sha512

比较下载的Debian软件包的 SHA 和发布的SHA是否一致

root@qhdata-dev:/home/qhdata/elasticsearch# shasum -a 512 -c kibana-8.8.2-amd64.deb.sha512 
kibana-8.8.2-amd64.deb: OK
root@qhdata-dev:/home/qhdata/elasticsearch# 

安装

root@qhdata-dev:/home/qhdata/elasticsearch# sudo dpkg -i kibana-8.8.2-amd64.deb
Selecting previously unselected package kibana.
(Reading database ... 155149 files and directories currently installed.)
Preparing to unpack kibana-8.8.2-amd64.deb ...
Unpacking kibana (8.8.2) ...
Setting up kibana (8.8.2) ...
Creating kibana group... OK
Creating kibana user... OK
Created Kibana keystore in /etc/kibana/kibana.keystore
root@qhdata-dev:/home/qhdata/elasticsearch# 

启动elasticsearch并生成kibana的注册令牌

当您第一次启动 Elasticsearch 时，会自动进行以下安全配置：

• elastic启用身份验证和授权，并为内置超级用户 生成密码。
• 为传输层和 HTTP 层生成 TLS 的证书和密钥，并使用这些密钥和证书启用和配置 TLS。

密码、证书和密钥将输出到您的终端。
然后，您可以使用elasticsearch-create-enrollment-token生成 Kibana 的注册令牌

/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana

将注册令牌保存在elasticsearch_create_enrollment_token_kibana.cfg中，方便后续查看。

启动 Kibana 并输入注册令牌以将 Kibana 与 Elasticsearch 安全连接。

运行kibana(在systemd下)

要将 Kibana 配置为在系统启动时自动启动，请运行以下命令：

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable kibana.service

Kibana 可以按如下方式启动和停止：

sudo systemctl start kibana.service
sudo systemctl stop kibana.service

这些命令不提供有关 Kibana 是否成功启动的反馈。可以通过journalctl -u kibana.service访问日志信息

kibana配置

外网访问

修改/etc/kibana/kibana.yml

然后重启服务，在外网页中输入http://:5601，第一次进入网页后会提示输入enrollment token

然后弹出下面的页面
ps: 这个页面有毒，只有火狐浏览器才能成功弹出。

然后在服务器中查看kibana日志，获得该code

root@qhdata-dev:/home/qhdata/elasticsearch# journalctl -u kibana.service
Jul 19 11:58:16 qhdata-dev kibana[207363]: [2023-07-19T11:58:16.566+08:00][INFO ][plugins-system.preboot] Stopping all plugins.
Jul 19 11:58:16 qhdata-dev systemd[1]: kibana.service: Succeeded.
Jul 19 11:58:16 qhdata-dev systemd[1]: Stopped Kibana.
Jul 19 11:58:19 qhdata-dev systemd[1]: Started Kibana.
Jul 19 11:58:21 qhdata-dev kibana[209136]: [2023-07-19T11:58:21.973+08:00][INFO ][node] Kibana process configured with roles: [background_tasks, ui]
Jul 19 11:58:36 qhdata-dev kibana[209136]: [2023-07-19T11:58:35.887+08:00][INFO ][plugins-service] Plugin "cloudChat" is disabled.
Jul 19 11:58:36 qhdata-dev kibana[209136]: [2023-07-19T11:58:35.890+08:00][INFO ][plugins-service] Plugin "cloudExperiments" is disabled.
Jul 19 11:58:36 qhdata-dev kibana[209136]: [2023-07-19T11:58:35.890+08:00][INFO ][plugins-service] Plugin "cloudFullStory" is disabled.
Jul 19 11:58:36 qhdata-dev kibana[209136]: [2023-07-19T11:58:35.890+08:00][INFO ][plugins-service] Plugin "cloudGainsight" is disabled.
Jul 19 11:58:36 qhdata-dev kibana[209136]: [2023-07-19T11:58:35.959+08:00][INFO ][plugins-service] Plugin "profiling" is disabled.
Jul 19 11:58:36 qhdata-dev kibana[209136]: [2023-07-19T11:58:36.147+08:00][INFO ][http.server.Preboot] http server running at http://0.0.0.0:5601
Jul 19 11:58:36 qhdata-dev kibana[209136]: [2023-07-19T11:58:36.355+08:00][INFO ][plugins-system.preboot] Setting up [1] plugins: [interactiveSetup]
Jul 19 11:58:36 qhdata-dev kibana[209136]: [2023-07-19T11:58:36.358+08:00][INFO ][preboot] "interactiveSetup" plugin is holding setup: Validating Elasticsearch connection configuration…
Jul 19 11:58:36 qhdata-dev kibana[209136]: [2023-07-19T11:58:36.402+08:00][INFO ][root] Holding setup until preboot stage is completed.
Jul 19 11:58:36 qhdata-dev kibana[209136]: i Kibana has not been configured.
Jul 19 11:58:36 qhdata-dev kibana[209136]: Go to http://0.0.0.0:5601/?code=437242 to get started.
Jul 19 11:59:06 qhdata-dev kibana[209136]: Your verification code is:  437 242

然后使用Elasticsearch 的超级用户跟密码即可登陆。

连接elasticsearch

修改/etc/kibana/kibana.yml
添加

enterpriseSearch.host: 'http://localhost:3002'

然后重启，重启后点击


能看到这个页面代表连接enterprise search成功。
页面查看数据