CentOS 7 离线迁移 Elasticsearch 数据

CentOS 7 离线迁移 Elasticsearch 数据

环境说明

说明	配置
源地址	10.10.200.15:9200
目的地址	192.168.68.129:9200
迁移索引	user_info
数据条数	7
迁移方法	elasticsearch-dump
安装包依赖	node-v16.16.0-linux-x64.tar.xz elasticdump.tar.gz

环境验证

源地址

• 获取 elasticsearch 集群可用性

[root@localhost ~]# curl 10.10.200.15:9200/_cat/health
1666593542 06:39:02 docker-cluster green 1 1 5 5 0 0 0 0 - 100.0%

• 获取 elasticsearch 索引情况

# 名称
[root@localhost ~]# curl 10.10.200.15:9200/_cat/indices
green open user_info md_IV4S9Qu6FN2haCNXDug 5 0 7 0 23.7kb 23.7kb

# 数据量
[root@localhost ~]# curl 10.10.200.15:9200/user_info/_doc/_count?pretty
{
  "count" : 7,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  }
}

目标地址

• 查看集群可用性

[root@localhost ~]# curl 192.168.68.129:9200/_cat/health
1666622621 14:43:41 docker-cluster green 1 1 0 0 0 0 0 0 - 100.0%

• 查看是否有对应索引

[root@localhost ~]# curl 192.168.68.129:9200/_cat/indices

elasticsearch-dump 环境依赖

# 该服务可安装在任意节点，需要同时能连接 elasticsearch 源地址、目的地址，安装依赖如下
- jdk
- npm
- elasticsearch-dumpo

验证 JDK 环境

[root@localhost ~]# java -version
java version "1.8.0_211"
Java(TM) SE Runtime Environment (build 1.8.0_211-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.211-b12, mixed mode)

验证 npm 环境

• 安装

# 下载
[root@localhost ~]# wget https://nodejs.org/dist/v16.16.0/node-v16.16.0-linux-x64.tar.xz

# 解压
[root@localhost ~]# tar zxf node-v16.16.0-linux-x64.tar.xz

# 分发
[root@localhost ~]# mv node-v16.16.0-linux-x64 /usr/local/nodejs

# 环境变量配置
[root@localhost ~]# tail - /etc/profile
export NODEJS_HOME=/usr/local/nodejs
export PATH=$PATH:$NODEJS_HOME/bin

# 刷新环境变量
[root@localhost ~]# source /etc/profile

• 验证

[root@localhost ~]# npm --version
8.11.0

elasticsearch-dump 安装

在线安装

# 会安装在 -- /usr/local/nodejs/lib/node_modules/elasticdump
[root@localhost ~]# npm install -g elasticdump

解压

# 内网环境 -- 需要提前准备 elasticdump 包
[root@localhost ~]# tar zxf elasticdump.tar.gz

分发

# 目录固定 -- 必须放到此目录
[root@localhost ~]# mv ./elasticdump /usr/local/nodejs/lib/node_modules

验证

[root@localhost ~]# /usr/local/nodejs/lib/node_modules/elasticdump/bin/elasticdump --version
6.92.1

使用 elasticsearch-dump 迁移数据

迁移索引 setting

[root@localhost ~]# /usr/local/nodejs/lib/node_modules/elasticdump/bin/elasticdump --input=http://10.10.200.15:9200/user_info --output=http://192.168.68.129:9200/user_info --type=settings
Mon, 24 Oct 2022 14:29:21 GMT | starting dump
Mon, 24 Oct 2022 14:29:21 GMT | got 1 objects from source elasticsearch (offset: 0)
Mon, 24 Oct 2022 14:29:21 GMT | sent 1 objects to destination elasticsearch, wrote 1
Mon, 24 Oct 2022 14:29:21 GMT | got 0 objects from source elasticsearch (offset: 1)
Mon, 24 Oct 2022 14:29:21 GMT | Total Writes: 1
Mon, 24 Oct 2022 14:29:21 GMT | dump complete

迁移索引 mapping

[root@localhost ~]# /usr/local/nodejs/lib/node_modules/elasticdump/bin/elasticdump --input=http://10.10.200.15:9200/user_info --output=http://192.168.68.129:9200/user_info --type=mapping
Mon, 24 Oct 2022 14:29:43 GMT | starting dump
Mon, 24 Oct 2022 14:29:43 GMT | got 1 objects from source elasticsearch (offset: 0)
Mon, 24 Oct 2022 14:29:43 GMT | sent 1 objects to destination elasticsearch, wrote 1
Mon, 24 Oct 2022 14:29:43 GMT | got 0 objects from source elasticsearch (offset: 1)
Mon, 24 Oct 2022 14:29:43 GMT | Total Writes: 1
Mon, 24 Oct 2022 14:29:43 GMT | dump complete

迁移索引 data

[root@localhost ~]# elasticdump --input=http://10.10.200.15:9200/user_info --output=http://192.168.68.129:9200/user_info --type=data
Mon, 24 Oct 2022 14:30:24 GMT | starting dump
Mon, 24 Oct 2022 14:30:24 GMT | got 7 objects from source elasticsearch (offset: 0)
Mon, 24 Oct 2022 14:30:25 GMT | sent 7 objects to destination elasticsearch, wrote 7
Mon, 24 Oct 2022 14:30:25 GMT | got 0 objects from source elasticsearch (offset: 7)
Mon, 24 Oct 2022 14:30:25 GMT | Total Writes: 7
Mon, 24 Oct 2022 14:30:25 GMT | dump complete

验证迁移结果

# 验证目的地址 -- 索引情况
[root@localhost ~]# curl 192.168.68.129:9200/user_info/_doc/_count?pretty
{
  "count" : 7,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  }
}