(一)准备工作
环境:
elasticsearch 5.6.3 & repository-hdfs plugin
centos 7
jdk1.8
hadoop 2.7.2
配置hadoop环境,后续可通过浏览器查看hadoop信息:
namenode 概况:http://localhost:50070
集群和应用信息:http://localhost:8088
节点管理: http://localhost:8042
创建测试索引并索引数据
(二)本地文件系统存储
1、elasticsearch.yml中配置仓库路径
path.repo: [“/home/zhang/backups”]
2、创建仓库,仓库名为:my_backup
curl -XPUT 'http://localhost:9200/_snapshot/my_backup' -H 'Content-Type: application/json' -d '{
{
"my_backup": {
"type": "fs",
"settings": {
"compress": true,
"location": "my_backup"
}
}
}'
3、创建快照,快照名为snapshot_1
curl -XPUT 'http://localhost:9200/_snapshot/my_backup/snapshot_1?wait_for_completion=true'
创建后的快照文件:
4、恢复快照数据
恢复快照数据之前需要关闭索引:
curl -XPOST 'http://localhost:9200/index_test/_close'
提供仓库名和快照名通过_restore命令恢复快照:
curl -XPOST 'http://localhost:9200/_snapshot/my_backup/snapshot_1/_restore'
(三)Hadoop HDFS文件系统存储
1、安装hdfs存储支持插件:
sudo bin/elasticsearch-plugin install repository-hdfs
2、创建仓库,仓库名为:my_hdfs_repository
路径为:/home/zhang/backups/my_hdfs_storage
namenode在/etc/hosts中配置为本地地址
curl -XPUT 'http://localhost:9200/_snapshot/my_hdfs_storage?pretty' -H 'Content-Type: application/json' -d'
{
"type": "hdfs",
"settings": {
"uri": "hdfs://namenode:9000/",
"path": "/home/zhang/backups/my_hdfs_storage"
}
}
'
3、创建快照,快照名为snapshot_hdfs_1
curl -XPUT 'http://localhost:9200/_snapshot/my_hdfs_storage/snapshot_hdfs_1'
4、恢复快照
首先应该关闭索引,然后恢复
curl -XPOST 'http://localhost:9200/index_test/_close'
curl -XPOST 'http://localhost:9200/_snapshot/my_hdfs_storage/snapshot_hdfs_1/_restore'
(四)验证是否能有效恢复
首先创建id 为1,2的文档,创建快照,再创建id为3的文档
关闭索引,恢复快照后,不存在id为3的文档,说明当前索引数据被快照恢复覆盖了,快照数据正常
(五)搭建Hadoop集群平台
单节点根据文档中搭建即可,可以参考centos7安装hadoop:
https://www.tecmint.com/install-configure-apache-hadoop-centos-7/
(六)相关官方文档:
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html
https://www.elastic.co/guide/en/elasticsearch/plugins/5.6/repository.html
https://www.elastic.co/guide/en/elasticsearch/plugins/5.6/repository-hdfs.html