glusterfs做快照
说明：
gluasterfs做快照需要创建精简卷（Thinly-Provisioned ）。
官网
中文
https://access.redhat.com/documentation/zh-cn/red_hat_enterprise_linux/7/html/logical_volume_manager_administration/thinly_provisioned_volume_creation
英文
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/logical_volume_manager_administration/lv#thinly_provisioned_volume_creation
创建精简卷
说明：共3台主机，每台主机做同样的操作。
1、加一块盘（20g）
2、创建卷组
vgcreate vg001 /dev/vdd
3、创建精简卷
lvcreate -L 19G -T vg001/mythinpool -V 19G -n thinvolume
4、查看LV Path
lvdisplay
--- Logical volume ---
LV Name mythinpool
VG Name vg001
LV UUID DVXGur-xtLK-MTFG-12Ed-vHo5-JdYQ-x0BukS
LV Write Access read/write (activated read only)
LV Creation host, time gluster01, 2023-07-19 10:14:58 +0800
LV Pool metadata mythinpool_tmeta
LV Pool data mythinpool_tdata
LV Status available
# open 0
LV Size 19.00 GiB
Allocated pool data 0.06%
Allocated metadata 10.57%
Current LE 4864
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:3
--- Logical volume ---
LV Path /dev/vg001/thinvolume
LV Name thinvolume
VG Name vg001
LV UUID OuzIKL-SG2j-BlHi-8P5e-qIoh-hh5v-PqXhPr
LV Write Access read/write
LV Creation host, time gluster01, 2023-07-19 10:14:58 +0800
LV Pool name mythinpool
LV Status available
# open 1
LV Size 19.00 GiB
Mapped size 0.06%
Current LE 4864
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:4
glusterfs创建新的卷
说明：只在一台主机操作。
1、格式化LV Path
mkfs.xfs -i size=512 /dev/vg001/thinvolume
2、添加挂载信息到/etc/fstab
echo "/dev/vg001/thinvolume /export/thinvolume01/ xfs defaults 0 0" >> /etc/fstab
3、创建挂载点并挂载
mkdir -p /export/thinvolume01 && mount -a && mkdir -p /export/thinvolume01/brick
4、创建新的卷
gluster volume create gv1 disperse 3 gluster{01..03}:/export/thinvolume01/brick
5、启动gv1卷
gluster volume start gv1
6、查看卷信息
gluster volume info
输出
Volume Name: gv1
Type: Disperse
Volume ID: afbf6c59-8229-4e38-acda-d0ba96fc7df2
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x (2 + 1) = 3
Transport-type: tcp
Bricks:
Brick1: gluster01:/export/thinvolume01/brick
Brick2: gluster02:/export/thinvolume01/brick
Brick3: gluster03:/export/thinvolume01/brick
Options Reconfigured:
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on
测试快照恢复数据
1、挂载glusterfs文件
mount -t glusterfs gluster01:/gv1 /mnt/glusterlocal/
2、生成数据
cd /mnt/glusterlocal/
for i in `seq 100`;do echo "this is $i file" >> $i.txt;cat /etc/passwd >> $i.txt;done
说明：可以看到挂载点目录和glusterfs后端都有数据生成。
glusterfs后端数据（任选一个节点查看）
ls /export/thinvolume02/brick/
100.txt 14.txt 1.txt 24.txt 36.txt 44.txt 57.txt 64.txt 6.txt 74.txt 84.txt 95.txt
10.txt 15.txt 20.txt 25.txt 38.txt 47.txt 5.txt 65.txt 70.txt 75.txt 87.txt 98.txt
11.txt 16.txt 21.txt 2.txt 3.txt 54.txt 60.txt 67.txt 71.txt 77.txt 88.txt
12.txt 18.txt 22.txt 30.txt 43.txt 55.txt 61.txt 69.txt 72.txt 7.txt 92.txt
[gluster] root@gluster03:~# ls /export/thinvolume01/brick/
13.txt 26.txt 31.txt 35.txt 41.txt 48.txt 51.txt 58.txt 66.txt 78.txt 82.txt 89.txt 93.txt 99.txt
17.txt 27.txt 32.txt 37.txt 42.txt 49.txt 52.txt 59.txt 68.txt 79.txt 83.txt 8.txt 94.txt 9.txt
19.txt 28.txt 33.txt 39.txt 45.txt 4.txt 53.txt 62.txt 73.txt 80.txt 85.txt 90.txt 96.txt test
23.txt 29.txt 34.txt 40.txt 46.txt 50.txt 56.txt 63.txt 76.txt 81.txt 86.txt 91.txt 97.txt
挂载点目录数据
ls /mnt/glusterlocal
100.txt 17.txt 24.txt 31.txt 39.txt 46.txt 53.txt 60.txt 68.txt 75.txt 82.txt 8.txt 97.txt
10.txt 18.txt 25.txt 32.txt 3.txt 47.txt 54.txt 61.txt 69.txt 76.txt 83.txt 90.txt 98.txt
11.txt 19.txt 26.txt 33.txt 40.txt 48.txt 55.txt 62.txt 6.txt 77.txt 84.txt 91.txt 99.txt
12.txt 1.txt 27.txt 34.txt 41.txt 49.txt 56.txt 63.txt 70.txt 78.txt 85.txt 92.txt 9.txt
13.txt 20.txt 28.txt 35.txt 42.txt 4.txt 57.txt 64.txt 71.txt 79.txt 86.txt 93.txt test
14.txt 21.txt 29.txt 36.txt 43.txt 50.txt 58.txt 65.txt 72.txt 7.txt 87.txt 94.txt
15.txt 22.txt 2.txt 37.txt 44.txt 51.txt 59.txt 66.txt 73.txt 80.txt 88.txt 95.txt
16.txt 23.txt 30.txt 38.txt 45.txt 52.txt 5.txt 67.txt 74.txt 81.txt 89.txt 96.txt
3、创建快照
gluster snapshot create gv1-snap01 gv1 description "/mnt/glusterlocal data"
输出
snapshot create: success: Snap gv1-snap01_GMT-2023.07.19-07.00.01 created successfully
4、删除数据
说明：删除数据后，glusterfs后端及挂载点数据都会删除。
cd /mnt/glusterlocal/
rm -rf *
5、激活快照
gluster snapshot activate gv1-snap01_GMT-2023.07.19-07.00.01
6、使用快照恢复数据
说明：恢复快照前要先停止卷。
gluster volume stop gv1
gluster snapshot restore gv1-snap01_GMT-2023.07.19-07.00.01
7、恢复快照后再开启卷
gluster volume start gv1
8、验证数据是否恢复
说明：恢复快照后，不开启卷，会报错。如下
ls
ls: cannot open directory '.': Transport endpoint is not connected
开启卷后发现数据已经恢复。
ls /mnt/glusterlocal/
100.txt 17.txt 24.txt 31.txt 39.txt 46.txt 53.txt 60.txt 68.txt 75.txt 82.txt 8.txt 97.txt
10.txt 18.txt 25.txt 32.txt 3.txt 47.txt 54.txt 61.txt 69.txt 76.txt 83.txt 90.txt 98.txt
11.txt 19.txt 26.txt 33.txt 40.txt 48.txt 55.txt 62.txt 6.txt 77.txt 84.txt 91.txt 99.txt
12.txt 1.txt 27.txt 34.txt 41.txt 49.txt 56.txt 63.txt 70.txt 78.txt 85.txt 92.txt 9.txt
13.txt 20.txt 28.txt 35.txt 42.txt 4.txt 57.txt 64.txt 71.txt 79.txt 86.txt 93.txt
14.txt 21.txt 29.txt 36.txt 43.txt 50.txt 58.txt 65.txt 72.txt 7.txt 87.txt 94.txt
15.txt 22.txt 2.txt 37.txt 44.txt 51.txt 59.txt 66.txt 73.txt 80.txt 88.txt 95.txt
16.txt 23.txt 30.txt 38.txt 45.txt 52.txt 5.txt 67.txt 74.txt 81.txt 89.txt 96.txt
警告：每次还原快照后，df查看挂载，挂载会变。
原来的挂载
/dev/mapper/vg001-thinvolume 19G 170M 19G 1% /export/thinvolume01
/dev/mapper/vg002-thinvolume 19G 170M 19G 1% /export/thinvolume02
还原快照后的挂载
/dev/mapper/vg001-94114825f8034fe49c7a2d9164404bfc_0 19G 170M 19G 1% /run/gluster/snaps/94114825f8034fe49c7a2d9164404bfc/brick1
/dev/mapper/vg002-94114825f8034fe49c7a2d9164404bfc_1 19G 170M 19G 1% /run/gluster/snaps/94114825f8034fe49c7a2d9164404bfc/brick4
附
glusterfs 快照常用操作
1、创建快照
gluster snapshot create gv1-snap01 gv1 description "k8s pod jfs-test-69b556c6df-nv2mx data"
输出
snapshot create: success: Snap gv1-snap01_GMT-2023.07.19-03.24.45 created successfully
2、查看可用的快照
不指定volume名字
gluster snapshot list
输出
gv1-snap01_GMT-2023.07.19-03.24.45
指定volume名字
gluster snapshot list gv1
输出
gv1-snap01_GMT-2023.07.19-03.24.45
3、查看快照信息
gluster snapshot info gv1-snap01_GMT-2023.07.19-03.24.45
Snapshot : gv1-snap01_GMT-2023.07.19-03.24.45
Snap UUID : 029f083f-10c4-42ce-800c-203612121e38
Description : k8s pod jfs-test-69b556c6df-nv2mx data
Created : 2023-07-19 03:24:45 +0000
Snap Volumes:
Snap Volume Name : 81bd41a880874426a3f9196603a1f44e
Origin Volume name : gv1
Snaps taken for gv1 : 1
Snaps available for gv1 : 255
Status : Stopped
4、查看快照状态
gluster snapshot status gv1-snap01_GMT-2023.07.19-03.24.45
Snap Name : gv1-snap01_GMT-2023.07.19-03.24.45
Snap UUID : 029f083f-10c4-42ce-800c-203612121e38
Brick Path : gluster01:/run/gluster/snaps/81bd41a880874426a3f9196603a1f44e/brick1/brick
Volume Group : N/A (Deactivated Snapshot)
Brick Running : No
Brick PID : N/A
Data Percentage : N/A
LV Size : N/A
Brick Path : gluster02:/run/gluster/snaps/81bd41a880874426a3f9196603a1f44e/brick2/brick
Volume Group : N/A (Deactivated Snapshot)
Brick Running : No
Brick PID : N/A
Data Percentage : N/A
LV Size : N/A
Brick Path : gluster03:/run/gluster/snaps/81bd41a880874426a3f9196603a1f44e/brick3/brick
Volume Group : N/A (Deactivated Snapshot)
Brick Running : No
Brick PID : N/A
Data Percentage : N/A
LV Size : N/A
6、激活快照
gluster snapshot activate gv1-snap01_GMT-2023.07.19-03.24.45
输出
Snapshot activate: gv1-snap01_GMT-2023.07.19-03.24.45: Snap activated successfully
说明：激活快照后，再次查看快照状态会发生变化。