XenServer共享存储不小心被格式化了如何修复

昨天有个客户那边重装了一台机器，但是没有拔掉光纤线，然后做选择的时候还把那个LUN给初始化了，导致新的服务器加到资源池里存储一直挂不上。有几个虚拟机一迁移就失败了，还开不了机。SSH上去看，磁盘已经被格式化了，还创建了一个分区。

 

以1.36和1.58为列 说一下解决步骤

1.36上是主节点，上面有几个虚拟机在运行。

 

在1.36上执行下面一系列命令

首先是fdisk -l 因为有两条路径 所以能看到/dev/sdb和/dev/sdc是一样的，与正常的SR的磁盘的区别是多了一个GPT分区

[root@localhost ~]# fdisk -l Disk /dev/sda: 2000.3 GB, 2000398934016 bytes 255 heads, 63 sectors/track, 243201 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes    Device Boot      Start         End      Blocks   Id  System /dev/sda1   *           1         522     4192933+  83  Linux /dev/sda2             523         783     2096482+  83  Linux /dev/sda3             784      243201  1947222585   8e  Linux LVM WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted. WARNING: The size of this disk is 2.6 TB (2576980377600 bytes). DOS partition table format can not be used on drives for volumes larger than 2.2 TB (2199023255040 bytes). Use parted(1) and GUID partition table format (GPT). Disk /dev/sdb: 2576.9 GB, 2576980377600 bytes 255 heads, 63 sectors/track, 313300 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes    Device Boot      Start         End      Blocks   Id  System /dev/sdb1               1      267350  2147483647+  ee  EFI GPT WARNING: GPT (GUID Partition Table) detected on '/dev/sdc'! The util fdisk doesn't support GPT. Use GNU Parted. WARNING: The size of this disk is 2.6 TB (2576980377600 bytes). DOS partition table format can not be used on drives for volumes larger than 2.2 TB (2199023255040 bytes). Use parted(1) and GUID partition table format (GPT). Disk /dev/sdc: 2576.9 GB, 2576980377600 bytes 255 heads, 63 sectors/track, 313300 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes    Device Boot      Start         End      Blocks   Id  System /dev/sdc1               1      267350  2147483647+  ee  EFI GPT

 

 

运行pvscan  可以看到Physical Volume已经不见了

[root@localhost ~]# pvscan   PV /dev/sda3   VG VG_XenStorage-04c0df5f-b878-9d62-5157-649b0ebc8980   lvm2 [1.81 TB / 1.81 TB free]   Total: 1 [1.81 TB] / in use: 1 [1.81 TB] / in no VG: 0 [0   ]

 

9ba32798-7dc6-0b2b-8abc-645192a9203a这个ID是我们出问题的存储(SR)的UUID

[root@localhost ~]# xe sr-list uuid ( RO)                : ee931812-2d09-fd43-8b42-73fd4f512512           name-label ( RW): DVD drives     name-description ( RW): Physical DVD drives                 host ( RO): XenServer58                 type ( RO): udev         content-type ( RO): iso uuid ( RO)                : 04c0df5f-b878-9d62-5157-649b0ebc8980           name-label ( RW): Local storage     name-description ( RW):                 host ( RO): localhost.localdomain                 type ( RO): lvm         content-type ( RO): user uuid ( RO)                : 9ba32798-7dc6-0b2b-8abc-645192a9203a           name-label ( RW): 硬件 HBA èæç£çå­å¨     name-description ( RW): 硬件 HBA SR [DELL - 2CC001B]                 host ( RO):                 type ( RO): lvmohba         content-type ( RO): uuid ( RO)                : 396812b6-e90b-0761-50a3-b08e57c19453           name-label ( RW): Local storage     name-description ( RW):                 host ( RO): XenServer58                 type ( RO): lvm         content-type ( RO): user ....

 

在/etc/lvm/backup中系统自动备份的lvm信息

[root@localhost ~]# cd /etc/lvm/backup/ [root@localhost backup]# ls -l total 8 -rw------- 1 root root 1311 Jan 17 14:02 VG_XenStorage-04c0df5f-b878-9d62-5157-649b0ebc8980 -rw------- 1 root root 3454 Jan 17 14:02 VG_XenStorage-9ba32798-7dc6-0b2b-8abc-645192a9203a

 

因为这个LUN还在被1.36上面的虚拟机使用，pv还在被占用，所以我们把文件拷贝到1.58上，去1.58上执行之后的操作

[root@localhost backup]# scp -r /etc/lvm/backup/ [email protected]:/root/lvm_backcup The authenticity of host '192.168.1.58 (192.168.1.58)' can't be established. RSA key fingerprint is 21:66:46:86:60:79:12:9a:23:f7:99:34:97:12:e7:a8. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.1.58' (RSA) to the list of known hosts. [email protected]'s password: VG_XenStorage-04c0df5f-b878-9d62-5157-649b0ebc8980                                                                                                           100% 1311     1.3KB/s   00:00    VG_XenStorage-9ba32798-7dc6-0b2b-8abc-645192a9203a                                                                                                           100% 3454     3.4KB/s   00:00

 

 

在1.58上执行下面的操作

首先删除多出来的/dev/sdb1分区

[root@localhost ~]# fdisk /dev/sdb WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted. The number of cylinders for this disk is set to 313300. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs    (e.g., DOS FDISK, OS/2 FDISK) WARNING: The size of this disk is 2.6 TB (2576980377600 bytes). DOS partition table format can not be used on drives for volumes larger than 2.2 TB (2199023255040 bytes). Use parted(1) and GUID  partition table format (GPT). Command (m for help): d Selected partition 1 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@localhost ~]# fdisk -l Disk /dev/sda: 146.8 GB, 146815733760 bytes 255 heads, 63 sectors/track, 17849 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes    Device Boot      Start         End      Blocks   Id  System /dev/sda1   *           1         522     4192933+  83  Linux /dev/sda2             523         783     2096482+  83  Linux /dev/sda3             784       17849   137082645   8e  Linux LVM WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted. WARNING: The size of this disk is 2.6 TB (2576980377600 bytes). DOS partition table format can not be used on drives for volumes larger than 2.2 TB (2199023255040 bytes). Use parted(1) and GUID  partition table format (GPT). Disk /dev/sdb: 2576.9 GB, 2576980377600 bytes 255 heads, 63 sectors/track, 313300 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes    Device Boot      Start         End      Blocks   Id  System WARNING: GPT (GUID Partition Table) detected on '/dev/sdc'! The util fdisk doesn't support GPT. Use GNU Parted. WARNING: The size of this disk is 2.6 TB (2576980377600 bytes). DOS partition table format can not be used on drives for volumes larger than 2.2 TB (2199023255040 bytes). Use parted(1) and GUID  partition table format (GPT). Disk /dev/sdc: 2576.9 GB, 2576980377600 bytes 255 heads, 63 sectors/track, 313300 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes    Device Boot      Start         End      Blocks   Id  System [root@localhost ~]#

 

 

查看lvm备份文件找到physical volume的id，这里是"wQbSMU-CAVF-8P1K-aruD-h8Ts-btVf-jdu0nM"

[root@localhost ~]# cd /root/lvm_backcup/ [root@localhost lvm_backcup]# ls -l total 8 -rw------- 1 root root 1311 Jan 16 16:13 VG_XenStorage-04c0df5f-b878-9d62-5157-649b0ebc8980 -rw------- 1 root root 3454 Jan 16 16:13 VG_XenStorage-9ba32798-7dc6-0b2b-8abc-645192a9203a [root@localhost lvm_backcup]# cat VG_XenStorage-9ba32798-7dc6-0b2b-8abc-645192a9203a | grep  physical_volumes -A3                                                                                      physical_volumes {                 pv0 {                         id = "wQbSMU-CAVF-8P1K-aruD-h8Ts-btVf-jdu0nM"

重建physical volume 

注意因为没有启用多路径，所以使用/dev/sdb或者/dev/sdc都可以。如果之前启用了多路径需要使用目录 /dev/mapper/ or /dev/disk/mpInuse/.

[root@localhost lvm_backcup]# pvcreate --uuid "wQbSMU-CAVF-8P1K-aruD-h8Ts-btVf-jdu0nM" --restorefile /root/lvm_backcup/VG_XenStorage-9ba32798-7dc6-0b2b-8abc-645192a9203a /dev/sdb   Couldn't find device with uuid wQbSMU-CAVF-8P1K-aruD-h8Ts-btVf-jdu0nM.   Physical volume "/dev/sdb" successfully created [root@localhost lvm_backcup]# pvscan   Found duplicate PV wQbSMUCAVF8P1KaruDh8TsbtVfjdu0nM: using /dev/sdc not /dev/sdb   PV /dev/sda3   VG VG_XenStorage-396812b6-e90b-0761-50a3-b08e57c19453   lvm2 [130.72 GB / 130.71 GB free]   PV /dev/sdc                                                            lvm2 [2.34 TB]   Total: 2 [2.47 TB] / in use: 1 [130.72 GB] / in no VG: 1 [2.34 TB]

重建Volume Group

先进行测试

[root@localhost lvm_backcup]# vgcfgrestore VG_XenStorage-9ba32798-7dc6-0b2b-8abc-645192a9203a --test -f /root/lvm_backcup/VG_XenStorage-9ba32798-7dc6-0b2b-8abc-645192a9203a                      Test mode: Metadata will NOT be updated and volumes will not be (de)activated.   Found duplicate PV wQbSMUCAVF8P1KaruDh8TsbtVfjdu0nM: using /dev/sdc not /dev/sdb   Restored volume group VG_XenStorage-9ba32798-7dc6-0b2b-8abc-645192a9203a

执行重建命令

[root@localhost lvm_backcup]# vgcfgrestore VG_XenStorage-9ba32798-7dc6-0b2b-8abc-645192a9203a -f /root/lvm_backcup/VG_XenStorage-9ba32798-7dc6-0b2b-8abc-645192a9203a                            Found duplicate PV wQbSMUCAVF8P1KaruDh8TsbtVfjdu0nM: using /dev/sdc not /dev/sdb   Restored volume group VG_XenStorage-9ba32798-7dc6-0b2b-8abc-645192a9203a

检查效果

[root@localhost lvm_backcup]# pvscan   Found duplicate PV wQbSMUCAVF8P1KaruDh8TsbtVfjdu0nM: using /dev/sdc not /dev/sdb   PV /dev/sdc    VG VG_XenStorage-9ba32798-7dc6-0b2b-8abc-645192a9203a   lvm2 [2.34 TB / 2.26 TB free]   PV /dev/sda3   VG VG_XenStorage-396812b6-e90b-0761-50a3-b08e57c19453   lvm2 [130.72 GB / 130.71 GB free]   Total: 2 [2.47 TB] / in use: 2 [2.47 TB] / in no VG: 0 [0   ] [root@localhost lvm_backcup]# vgscan   Reading all physical volumes.  This may take a while...   Found duplicate PV wQbSMUCAVF8P1KaruDh8TsbtVfjdu0nM: using /dev/sdc not /dev/sdb   Found volume group "VG_XenStorage-9ba32798-7dc6-0b2b-8abc-645192a9203a" using metadata type lvm2   Found volume group "VG_XenStorage-396812b6-e90b-0761-50a3-b08e57c19453" using metadata type lvm2

在Console上修复存储，可以成功挂载了。关闭虚拟机，重启所有的服务器，之后再启动虚拟机就没问题了。