[原创]aix rootvg 一盘丢失并人为从新生成后的恢复

描述：P550两块盘hdisk0和hdisk1属rootvg且做了mirror，由于是实验机有时unmirror或重新mirror，一次做了这样的操
作，之前有同事做了实验后通过OP关了P550,过了几天，我重新开机，发现
# lsdev -Cc disk
hdisk0 Available 09-08-01-3,0 16 Bit LVD SCSI Disk Drive
hdisk1 Defined 09-08-01-4,0 16 Bit LVD SCSI Disk Drive
# lspv
hdisk0 00c8975e3c78c55f rootvg active
# chdev -l hdisk1 -a pv=yes
Method error (/etc/methods/chgdisk):
0514-053 Error returned from sys_config.
# mkdev -l hdisk1
Method error (/etc/methods/cfgscdisk):
0514-077 Cannot perform the requested function because none of the
specified paths match those for the specified device.
# rmdev -l hdisk1 -d
hdisk1 deleted
# cfgmgr -v |grep disk
ses0 hdisk1
attempting to configure device 'hdisk1'
invoking /etc/methods/cfgscdisk -l hdisk1
Completed method for: hdisk1, Elapsed time = 0
# lspv
hdisk0 00c8975e3c78c55f rootvg active
hdisk1 none None
# mkdev -l hdisk1
hdisk1 Available
# lspv
hdisk0 00c8975e3c78c55f rootvg active
hdisk1 none None
# lspv
hdisk0 00c8975e3c78c55f rootvg active
hdisk1 none None
# chdev -l hdisk1 -a pv=yes
hdisk1 changed
# lspv
hdisk0 00c8975e3c78c55f rootvg active
hdisk1 00c8975e9e90752a None
# lsvg -p rootvg
rootvg:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION
hdisk0 active 546 318 107..00..00..102..109
0516-304 : Unable to find device id 005787fd5f71a161 in the Device
Configuration Database.
005787fd5f71a161 missing 546 318 47..50..03..109..109
redefinevg -d disk0 rootvg
0516-306 : Unable to find disk0 in the Device
Configuration Database.
0516-008 redefinevg: LVM system call returned an unknown
error code (LVM_INV_DEVENT).
# synclvodm -P -v rootvg
0516-510 synclvodm: Physical volume not found for physical volume
identifier 005787fd5f71a1610000000000000000.
0516-548 synclvodm: Partially successful with updating volume
group rootvg.
synclvodm: Logical volume hd5 updated.
synclvodm: Logical volume hd6 updated.
synclvodm: Logical volume hd8 updated.
synclvodm: Logical volume hd4 updated.
synclvodm: Logical volume hd2 updated.
synclvodm: Logical volume hd9var updated.
synclvodm: Logical volume hd3 updated.
synclvodm: Logical volume hd1 updated.
synclvodm: Logical volume hd10opt updated.
synclvodm: Logical volume fwdump updated.
synclvodm: Logical volume software updated.
# lsvg -p rootvg
rootvg:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION
hdisk0 active 546 318 107..00..00..102..109
0516-304 : Unable to find device id 005787fd5f71a161 in the Device
Configuration Database.
005787fd5f71a161 missing 546 318 47..50..03..109..109
依然报错
# lsvg -l rootvg
rootvg:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT
hd5 boot 1 2 2 closed/stale N/A
hd6 paging 57 114 2 open/syncd N/A
hd8 jfs2log 1 2 2 open/stale N/A
hd4 jfs2 8 16 2 open/stale /
hd2 jfs2 30 60 2 open/stale /usr
hd9var jfs2 1 2 2 open/stale /var
hd3 jfs2 64 128 2 open/stale /tmp
hd1 jfs2 1 2 2 open/stale /home
hd10opt jfs2 1 2 2 open/stale /opt
fwdump jfs2 2 4 2 open/stale /var/adm/ras/platform
software jfs2 62 124 2 open/stale /software
# lsvg rootvg
VOLUME GROUP: rootvg VG IDENTIFIER: 00c8975e00004c000000013608eda4e0
VG STATE: active PP SIZE: 128 megabyte(s)
VG PERMISSION: read/write TOTAL PPs: 1092 (139776 megabytes)
MAX LVs: 256 FREE PPs: 636 (81408 megabytes)
LVs: 11 USED PPs: 456 (58368 megabytes)
OPEN LVs: 10 QUORUM: 1 (Disabled)
TOTAL PVs: 2 VG DESCRIPTORS: 3
STALE PVs: 1 STALE PPs: 27
ACTIVE PVs: 1 AUTO ON: yes
MAX PPs per VG: 32512
MAX PPs per PV: 1016 MAX PVs: 32
LTG size (Dynamic): 128 kilobyte(s) AUTO SYNC: no
HOT SPARE: no BB POLICY: relocatable
原来两盘做了mirror,及加上之前的一系列操作造成破坏了rootvg的mirror从而产生STALE，且新生成的hdisk1的ID与原来
在rootvg中hdisk1的ID不一致，因此进行redefinevg与synclvodm操作，错误都在；
接着，尝试extendvg
# extendvg rootvg hdisk1
0516-322 extendvg: The Device Configuration Database is inconsistent.
Execute redefinevg to correct the database.
0516-792 extendvg: Unable to extend volume group.
显然，不行的
然后，从mirror上入手先，清除rootvg上的mirror
# unmirrorvg rootvg
0516-304 getlvodm: Unable to find device id 005787fd5f71a161 in the Device
Configuration Database.
0516-1246 rmlvcopy: If hd5 is the boot logical volume, please run 'chpv -c <diskname>'
as root user to clear the boot record and avoid a potential boot
off an old boot image that may reside on the disk from which this
logical volume is moved/removed.
0516-1804 chvg: The quorum change takes effect immediately.
0516-1144 unmirrorvg: rootvg successfully unmirrored, user should perform
bosboot of system to reinitialize boot records. Then, user must modify
bootlist to just include: hdisk0.
有报错，但是提示起码说明，rootvg上的mirror的能清除；如下所示，且LV状态从stale变成syncd ，这是好事啊
# lsvg -l rootvg
rootvg:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT
hd5 boot 1 1 1 closed/syncd N/A
hd6 paging 57 57 1 open/syncd N/A
hd8 jfs2log 1 1 1 open/syncd N/A
hd4 jfs2 8 8 1 open/syncd /
hd2 jfs2 30 30 1 open/syncd /usr
hd9var jfs2 1 1 1 open/syncd /var
hd3 jfs2 64 64 1 open/syncd /tmp
hd1 jfs2 1 1 1 open/syncd /home
hd10opt jfs2 1 1 1 open/syncd /opt
fwdump jfs2 2 2 1 open/syncd /var/adm/ras/platform
software jfs2 62 62 1 open/syncd /software
# synclvodm -P -v rootvg
0516-510 synclvodm: Physical volume not found for physical volume
identifier 005787fd5f71a1610000000000000000.
0516-548 synclvodm: Partially successful with updating volume
group rootvg.
synclvodm: Logical volume hd5 updated.
synclvodm: Logical volume hd6 updated.
synclvodm: Logical volume hd8 updated.
synclvodm: Logical volume hd4 updated.
synclvodm: Logical volume hd2 updated.
synclvodm: Logical volume hd9var updated.
synclvodm: Logical volume hd3 updated.
synclvodm: Logical volume hd1 updated.
synclvodm: Logical volume hd10opt updated.
synclvodm: Logical volume fwdump updated.
synclvodm: Logical volume software updated.
# lsvg -p rootvg
rootvg:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION
hdisk0 active 546 318 107..00..00..102..109
0516-304 : Unable to find device id 005787fd5f71a161 in the Device
Configuration Database.
005787fd5f71a161 missing 546 546 110..109..109..109..109
还是报原来的错，那就直接把rootvg上原来hdisk1存在的ID清除掉看看
# reducevg rootvg 005787fd5f71a161
0516-304 putlvodm: Unable to find device id 005787fd5f71a1610000000000000000 in the Device
Configuration Database.
0516-896 reducevg: Warning, cannot remove physical volume 005787fd5f71a161 from
Device Configuration Database.
# reducevg -f rootvg 005787fd5f71a161
0516-022 lquerypv: Illegal parameter or structure value.
0516-022 ldeletepv: Illegal parameter or structure value.
0516-884 reducevg: Unable to remove physical volume 005787fd5f71a161.
# lsvg -p rootvg
rootvg:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION
hdisk0 active 546 318 107..00..00..102..109
# synclvodm -P -v rootvg
synclvodm: Physical volume data updated.
synclvodm: Logical volume hd5 updated.
synclvodm: Logical volume hd6 updated.
synclvodm: Logical volume hd8 updated.
synclvodm: Logical volume hd4 updated.
synclvodm: Logical volume hd2 updated.
synclvodm: Logical volume hd9var updated.
synclvodm: Logical volume hd3 updated.
synclvodm: Logical volume hd1 updated.
synclvodm: Logical volume hd10opt updated.
synclvodm: Logical volume fwdump updated.
synclvodm: Logical volume software updated.
# lsvg rootvg
VOLUME GROUP: rootvg VG IDENTIFIER: 00c8975e00004c000000013608eda4e0
VG STATE: active PP SIZE: 128 megabyte(s)
VG PERMISSION: read/write TOTAL PPs: 546 (69888 megabytes)
MAX LVs: 256 FREE PPs: 318 (40704 megabytes)
LVs: 11 USED PPs: 228 (29184 megabytes)
OPEN LVs: 10 QUORUM: 2 (Enabled)
TOTAL PVs: 1 VG DESCRIPTORS: 2
STALE PVs: 0 STALE PPs: 0
ACTIVE PVs: 1 AUTO ON: yes
MAX PPs per VG: 32512
MAX PPs per PV: 1016 MAX PVs: 32
LTG size (Dynamic): 256 kilobyte(s) AUTO SYNC: no
HOT SPARE: no BB POLICY: relocatable
总算，把这次由于人为的原因造成rootvg磁盘成员的故障的解决掉，现在就是要mirror回去了。

# extendvg rootvg hdisk1
0516-1398 extendvg: The physical volume hdisk1, appears to belong to
another volume group. Use the force option to add this physical volume
to a volume group.
0516-792 extendvg: Unable to extend volume group.
# extendvg -f rootvg hdisk1
# mirrorvg -S rootvg hdisk1
0516-1804 chvg: The quorum change takes effect immediately.
0516-1126 mirrorvg: rootvg successfully mirrored, user should perform
bosboot of system to initialize boot records. Then, user must modify
bootlist to include: hdisk0 hdisk1.
# lsvg -l rootvg
rootvg:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT
hd5 boot 1 2 2 closed/syncd N/A
hd6 paging 57 114 2 open/syncd N/A
hd8 jfs2log 1 2 2 open/syncd N/A
hd4 jfs2 8 16 2 open/stale /
hd2 jfs2 30 60 2 open/stale /usr
hd9var jfs2 1 2 2 open/stale /var
hd3 jfs2 64 128 2 open/stale /tmp
hd1 jfs2 1 2 2 open/stale /home
hd10opt jfs2 1 2 2 open/stale /opt
fwdump jfs2 2 4 2 open/stale /var/adm/ras/platform
software jfs2 62 124 2 open/stale /software
OK，问题到这总算完成。
总结，一，做任何操作前一定要对rootvg或其他vg做详细的检查，以避免人为造成故障；
二，对于本次的造成故障的解决归纳：# unmirrorvg rootvg，# reducevg rootvg hdisk#ID，
# synclvodm -P -v rootvg，# extendvg -f rootvg hdisk1，mirrorvg -S rootvg hdisk1