aix mount: 0506-351 错误处理

 

故障现象：
1、主库日志报错
Sun Nov 27 12:31:34 2011
Errors in file /home/oracle/admin/zjtv/bdump/zjtv1_arc0_803242.trc:
ORA-16055: FAL request rejected
ARCH: FAL archive failed. Archiver continuing
Sun Nov 27 12:31:34 2011
ORACLE Instance zjtv1 - Archival Error. Archiver continuing.
Sun Nov 27 12:36:34 2011
Errors in file /home/oracle/admin/zjtv/bdump/zjtv1_arc2_709010.trc:
ORA-00270: error creating archive log
Sun Nov 27 12:36:34 2011
FAL[server, ARC2]: FAL archive failed, see trace file.

2、从库日志草错，显示归档文件没有权限创建
Sun Nov 27 13:26:59 2011
Errors in file /home/oracle/admin/standby/udump/standby_rfs_450640.trc:
ORA-00270: error creating archive log /archivelog/arch_2_22493_614088933.arc
ORA-19504: failed to create file "/archivelog/arch_2_22493_614088933.arc"
ORA-27040: file create error, unable to create file
IBM AIX RISC System/6000 Error: 13: Permission denied

故障分析：
该从库上周刚刚重新调整归档目录/archivelog条带化到hdisk0\hdisk1，
故障发生前从库刚刚进行过重启动作，重启后就出现如上错误。
怀疑是没有设置自动MOUNT，根据如下思路有如下具体排查经过：

1、首先查看目录是否存在
P560A:/archivelog#df -g
Filesystem    GB blocks      Free %Used    Iused %Iused Mounted on
/dev/hd4           2.00      0.56   73%     2135     2% /
/dev/hd2           4.00      1.66   59%    50260    12% /usr
/dev/hd9var        1.00      0.27   74%     3219     5% /var
/dev/hd3          10.00      5.62   44%   141687    10% /tmp
/dev/hd1           1.00      0.54   47%      865     1% /home
/proc                 -         -    -         -     -  /proc
/dev/hd10opt       2.00      1.53   24%     7221     2% /opt
/dev/weblogiclv     10.00      9.47    6%       47     1% /weblogic
/dev/weblogic9lv     10.00      1.85   82%    93440    18% /weblogic9
/dev/tsmdb         7.50      2.92   62%     7918     2% /tsmdb
/dev/oraclelv     10.00      2.08   80%    27758     6% /home/oracle
/dev/oradata_lv    300.00    138.46   54%       94     1% /oradata
/dev/fslv00      150.00     26.45   83%       25     1% /backup
发现/archivelog未挂载

2、尝试挂载，出现错误
P560A:/#mount /archivelog
mount: 0506-351 The log device and fs device must be in the same volume group.
显然原因是日志设备和你要挂载的文件系统不在一个卷组里

3、查看archive_lv在哪个VG上，结果是rootvg
P560A:/archivelog#lslv archive_lv
LOGICAL VOLUME:     archive_lv             VOLUME GROUP:   rootvg
LV IDENTIFIER:      00c3ee9e00004c000000011c74979ab0.16 PERMISSION:     read/write
VG STATE:           active/complete        LV STATE:       opened/syncd
TYPE:               jfs2                   WRITE VERIFY:   off
MAX LPs:            512                    PP SIZE:        256 megabyte(s)
COPIES:             1                      SCHED POLICY:   striped
LPs:                400                    PPs:            400
STALE PPs:          0                      BB POLICY:      relocatable
INTER-POLICY:       maximum                RELOCATABLE:    no
INTRA-POLICY:       middle                 UPPER BOUND:    2
MOUNT POINT:        /archivelog            LABEL:          /archivelog
MIRROR WRITE CONSISTENCY: on/ACTIVE
EACH LP COPY ON A SEPARATE PV ?: yes (superstrict)
Serialize IO ?:     NO
STRIPE WIDTH:       2
STRIPE SIZE:        128K
P560A:/archivelog#

4、查看rootvg的日志设备jfs2log在hd8上
P560A:/archivelog#lsvg -l rootvg
rootvg:
LV NAME             TYPE       LPs   PPs   PVs  LV STATE      MOUNT POINT
hd5                 boot       1     2     2    closed/syncd  N/A
hd6                 paging     32    64    2    open/syncd    N/A
hd8                 jfs2log    1     2     2    open/syncd    N/A
hd4                 jfs2       8     16    2    open/syncd    /
hd2                 jfs2       16    32    2    open/syncd    /usr
hd9var              jfs2       4     8     2    open/syncd    /var
hd3                 jfs2       40    80    2    open/syncd    /tmp
hd1                 jfs2       4     8     2    open/syncd    /home
hd10opt             jfs2       8     16    2    open/syncd    /opt
lg_dumplv           sysdump    8     8     1    open/syncd    N/A
fwdump              jfs2       2     2     1    closed/syncd  /var/adm/ras/platform
oraclelv            jfs2       40    40    1    open/syncd    /home/oracle
tsmdb               jfs2       30    60    2    open/syncd    /tsmdb
archive_lv          jfs2       400   400   2    closed/syncd    /archivelog

5、查看/etc/filesystems文件的/archivelog的LOG属性在/dev/loglv00，而正常的应该是/dev/hd8

6、修改/archivelog的LOG属性
vi /etc/filesystems
/oradata:
        dev             = /dev/oradata_lv
        vfs             = jfs2
        log             = /dev/loglv00
        mount           = true
        options         = rw
        account         = false

/archivelog:
        dev             = /dev/archive_lv
        vfs             = jfs2
        log             = /dev/loglv00 --> hd8
        mount           = true
        options         = rw
        account         = false

7、mount成功
mount /archivelog

8、P560A:/archivelog#df -g发现已经挂载
Filesystem    GB blocks      Free %Used    Iused %Iused Mounted on
/dev/hd4           2.00      0.56   73%     2135     2% /
/dev/hd2           4.00      1.66   59%    50260    12% /usr
/dev/hd9var        1.00      0.27   74%     3219     5% /var
/dev/hd3          10.00      5.62   44%   141687    10% /tmp
/dev/hd1           1.00      0.54   47%      865     1% /home
/proc                 -         -    -         -     -  /proc
/dev/hd10opt       2.00      1.53   24%     7221     2% /opt
/dev/weblogiclv     10.00      9.47    6%       47     1% /weblogic
/dev/weblogic9lv     10.00      1.85   82%    93440    18% /weblogic9
/dev/tsmdb         7.50      2.92   62%     7918     2% /tsmdb
/dev/oraclelv     10.00      2.08   80%    27758     6% /home/oracle
/dev/oradata_lv    300.00    138.46   54%       94     1% /oradata
/dev/fslv00      150.00     26.45   83%       25     1% /backup
/dev/archive_lv    100.00     80.91   20%       59     1% /archivelog