incremental backup恢复错误一例

incremental backup恢复归档GAP的DG的方法本文不再详述，可参照链接：http://yunlongzheng.blog.51cto.com/788996/717249
本文着重讲一次恢复过程中出现的错误的解决方法，给自己备忘，也给需要的人参考。

错误现象：关闭standby数据库，将restore standby controlfile to '/u01/rmanbak/fordg/control01.ctl'恢复出来的控制文件覆盖现有的控制文件，重新启动后出现以下问题：
Sat Mar 23 06:42:37 CST 2013
alter database recover managed standby database disconnect from session using current logfile
Sat Mar 23 06:42:37 CST 2013
Attempt to start background Managed Standby Recovery process (stdby)
MRP0 started with pid=29, OS id=11519
Sat Mar 23 06:42:37 CST 2013
MRP0: Background Managed Standby Recovery process started (stdby)
Sat Mar 23 06:42:37 CST 2013
RFS[2]: Archived Log: '/u01/archivelog/stdby/arc_2_9373_789583539.dbf'
RFS[2]: Archived Log: '/u01/archivelog/stdby/arc_2_9374_789583539.dbf'
RFS[2]: Archived Log: '/u01/archivelog/stdby/arc_2_9375_789583539.dbf'
RFS[2]: Archived Log: '/u01/archivelog/stdby/arc_2_9376_789583539.dbf'
RFS[2]: Archived Log: '/u01/archivelog/stdby/arc_2_9377_789583539.dbf'
RFS[2]: Archived Log: '/u01/archivelog/stdby/arc_2_9378_789583539.dbf'
RFS[2]: Archived Log: '/u01/archivelog/stdby/arc_2_9379_789583539.dbf'
RFS[2]: Archived Log: '/u01/archivelog/stdby/arc_2_9380_789583539.dbf'
Sat Mar 23 06:42:41 CST 2013
RFS[1]: Archived Log: '/u01/archivelog/stdby/arc_1_7751_789583539.dbf'
Sat Mar 23 06:42:42 CST 2013
RFS[2]: Archived Log: '/u01/archivelog/stdby/arc_2_9381_789583539.dbf'
Sat Mar 23 06:42:42 CST 2013
Managed Standby Recovery starting Real Time Apply
Sat Mar 23 06:42:42 CST 2013
Errors in file /u01/app/oracle/admin/stdby/bdump/stdby_dbw0_11458.trc:
ORA-01157: cannot identify/lock data file 24 - see DBWR trace file
ORA-01110: data file 24: '+DATA/in_sz_data'
ORA-17503: ksfdopn:2 Failed to open file +DATA/in_sz_data
ORA-15001: diskgroup "DATA" does not exist or is not mounted
ORA-15077: could not locate ASM instance serving a required diskgroup
ORA-29701: unable to connect to Cluster Manager
Sat Mar 23 06:42:42 CST 2013
Errors in file /u01/app/oracle/admin/stdby/bdump/stdby_dbw0_11458.trc:
ORA-01157: cannot identify/lock data file 25 - see DBWR trace file
ORA-01110: data file 25: '/u01/oradata/stdby/in_sz_data'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sat Mar 23 06:42:42 CST 2013
Errors in file /u01/app/oracle/admin/stdby/bdump/stdby_dbw0_11458.trc:
ORA-01157: cannot identify/lock data file 26 - see DBWR trace file
ORA-01110: data file 26: '/u01/oradata/stdby/in_sz_data.9154.810742355'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
Sat Mar 23 06:42:42 CST 2013
Errors in file /u01/app/oracle/admin/stdby/bdump/stdby_dbw0_11458.trc:
ORA-01157: cannot identify/lock data file 27 - see DBWR trace file
ORA-01110: data file 27: '/u01/oradata/stdby/in_ac_data.9090.810742665'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
MRP0: Background Media Recovery terminated with error 1110
Sat Mar 23 06:42:43 CST 2013
Errors in file /u01/app/oracle/admin/stdby/bdump/stdby_mrp0_11519.trc:
ORA-01110: data file 24: '+DATA/in_sz_data'
ORA-01157: cannot identify/lock data file 24 - see DBWR trace file
ORA-01110: data file 24: '+DATA/in_sz_data'
Managed Standby Recovery not using Real Time Apply
Sat Mar 23 06:42:43 CST 2013
Errors in file /u01/app/oracle/admin/stdby/bdump/stdby_mrp0_11519.trc:
ORA-01110: data file 24: '+DATA/in_sz_data'
ORA-01157: cannot identify/lock data file 24 - see DBWR trace file
ORA-01110: data file 24: '+DATA/in_sz_data'
Sat Mar 23 06:42:43 CST 2013
MRP0: Background Media Recovery process shutdown (stdby)
Sat Mar 23 06:42:43 CST 2013
RFS[2]: Archived Log: '/u01/archivelog/stdby/arc_2_9382_789583539.dbf'
分析：因为主库新建过数据文件，从主库恢复过来的控制文件中中包含了这些文件，而从库却没有这些文件。

解决方法：
执行创建文件：
SQL> alter database create datafile '+DATA/in_sz_data' as '/u01/oradata/stdby/in_sz_data.312.789659222';
Database altered.
SQL> alter database create datafile '/u01/oradata/stdby/in_sz_data' as '/u01/oradata/stdby/in_sz_data.312.789659223';
Database altered.
SQL> alter database create datafile '/u01/oradata/stdby/in_sz_data.9154.810742355' as '/u01/oradata/stdby/in_sz_data.312.789659224';
Database altered.
SQL> alter database create datafile '/u01/oradata/stdby/in_ac_data.9090.810742665' as '/u01/oradata/stdby/in_ac_data.9090.810742665';

再次执行恢复：
RMAN> catalog start with '/u01/rmanbak/fordg/';
 
searching for all files that match the pattern /u01/rmanbak/fordg/
 
List of Files Unknown to the Database
=====================================
File Name: /u01/rmanbak/fordg/standby_STD_20130323_n9o57oao_1_1.bak
File Name: /u01/rmanbak/fordg/control01.ctl
File Name: /u01/rmanbak/fordg/standby_STD_20130323_nbo57oao_1_1.bak
File Name: /u01/rmanbak/fordg/standby_STD_20130323_nfo57p2c_1_1.bak
File Name: /u01/rmanbak/fordg/standby_STD_20130323_nco57oao_1_1.bak
File Name: /u01/rmanbak/fordg/standby_STD_20130323_nao57oao_1_1.bak
File Name: /u01/rmanbak/fordg/standby_STD_20130323_ngo57p2h_1_1.bak
File Name: /u01/rmanbak/fordg/standby_STD_20130323_ndo57ov7_1_1.bak
File Name: /u01/rmanbak/fordg/standby_STD_20130323_neo57p0b_1_1.bak
 
Do you really want to catalog the above files (enter YES or NO)? yes
cataloging files...
cataloging done
 
List of Cataloged Files
=======================
File Name: /u01/rmanbak/fordg/standby_STD_20130323_n9o57oao_1_1.bak
File Name: /u01/rmanbak/fordg/control01.ctl
File Name: /u01/rmanbak/fordg/standby_STD_20130323_nbo57oao_1_1.bak
File Name: /u01/rmanbak/fordg/standby_STD_20130323_nfo57p2c_1_1.bak
File Name: /u01/rmanbak/fordg/standby_STD_20130323_nco57oao_1_1.bak
File Name: /u01/rmanbak/fordg/standby_STD_20130323_nao57oao_1_1.bak
File Name: /u01/rmanbak/fordg/standby_STD_20130323_ngo57p2h_1_1.bak
File Name: /u01/rmanbak/fordg/standby_STD_20130323_ndo57ov7_1_1.bak
File Name: /u01/rmanbak/fordg/standby_STD_20130323_neo57p0b_1_1.bak
 
RMAN>

RMAN> RMAN> run {  
allocate channel dsk0 type disk;
allocate channel dsk1 type disk;
allocate channel dsk2 type disk;
restore standby controlfile to '/u01/rmanbak/fordg/control01.ctl';
recover database noredo;
}2> 3> 4> 5> 6> 7>
 
allocated channel: dsk0
channel dsk0: sid=1481 devtype=DISK
 
allocated channel: dsk1
channel dsk1: sid=1480 devtype=DISK
 
allocated channel: dsk2
channel dsk2: sid=1479 devtype=DISK
 
Starting restore at 28-MAR-13
 
channel dsk0: starting datafile backupset restore
channel dsk0: restoring control file
output filename=/u01/rmanbak/fordg/control01.ctl
channel dsk0: reading from backup piece /u01/rmanbak/fordg/standby_STD_20130323_nfo57p2c_1_1.bak
channel dsk0: restored backup piece 1
piece handle=/u01/rmanbak/fordg/standby_STD_20130323_nfo57p2c_1_1.bak tag=FOR STANDBY
channel dsk0: restore complete, elapsed time: 00:00:05
Finished restore at 28-MAR-13
 
Starting recover at 28-MAR-13
channel dsk0: starting incremental datafile backupset restore
channel dsk0: specifying datafile(s) to restore from backup set
destination for restore of datafile 00024: /u01/oradata/stdby/in_sz_data.312.789659222
destination for restore of datafile 00025: /u01/oradata/stdby/in_sz_data.312.789659223
destination for restore of datafile 00026: /u01/oradata/stdby/in_sz_data.312.789659224
destination for restore of datafile 00027: /u01/oradata/stdby/in_ac_data.9090.810742665
channel dsk0: reading from backup piece /u01/rmanbak/inc2_STD_n2o572go_1_1
released channel: dsk0
released channel: dsk1
released channel: dsk2
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 03/28/2013 15:37:14
ORA-19870: error reading backup piece /u01/rmanbak/inc2_STD_n2o572go_1_1
ORA-19505: failed to identify file "/u01/rmanbak/inc2_STD_n2o572go_1_1"
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
 
分析：通过WINDOWS系统FTP上传下载后文件发生了变化，改用NFS传输后恢复正常
顺带下NFS的设置
服务器端：
[root@dgserver stdby]# vi /etc/exports
 
[1]+  Stopped                 vi /etc/exports
[root@dgserver stdby]# vi /etc/exports
[root@dgserver stdby]# /etc/rc.d/init.d/portmap start
Starting portmap:                                          [  OK  ]
[root@dgserver stdby]# /etc/rc.d/init.d/nfs start
Starting NFS services:                                     [  OK  ]
Starting NFS quotas:                                       [  OK  ]
Starting NFS daemon:                                       [  OK  ]
Starting NFS mountd:                                       [  OK  ]
[root@dgserver stdby]# exportfs -rv
exporting *:/u01/rmanbak
[root@dgserver stdby]#

客户端
[root@oracle1 ~]# showmount -e 192.168.13.109
Export list for 192.168.13.109:
/u01/rmanbak *
[root@oracle1 ~]#

 

 
RMAN> run {  
allocate channel dsk0 type disk;
allocate channel dsk1 type disk;
allocate channel dsk2 type disk;
restore standby controlfile to '/u01/rmanbak/fordg/control01.ctl';
recover database noredo;
}2> 3> 4> 5> 6> 7>
 
allocated channel: dsk0
channel dsk0: sid=1481 devtype=DISK
 
allocated channel: dsk1
channel dsk1: sid=1480 devtype=DISK
 
allocated channel: dsk2
channel dsk2: sid=1479 devtype=DISK
 
Starting restore at 28-MAR-13
 
control file is already restored to file /u01/rmanbak/fordg/control01.ctl
restore not done; all files readonly, offline, or already restored
Finished restore at 28-MAR-13
 
Starting recover at 28-MAR-13
channel dsk0: starting incremental datafile backupset restore
channel dsk0: specifying datafile(s) to restore from backup set
destination for restore of datafile 00024: /u01/oradata/stdby/in_sz_data.312.789659222
destination for restore of datafile 00025: /u01/oradata/stdby/in_sz_data.312.789659223
destination for restore of datafile 00026: /u01/oradata/stdby/in_sz_data.312.789659224
destination for restore of datafile 00027: /u01/oradata/stdby/in_ac_data.9090.810742665
channel dsk0: reading from backup piece /u01/rmanbak/inc2_STD_n2o572go_1_1
released channel: dsk0
released channel: dsk1
released channel: dsk2
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 03/28/2013 16:32:26
ORA-19870: error reading backup piece /u01/rmanbak/inc2_STD_n2o572go_1_1
ORA-19573: cannot obtain exclusive enqueue for datafile 26

查询该FILE
SQL> select name from v$datafile where file#=26;
 
NAME
--------------------------------------------------------------------------------
/u01/oradata/stdby/in_sz_data.312.789659224
 
SQL>
分析：由于是MOUNT STANDBY方式启动的，决定重启到普通MOUNT状态
SQL> shutdown immediate;
ORACLE instance shut down.
SQL> startup mount;
ORACLE instance started.
 
Total System Global Area 1.9327E+10 bytes
Fixed Size                  2130592 bytes
Variable Size            4362079584 bytes
Database Buffers         1.4948E+10 bytes
Redo Buffers               14643200 bytes
Database mounted.
SQL>

 

[oracle@dgserver ~]$ rman target /
 
Recovery Manager: Release 10.2.0.5.0 - Production on Thu Mar 28 16:49:28 2013
 
Copyright (c) 1982, 2007, Oracle.  All rights reserved.
 
connected to target database: STD (DBID=1656746419, not open)
 
 
RMAN> run {  
allocate channel dsk0 type disk;
allocate channel dsk1 type disk;
allocate channel dsk2 type disk;
restore standby controlfile to '/u01/rmanbak/fordg/control01.ctl';
recover database noredo;
}2> 3> 4> 5> 6> 7>
 
using target database control file instead of recovery catalog
allocated channel: dsk0
channel dsk0: sid=1481 devtype=DISK
 
allocated channel: dsk1
channel dsk1: sid=1480 devtype=DISK
 
allocated channel: dsk2
channel dsk2: sid=1479 devtype=DISK
 
Starting restore at 28-MAR-13
 
control file is already restored to file /u01/rmanbak/fordg/control01.ctl
restore not done; all files readonly, offline, or already restored
Finished restore at 28-MAR-13
 
Starting recover at 28-MAR-13
channel dsk0: starting incremental datafile backupset restore
channel dsk0: specifying datafile(s) to restore from backup set
destination for restore of datafile 00024: /u01/oradata/stdby/in_sz_data.312.789659222
destination for restore of datafile 00025: /u01/oradata/stdby/in_sz_data.312.789659223
destination for restore of datafile 00026: /u01/oradata/stdby/in_sz_data.312.789659224
destination for restore of datafile 00027: /u01/oradata/stdby/in_ac_data.9090.810742665
channel dsk0: reading from backup piece /u01/rmanbak/inc2_STD_n2o572go_1_1
channel dsk0: restored backup piece 1
piece handle=/u01/rmanbak/inc2_STD_n2o572go_1_1 tag=INC2
channel dsk0: restore complete, elapsed time: 00:24:27
channel dsk0: starting incremental datafile backupset restore
channel dsk0: specifying datafile(s) to restore from backup set
destination for restore of datafile 00024: /u01/oradata/stdby/in_sz_data.312.789659222
destination for restore of datafile 00025: /u01/oradata/stdby/in_sz_data.312.789659223
channel dsk0: reading from backup piece /u01/rmanbak/fordg/standby_STD_20130323_nbo57oao_1_1.bak
channel dsk1: starting incremental datafile backupset restore
channel dsk1: specifying datafile(s) to restore from backup set
destination for restore of datafile 00026: /u01/oradata/stdby/in_sz_data.312.789659224
channel dsk1: reading from backup piece /u01/rmanbak/fordg/standby_STD_20130323_nco57oao_1_1.bak
channel dsk2: starting incremental datafile backupset restore
channel dsk2: specifying datafile(s) to restore from backup set
destination for restore of datafile 00027: /u01/oradata/stdby/in_ac_data.9090.810742665
channel dsk2: reading from backup piece /u01/rmanbak/fordg/standby_STD_20130323_neo57p0b_1_1.bak
channel dsk0: restored backup piece 1
piece handle=/u01/rmanbak/fordg/standby_STD_20130323_nbo57oao_1_1.bak tag=FOR STANDBY
channel dsk0: restore complete, elapsed time: 00:00:03
channel dsk1: restored backup piece 1
piece handle=/u01/rmanbak/fordg/standby_STD_20130323_nco57oao_1_1.bak tag=FOR STANDBY
channel dsk1: restore complete, elapsed time: 00:00:10
channel dsk2: restored backup piece 1
piece handle=/u01/rmanbak/fordg/standby_STD_20130323_neo57p0b_1_1.bak tag=FOR STANDBY
channel dsk2: restore complete, elapsed time: 00:10:35
 
Finished recover at 28-MAR-13
released channel: dsk0
released channel: dsk1
released channel: dsk2
 
RMAN>