FAL[server, ARC3]: FAL archive failed Error 16401 creating standby archive log file at host

主库alert相关报错如下：

Tue Nov 17 21:04:26 2015

ARC3: Archive log rejected (thread 2 sequence 11378) at host 'PRODS'
FAL[server, ARC3]: FAL archive failed, see trace file.
ARCH: FAL archive failed. Archiver continuing
ORACLE Instance PROD2 - Archival Error. Archiver continuing.
Tue Nov 17 21:10:05 2015

查找主库相关trace：

[root@cisdb02 trace]# ls -lrt *arc*
-rw-r----- 1 oracle asmadmin  446473 Nov 19 07:01 PROD2_arc3_92713.trm
-rw-r----- 1 oracle asmadmin 3534044 Nov 19 07:01 PROD2_arc3_92713.trc

*** 2015-11-17 21:04:26.332
Error 16401 creating standby archive log file at host 'PRODS'
kcrrwkx: unknown error:16401
OSSIPC:SKGXP:[c9ed480.38243]{0}: (92713 <- 13377)SKGXPDOAINVALCON: connection 0xcb4ca70 admno 0x336c50eb scoono 0x69c551c8 acconn 0x534de827 getting closed. inactive: 0

备库alert相关报错如下：

Tue Nov 17 21:04:26 2015
ARC3: Archive log rejected (thread 2 sequence 11378) at host 'PRODS'
FAL[server, ARC3]: FAL archive failed, see trace file.
ARCH: FAL archive failed. Archiver continuing
ORACLE Instance PROD2 - Archival Error. Archiver continuing.
Tue Nov 17 21:10:05 2015

增加日志：

+ standby 
Tue Nov 17 21:02:11 2015 
RFS[115]: Selected log 12 for thread 2 sequence 11378 dbid 244525669 branch 832687336 
Tue Nov 17 21:02:17 2015 
Archived Log entry 7021 added for thread 2 sequence 11377 ID 0xe935962 dest 1: 
Tue Nov 17 21:02:21 2015 
Media Recovery Log /u01/CCBPROD/oracle/oradata/archive/prod_11377_2_832687336.arc 
Tue Nov 17 21:04:11 2015 
RFS[115]: Selected log 11 for thread 2 sequence 11379 dbid 244525669 branch 832687336 

+ primary 
Tue Nov 17 21:02:11 2015 
Thread 2 advanced to log sequence 11378 (LGWR switch) 
Current log# 16 seq# 11378 mem# 0: +DATA_CIS/prod/onlinelog/group_16.343.862077675 
Current log# 16 seq# 11378 mem# 1: +RECO_CIS/prod/onlinelog/group_16.1464.862077677 
Tue Nov 17 21:02:11 2015 
LNS: Standby redo logfile selected for thread 2 sequence 11378 for destination LOG_ARCHIVE_DEST_2 
Tue Nov 17 21:02:16 2015 
Archived Log entry 48606 added for thread 2 sequence 11377 ID 0xe935962 dest 1: 
Tue Nov 17 21:04:08 2015 
Thread 2 advanced to log sequence 11379 (LGWR switch) 
Current log# 4 seq# 11379 mem# 0: +DATA_CIS/prod/onlinelog/group_4.278.841237531 
Current log# 4 seq# 11379 mem# 1: +RECO_CIS/prod/onlinelog/group_4.260.841237531 
Tue Nov 17 21:04:11 2015 
LNS: Standby redo logfile selected for thread 2 sequence 11379 for destination LOG_ARCHIVE_DEST_2 
Tue Nov 17 21:04:13 2015 
Archived Log entry 48608 added for thread 2 sequence 11378 ID 0xe935962 dest 1: 
Tue Nov 17 21:04:26 2015 
ARC3: Archive log rejected (thread 2 sequence 11378) at host 'PRODS' 
FAL[server, ARC3]: FAL archive failed, see trace file. 
ARCH: FAL archive failed. Archiver continuing 
ORACLE Instance PROD2 - Archival Error. Archiver continuing. 

问题原因：

This is false gap so can be ignored: 
primary switched from 11378 to 11379 in 2 mins, meanwhile standby has not finished archiving 11378 before it saw 11379, hence gap was detected and FAL sent to get 11378 from primary. however it was not needed. 

More information please see following note: 
ORA-16401 and ORA-16055 reported in primary alert.log when redolog switch is over frequently ( Doc ID 1243177.1 )

there is no real gap so error can be ignored. make sure log file switch doesn't happen that frequently so there will be no false gap detected. 

ignore the error as there is no real gap. Also increase the size of online redo log of primary database so it won't switch frequently.

ignore the error as there is no real gap. Also increase the size of online redo log of primary database so it won't switch frequently.

ignore the error as there is no real gap. Also increase the size of online redo log of primary database so it won't switch frequently.

ORA-16401 and ORA-16055 reported in primary alert.log when redolog switch is over frequently (文档 ID 1243177.1)

转到底部

---

In this Document Symptoms   Changes   Cause   Solution   References APPLIES TO: Oracle Database - Enterprise Edition - Version 10.2.0.1 to 11.2.0.1.0 [Release 10.2 to 11.2] Linux x86-64 ***Checked for relevance on 31-Aug-2012***  ***Checked for relevance on 8-Jul-2015*** SYMPTOMS - ORA-16401 and ORA-16055 Error reported sometimes at Primary ALERT.LOG: ARC3: Archive log rejected (thread 1 sequence 136480) at host '(DESCRIPTION=  (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.26)(PORT=1530))(CONNECT_DATA=(SERVICE_NAME=sdbinfo)(SERVER=DEDICATED)))' Errors in file /opt/dbinfo/diag/rdbms/dbinfo/dbinfo/trace/dbinfo_arc3_7445.trc: ORA-16401: archivelog rejected by RFS FAL[server, ARC3]: FAL archive failed, see trace file. Errors in file /opt/dbinfo/diag/rdbms/dbinfo/dbinfo/trace/dbinfo_arc3_7445.trc: ORA-16055: FAL request rejected ARCH: FAL archive failed. Archiver continuing ORACLE Instance dbinfo - Archival Error. Archiver continuing. - RedoLog switch happens very frequently before the above Error reported - The requested ArchiveLog by FAL requests the current Log Sequence or the Sequence currently being archived CHANGES There is no change made. CAUSE The Problem here is that the Primary Database is switching Logs too frequently. Using ARCH to send the archives, every time there's a log switch the Primary has to send the Archivelog to the Standby, meanwhile another Log Switch occurred on the Primary which causes also another Archivelog to be sent to the Standby, but the first one has not finished yet, a GAP is formed and detected by the Standby. At this Time the first Archivelog is also sent as FAL Request, but this one will fail because the first one is still being archiving, locked, so the second one fails. Also check for the case explained in below note, Ora-16401: Archivelog Rejected By Rfs (Doc ID 1183143.1) SOLUTION Ignore these Messages as long as the Standby Database keeps synchronized with the Primary Database Increase the Size of the Online Redologs to reduce Redolog Switch Frequency Increase Network Bandwith between the Primary and Standby Database REFERENCES BUG:9848109 - ORA-16401: ARCHIVE LOG REJECTED BY REMOTE FILE SERVER

ignore the error as there is no real gap. Also increase the size of online redo log of primary database so it won't switch frequently.

ignore the error as there is no real gap. Also increase the size of online redo log of primary database so it won't switch frequently.