细粒度审计导致SYSTEM表空间异常引发ORA-01653同时性能异常

故障现象：用户反馈无法登录，WEBLOGIC日志显示ERROR 2010-08-17 23:05:36,889 JDBCExceptionReporter:logExceptions - ORA-01653: unable to extend table SYS.FGA_LOG$ by 8192 in tablespace SYSTEM

 

推理：从ORA-01653: unable to extend table SYS.FGA_LOG$ by 8192 in tablespace SYSTEM分析，可能是SYSTEM表空间的问题，通过OEM查看发现SYSTEM表空间确实已经满。但是应用方反馈没有对SYSTEM相关的操作。

 

继续分析：由于用户反馈昨天18：00开始进行应用加压测试，但24点应用出错，后台提示

ORA-01653错误。为了精确定位，采用OEM的ADDM采集了18：00-24：00的信息，从报告中发现如下语句占据了50%左右的开销。

insert into sys.fga_log$ (sessionid, ntimestamp#, dbuid, osuid, obj$schema, obj$name, policyname, scn, oshst, clientid, extid, lsqltext, proxy$sid,user$guid, instance#, process#, xid, statement, entryid, stmt_type, lsqlbind, auditid) values( :1, SYS_EXTRACT_UTC(SYSTIMESTAMP), :2, :3, :4, :5, :6, :7, :8, :9, :10, :11, :12, :13, :14, :15, :16, :17, :18, :19, :20, :21 )

很显然，从sys.fga_log$可以看出以上是一个细粒度审计。

向应用方提出，但是应用方反馈未加审计策略。只好出手找证据。

------------------------------------------------------------------------------------------

SETP1：通过select * from DBA_AUDIT_POLICIES;找出审计事项：

SQL> select * from DBA_AUDIT_POLICIES;

OBJECT_SCHEMA                  OBJECT_NAME                    POLICY_NAME                    POLICY_TEXT                                                                      POLICY_COLUMN                  PF_SCHEMA                      PF_PACKAGE                     PF_FUNCTION                    ENABLED SEL INS UPD DEL AUDIT_TRAIL  POLICY_COLUMN_OPTIONS

------------------------------ ------------------------------ ------------------------------ -------------------------------------------------------------------------------- ------------------------------ ------------------------------ ------------------------------ ------------------------------ ------- --- --- --- --- ------------ ---------------------

OPERATION                      SK_SSO_GRANTING_TICKET         FGA_SK_SSO_GRANTING_TICKET                                                                                                                                                                                                                  YES     YES YES YES YES DB+EXTENDED  ANY_COLUMNS

 

再通过sys.fga_log$来看审计内容，很显然对表更新做了审计。

SQL> SELECT dbuid, lsqltext FROM sys.fga_log$;

 

DBUID                          LSQLTEXT

------------------------------ --------

OPERATION                      insert i

OPERATION                      update S

OPERATION                      select s

OPERATION                      insert i

计算下条数165万条。

SQL> select count(*) from sys.fga_log$;

  COUNT(*)

----------

   1650749

 

此时把审计对象反馈给应用，一轮闻讯下来总算有人”坦白”了。 

----------------------------------------------------------------------------------

SETP2：执行execute dbms_fga.drop_policy去除审计

SQL> execute dbms_fga.drop_policy(object_schema=>'OPERATION',object_name=>'SK_SSO_GRANTING_TICKET',policy_name=>'FGA_SK_SSO_GRANTING_TICKET');

 

PL/SQL procedure successfully completed

----------------------------------------------------------------------------------

SETP3：重复SETP1发现审计消失

SQL> select * from DBA_AUDIT_POLICIES;

 

OBJECT_SCHEMA                  OBJECT_NAME                    POLICY_NAME                    POLICY_TEXT                                                                      POLICY_COLUMN                  PF_SCHEMA                      PF_PACKAGE                     PF_FUNCTION                    ENABLED SEL INS UPD DEL AUDIT_TRAIL  POLICY_COLUMN_OPTIONS

------------------------------ ------------------------------ ------------------------------ -------------------------------------------------------------------------------- ------------------------------ ------------------------------ ------------------------------ ------------------------------ ------- --- --- --- --- ------------ ---------------------

 

SQL>

删除现有审计数据：

SQL> truncate table Sys.fga_log$;

Table truncated