dw sqlldr example03 a bit of advance

3)dw sqlldr
1@@@@example03 a bit of advanced
guide:
  In this example, aim to express how to speed up your sql*loader.
  ~ disable contstraint,
  ~ rebuild index after loading data,
  ~ use position key word
  ~ loading data to a partition of sales, do not impact other partitions.
  ~ use direct path loading without logging, speed up but danger.

@@@
@@@<1>check the index, create tbs for new partition of table sh.sales.
@@@
@@@In this case, I use table sales of "sh" schema.
SH@ocp> select index_name, index_type,compression from user_indexes where table_name='SALES';
INDEX_NAME               INDEX_TYPE           COMPRESS
------------------------------ --------------------------- --------
LOCAL_SALES_IDX            BITMAP               DISABLED

@@@
SYS@ocp> create tablespace dw_sh_sqlldr datafile size 5M
> autoextend on default
> storage(initial 64k next 64k pctincrease 0 maxextents unlimited);
Tablespace created.

@@@
alter table sh.sales
  add partition sales_q1_2012
  values less than (to_date('01-02-2012','dd-mm-yyyy'))
  pctfree 0 pctused 99
  storage (initial 64k next 64k pctincrease 0)
  tablespace dw_sh_sqlldr
/
SYS@ocp> /
Table altered.

@@@
@@@<2>disable constraints and trigger, however I leave the check constraint.
@@@
@@@disable all constraints, for speed up.
SYS@ocp> ALTER TABLE sh.sales DISABLE CONSTRAINT SALES_TIME_FK;
Table altered.
SYS@ocp> ALTER TABLE sh.sales DISABLE CONSTRAINT SALES_PROMO_FK;
Table altered.
SYS@ocp> ALTER TABLE sh.sales DISABLE CONSTRAINT SALES_PRODUCT_FK;
Table altered.
SYS@ocp> ALTER TABLE sh.sales DISABLE CONSTRAINT SALES_CUSTOMER_FK;
Table altered.
SYS@ocp> ALTER TABLE sh.sales DISABLE CONSTRAINT SALES_CHANNEL_FK;
Table altered.

@@@
SH@ocp> select CONSTRAINT_NAME,CONSTRAINT_TYPE, status from user_constraints where table_name='SALES';
CONSTRAINT_NAME            C STATUS
------------------------------ - --------
SALES_TIME_FK               R DISABLED
SALES_PRODUCT_FK           R DISABLED
SALES_CUSTOMER_FK           R DISABLED
SALES_PROMO_FK               R DISABLED
SYS_C005209               C ENABLED
SYS_C005210               C ENABLED
SYS_C005211               C ENABLED
SYS_C005212               C ENABLED
SYS_C005213               C ENABLED
SYS_C005214               C ENABLED
SYS_C005215               C ENABLED
SALES_CHANNEL_FK           R DISABLED

@@@
@@@make sure all trigger about this table are disable.
SYS@ocp> SELECT trigger_name, status FROM all_triggers WHERE table_name = 'SALES';
no rows selected


@@@
@@@<3>create datafile to test.
@@@
[oracle@station78 dwdir]$ cat sales_2012.dat
  |   |          |   |      |     ||
123123422-AUG-2010123412345671234561
123123416-AUG-2004123412345671234561
123123415-AUG-2011123412345671234561
123123414-AUG-2011123412345671234561
123123412-AUG-2012123412345671234561


@@@
@@@<4>write control file, and loading data.
@@@
[oracle@station78 dwdir]$ cat control02.ctl
options (direct=true)
unrecoverable load data
infile 'sales_2012.dat' badfile 'sales_2012.bad'
append
into table sales
partition (SALES_Q1_2012)
(
 PROD_ID   position (1-3)   integer external,
 CUST_ID   position (4-7)  integer external,
 TIME_ID   position (8-18)  DATE "DD-MON-YYYY", 
 CHANNEL_ID position (19-22)  integer external,
 PROMO_ID   position (23-29)  integer external,
 QUANTITY_SOLD position (30-35) integer external,
 AMOUNT_SOLD   position (36)    integer external
 )

[oracle@station78 dwdir]$ sqlldr USERID=sh/sh
> CONTROL=control02.ctl LOG=control02.log DIRECT=true SKIP_INDEX_MAINTENANCE=true;


@@@
@@@<5>inspect result
@@@
[oracle@station78 dwdir]$ pwd
/home/oracle/dwdir
[oracle@station78 dwdir]$ ls
control02.ctl  control02.log  sales_2012.bad  sales_2012.dat

@@@
[oracle@station78 dwdir]$ cat control02.log
SQL*Loader: Release 10.2.0.1.0 - Production on Wed Aug 22 19:54:44 2012
Copyright (c) 1982, 2005, Oracle.  All rights reserved.

Control File:   control02.ctl
Data File:      sales_2012.dat
  Bad File:     sales_2012.bad
  Discard File:  none specified
 
 (Allow all discards)
Number to load: ALL
Number to skip: 0
Errors allowed: 50
Continuation:    none specified
Path used:      Direct

Load is UNRECOVERABLE; invalidation redo is produced.

Table SALES, partition SALES_Q1_2012, loaded from every logical record.
Insert option in effect for this partition: APPEND

   Column Name                  Position   Len  Term Encl Datatype
------------------------------ ---------- ----- ---- ---- ---------------------
PROD_ID                               1:3     3           CHARACTER           
CUST_ID                               4:7     4           CHARACTER           
TIME_ID                              8:18    11           DATE DD-MON-YYYY    
CHANNEL_ID                          19:22     4           CHARACTER           
PROMO_ID                            23:29     7           CHARACTER           
QUANTITY_SOLD                       30:35     6           CHARACTER           
AMOUNT_SOLD                            36     1           CHARACTER           

Record 1: Rejected - Error on table SALES, column PROD_ID.
ORA-01722: invalid number

Record 6: Rejected - Error on table SALES.
ORA-14400: inserted partition key does not map to any partition

The following index(es) on table SALES were processed:
index SH.LOCAL_SALES_IDX partition SALES_Q1_2012 was made unusable due to:
SKIP_INDEX_MAINTENANCE option requested

Table SALES, partition SALES_Q1_2012:
  4 Rows successfully loaded.
  2 Rows not loaded due to data errors.
  0 Rows not loaded because all WHEN clauses were failed.
  0 Rows not loaded because all fields were null.

  Date cache:
   Max Size:      1000
   Entries :         5
   Hits    :         0
   Misses  :         0

Bind array size not used in direct path.
Column array  rows :    5000
Stream buffer bytes:  256000
Read   buffer bytes: 1048576

Total logical records skipped:          0
Total logical records read:             6
Total logical records rejected:         2
Total logical records discarded:        0
Total stream buffers loaded by SQL*Loader main thread:        2
Total stream buffers loaded by SQL*Loader load thread:        0

Run began on Wed Aug 22 19:54:44 2012
Run ended on Wed Aug 22 19:54:44 2012

Elapsed time was:     00:00:00.64
CPU time was:         00:00:00.02


@@@
@@@<6>summary the falut column
@@@
summary:
  first line was failed in first column checking,
  second line is out of range in date column. The thrid column
  is the partition key column.
[oracle@station78 dwdir]$ cat sales_2012.bad
  |   |          |   |      |     ||
123123412-AUG-2012123412345671234561

@@@
@@@there is another problem, the data what I import using sql*load are not
@@@exist in parent keys.
@@@actually, data must be corresponding with parent key.
SH@ocp> ALTER TABLE sh.sales ENABLE  CONSTRAINT sales_time_fk ;
ALTER TABLE sh.sales ENABLE  CONSTRAINT sales_time_fk
                                        *
ERROR at line 1:
ORA-02298: cannot validate (SH.SALES_TIME_FK) - parent keys not found
 

@@@
@@@use execptions to save exceptions record.
[oracle@station78 ~]$ cd $ORACLE_HOME
[oracle@station78 db_1]$ find ./ -name utlexcpt.sql
./rdbms/admin/utlexcpt.sql
[oracle@station78 db_1]$ cat ./rdbms/admin/utlexcpt.sql
rem
rem $Header: utlexcpt.sql,v 1.1 1992/10/20 11:57:02 GLUMPKIN Stab $
rem
Rem  Copyright (c) 1991 by Oracle Corporation
Rem    NAME
Rem      except.sql - <one-line expansion of the name>
Rem    DESCRIPTION
Rem      <short description of component this file declares/defines>
Rem    RETURNS
Rem
Rem    NOTES
Rem      <other useful comments, qualifications, etc.>
Rem    MODIFIED   (MM/DD/YY)
Rem     glumpkin   10/20/92 -  Renamed from EXCEPT.SQL
Rem     epeeler    07/22/91 -         add comma
Rem     epeeler    04/30/91 -         Creation

create table exceptions(row_id rowid,
                    owner varchar2(30),
                    table_name varchar2(30),
                constraint varchar2(30));

@@@
[oracle@station78 ~]$ echo $ORACLE_HOME
/u01/app/oracle/product/10.2.0/db_1
[oracle@station78 ~]$ sqlplus sh/sh
SH@ocp> @/u01/app/oracle/product/10.2.0/db_1/rdbms/admin/utlexcpt.sql
Table created.

@@@
SH@ocp> ALTER TABLE sales ENABLE CONSTRAINTS sales_channel_fk EXCEPTIONS INTO exceptions;
ALTER TABLE sales ENABLE CONSTRAINTS sales_channel_fk EXCEPTIONS INTO exceptions
                                     *
ERROR at line 1:
ORA-02298: cannot validate (SH.SALES_CHANNEL_FK) - parent keys not found

@@@
@@@find the data using rowid where store in exceptions which are not match parent key.
@@@(-.-)actually, all I input data do not match.
@@@now find the dismatch data, you could remove them.
SH@ocp> select * from exceptions;
ROW_ID           OWNER              TABLE_NAME             CONSTRAINT
------------------ ------------------------------ ------------------------------ ------------------------------
AAANiJAAPAAAAAMAAA SH                  SALES              SALES_CHANNEL_FK
AAANiJAAPAAAAAMAAB SH                  SALES              SALES_CHANNEL_FK
AAANiJAAPAAAAAMAAC SH                  SALES              SALES_CHANNEL_FK
AAANiJAAPAAAAAMAAD SH                  SALES              SALES_CHANNEL_FK
SH@ocp> select * from sales where rowid in (select row_id from exceptions);
   PROD_ID    CUST_ID TIME_ID         CHANNEL_ID   PROMO_ID QUANTITY_SOLD AMOUNT_SOLD
---------- ---------- ------------------ ---------- ---------- ------------- -----------
       123     1234 22-AUG-10            1234    1234567          123456           1
       123     1234 16-AUG-04            1234    1234567          123456           1
       123     1234 15-AUG-11            1234    1234567          123456           1
       123     1234 14-AUG-11            1234    1234567          123456           1


@@@
@@@if you think it is not impact on your db. you could stand these data.
SH@ocp> ALTER TABLE sales ENABLE NOVALIDATE CONSTRAINTS sales_channel_fk;
Table altered.
SH@ocp> ALTER TABLE sales ENABLE NOVALIDATE CONSTRAINTS sales_customer_fk;
Table altered.
SH@ocp> ALTER TABLE sales ENABLE NOVALIDATE CONSTRAINTS sales_product_fk;
Table altered.
SH@ocp> ALTER TABLE sales ENABLE NOVALIDATE CONSTRAINTS sales_promo_fk;
Table altered.
SH@ocp> ALTER TABLE sales ENABLE NOVALIDATE CONSTRAINTS sales_time_fk;

@@@
@@@add RELY for query write
SH@ocp> ALTER TABLE sales MODIFY CONSTRAINTS sales_channel_fk RELY; 
Table altered.
SH@ocp> ALTER TABLE sales MODIFY CONSTRAINTS sales_product_fk RELY;
Table altered.
SH@ocp> ALTER TABLE sales MODIFY CONSTRAINTS sales_promo_fk RELY;
Table altered.
SH@ocp> ALTER TABLE sales MODIFY CONSTRAINTS sales_product_fk RELY;
Table altered.
SH@ocp> ALTER TABLE sales MODIFY CONSTRAINTS sales_time_fk RELY;
Table altered.

@@@
SH@ocp> select CONSTRAINT_NAME,CONSTRAINT_TYPE, status,rely from user_constraints where table_name='SALES';
CONSTRAINT_NAME            C STATUS   RELY
------------------------------ - -------- ----
SALES_CHANNEL_FK           R ENABLED  RELY
SALES_TIME_FK               R ENABLED  RELY
SALES_PRODUCT_FK           R ENABLED  RELY
SALES_CUSTOMER_FK           R ENABLED  RELY
SALES_PROMO_FK               R ENABLED  RELY
SYS_C005209               C ENABLED
SYS_C005210               C ENABLED
SYS_C005211               C ENABLED
SYS_C005212               C ENABLED
SYS_C005213               C ENABLED
SYS_C005214               C ENABLED
SYS_C005215               C ENABLED



2@@@@maintance partition index
@@@Note: N/A means the index was partitioned.
@@@check the unusable index after loading data to your db.
SH@ocp> select index_name, status from user_indexes where table_name='SALES';
INDEX_NAME               STATUS
------------------------------ --------
LOCAL_SALES_IDX            N/A

SH@ocp> select index_name, partition_name, status from user_ind_partitions where status != 'USABLE';
INDEX_NAME               PARTITION_NAME              STATUS
------------------------------ ------------------------------ --------
LOCAL_SALES_IDX            SALES_Q1_2012              UNUSABLE

SH@ocp> ALTER INDEX local_sales_idx REBUILD PARTITION sales_q1_2012;
Index altered.

SH@ocp> select index_name, partition_name, status from user_ind_partitions where status != 'USABLE';
no rows selected