[2021-01-04] ogg使用reperror处理1403和主键冲突问题
--//源端和目标端都创建测试表T7
SQL> create table t7(id int primary key);
Table created.
SQL> --//目标端先插入数据
SQL> insert into t7 values (10);
1 row created.
SQL> commit;
Commit complete.
SQL> --//源端插入数据
SQL> insert into t7 values (10);
1 row created.
SQL> commit;
Commit complete.
SQL>
--//查看目标的rep进程
GGSCI (QXY1.localdomain) 26> view report rora_hr
.
.
.
Wildcard MAP resolved (entry postgres.*):
map "POSTGRES"."T7" ,target postgres."T7";
Using following columns in default map by name:
ID
Using the following key columns for target table POSTGRES.T7: ID.
2021-01-01 00:52:06 WARNING OGG-00869 OCI Error ORA-00001: unique constraint (POSTGRES.SYS_C0011488) violated (status = 1), SQL .
2021-01-01 00:52:06 WARNING OGG-01004 Aborted grouped transaction on 'POSTGRES.T7', Database error 1 (OCI Error ORA-00001: unique c
onstraint (POSTGRES.SYS_C0011488) violated (status = 1), SQL ).
2021-01-01 00:52:06 WARNING OGG-01003 Repositioning to rba 3843 in seqno 2.
2021-01-01 00:52:06 WARNING OGG-01154 SQL error 1 mapping POSTGRES.T7 to POSTGRES.T7 OCI Error ORA-00001: unique constraint (POSTGR
ES.SYS_C0011488) violated (status = 1), SQL .
2021-01-01 00:52:06 WARNING OGG-01003 Repositioning to rba 3843 in seqno 2.
--//查看目标端的dsc文件
[oracle@QXY1 dirrpt]$ vi rora_hr.dsc
Oracle GoldenGate Delivery for Oracle process started, group RORA_HR discard file opened: 2021-01-01 00:49:02.976092
Current time: 2021-01-01 00:52:06
Discarded record from action ABEND on error 1
OCI Error ORA-00001: unique constraint (POSTGRES.SYS_C0011488) violated (status = 1), SQL
Aborting transaction on ./dirdat/pg beginning at seqno 2 rba 3843
error at seqno 2 rba 3843
Problem replicating POSTGRES.T7 to POSTGRES.T7
Mapping problem with insert record (target format)...
*
ID = 10
000000: 31 30 |10 |
*
Process Abending : 2021-01-01 00:52:06
--//目标的rep进程添加reperror参数
GGSCI (QXY1.localdomain) 1> edit params rora_hr
replicat rora_hr
setenv (ORACLE_SID=QXY2)
setenv (ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1)
setenv (NLS_LANG=AMERICAN_AMERICA.AL32UTF8)
userid goldengate,password goldengate
--handlecollisions
assumetargetdefs
NOALLOWDUPTARGETMAP
discardfile ./dirrpt/rora_hr.dsc,purge
--reperror default, discard
reperror (1, discard)
ddlerror default ignore
tableexclude hr.t4
map postgres.t5, target postgres.t5, filter (@GETENV('transaction','csn') >1992341);
map postgres.* ,target postgres.*;--//启动rep进程，正常启动
GGSCI (QXY1.localdomain) 2> start rora_hr
Sending START request to MANAGER ...
REPLICAT RORA_HR starting
GGSCI (QXY1.localdomain) 3> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
REPLICAT RUNNING RORA_HR 00:00:00 00:00:01 --//查看report日志
Wildcard MAP resolved (entry postgres.*):
map "POSTGRES"."T7" ,target postgres."T7";
Using following columns in default map by name:
ID
Using the following key columns for target table POSTGRES.T7: ID.--//查看目标端的dsc文件
[oracle@QXY1 dirrpt]$ cat rora_hr.dsc
Oracle GoldenGate Delivery for Oracle process started, group RORA_HR discard file opened: 2021-01-01 00:55:02.147733
Current time: 2021-01-01 00:55:02
OCI Error ORA-00001: unique constraint (POSTGRES.SYS_C0011488) violated (status = 1), SQL
Operation failed at seqno 2 rba 3843
Discarding record on action DISCARD on error 1
Problem replicating POSTGRES.T7 to POSTGRES.T7
Mapping problem with insert record (target format)...
*
ID = 10
000000: 31 30 |10 |
*
[oracle@QXY1 dirrpt]$
--//reperror 可以只针对某一张表，修改参数文件如下，只处理t7表的主键错误
GGSCI (QXY1.localdomain) 24> view params rora_hr
replicat rora_hr
setenv (ORACLE_SID=QXY2)
setenv (ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1)
setenv (NLS_LANG=AMERICAN_AMERICA.AL32UTF8)
userid goldengate,password goldengate
--handlecollisions
assumetargetdefs
NOALLOWDUPTARGETMAP
discardfile ./dirrpt/rora_hr.dsc,purge
--reperror default, discard
reperror (1, discard)
map postgres.t7, target postgres.t7; <=====只针对t7这张表，之后reperror reset
reperror reset
ddlerror default ignore
tableexclude hr.t4
map postgres.t5, target postgres.t5, filter (@GETENV('transaction','csn') >1992341);
map postgres.* ,target postgres.*;--//测试如下
--//多创建一个t8表，源端和目标端都创建
--//目标端先插入记录10
SQL>
SQL> create table t8(id int primary key);
Table created.
SQL> insert into t8 values (1);
1 row created.
SQL> commit;
Commit complete.
SQL>
—//源端插入记录10
SQL>
SQL> create table t8(id int primary key);
Table created.
SQL> insert into t8 values (1);
1 row created.
SQL> commit;
Commit complete.
SQL> --//查看目标端的r进程
GGSCI (QXY1.localdomain) 22> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
REPLICAT ABENDED RORA_HR 00:00:07 00:00:10
GGSCI (QXY1.localdomain) 23>
GGSCI (QXY1.localdomain) 23> view report rora_hr
Wildcard MAP resolved (entry postgres.*):
map "POSTGRES"."T8" ,target postgres."T8";
Using following columns in default map by name:
ID
Using the following key columns for target table POSTGRES.T8: ID.
2021-01-01 01:19:34 WARNING OGG-00869 OCI Error ORA-00001: unique constraint (POSTGRES.SYS_C0011489) violated (status = 1), SQL .
2021-01-01 01:19:34 WARNING OGG-01004 Aborted grouped transaction on 'POSTGRES.T8', Database error 1 (OCI Error ORA-00001: unique c
onstraint (POSTGRES.SYS_C0011489) violated (status = 1), SQL ).
2021-01-01 01:19:34 WARNING OGG-01003 Repositioning to rba 4432 in seqno 2.
2021-01-01 01:19:34 WARNING OGG-01154 SQL error 1 mapping POSTGRES.T8 to POSTGRES.T8 OCI Error ORA-00001: unique constraint (POSTGR
ES.SYS_C0011489) violated (status = 1), SQL .
2021-01-01 01:19:34 WARNING OGG-01003 Repositioning to rba 4432 in seqno 2.
--//处理1403的问题一样
--//源端和目标的t7表数据一致，都是
SQL> select * from t7;
ID
----------
2
3
4
5
10
SQL>--//先更新目标端的t7表 id=10的记录
SQL> update t7 set id = 11 where id = 10;
1 row updated.
SQL> commit;
Commit complete.
SQL> --//源端同样更新id=10的记录
SQL> update t7 set id = 11 where id = 10;
1 row updated.
SQL> commit;
Commit complete.
SQL> --//查看目标端的rep进程
GGSCI (QXY1.localdomain) 32> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
REPLICAT ABENDED RORA_HR 00:00:06 00:00:01 --//report日志
GGSCI (QXY1.localdomain) 33> view report rora_hr
MAP resolved (entry postgres.t7):
map "POSTGRES"."T7", target postgres.t7;
Using following columns in default map by name:
ID
Using the following key columns for target table POSTGRES.T7: ID.
Wildcard MAP resolved (entry postgres.*):
map "POSTGRES"."T7" ,target postgres."T7";
2021-01-01 01:31:48 WARNING OGG-02081 Detected duplicate TABLE/MAP entry for source table POSTGRES.T7 and target table postgres.T7.
Using prior TABLE/MAP specification.
2021-01-01 01:31:48 WARNING OGG-01004 Aborted grouped transaction on 'POSTGRES.T7', Database error 1403 (OCI Error ORA-01403: no da
ta found, SQL ).
2021-01-01 01:31:48 WARNING OGG-01003 Repositioning to rba 4555 in seqno 2.
2021-01-01 01:31:48 WARNING OGG-01154 SQL error 1403 mapping POSTGRES.T7 to POSTGRES.T7 OCI Error ORA-01403: no data found, SQL .
2021-01-01 01:31:48 WARNING OGG-01003 Repositioning to rba 4555 in seqno 2.
--//修改目标库的reperror参数再次启动
GGSCI (QXY1.localdomain) 34> edit params rora_hr
replicat rora_hr
setenv (ORACLE_SID=QXY2)
setenv (ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1)
setenv (NLS_LANG=AMERICAN_AMERICA.AL32UTF8)
userid goldengate,password goldengate
--handlecollisions
assumetargetdefs
NOALLOWDUPTARGETMAP
discardfile ./dirrpt/rora_hr.dsc,purge
--reperror default, discard
reperror (1403, discard)
map postgres.t7, target postgres.t7;
--reperror reset
ddlerror default ignore
tableexclude hr.t4
map postgres.t5, target postgres.t5, filter (@GETENV('transaction','csn') >1992341);
map postgres.* ,target postgres.*;
~--//再次启动查看
GGSCI (QXY1.localdomain) 35> start rora_hr
Sending START request to MANAGER ...
REPLICAT RORA_HR starting
GGSCI (QXY1.localdomain) 36> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
REPLICAT RUNNING RORA_HR 00:00:00 00:00:05 --//discard记录
Oracle GoldenGate Delivery for Oracle process started, group RORA_HR discard file opened: 2021-01-01 01:33:34.855011
Current time: 2021-01-01 01:33:34
OCI Error ORA-01403: no data found, SQL
Operation failed at seqno 2 rba 4555
Discarding record on action DISCARD on error 1403
Problem replicating POSTGRES.T7 to POSTGRES.T7
Record not found
Mapping problem with compressed key update record (target format)...
*
ID = 10
000000: 31 30 |10 |
ID = 11
000000: 31 31 |11 | 使用reperror的方式处理此问题会导致数据不一致