Oracle Golden Gate 系列十一 -- 配置 GG DDL 同步 说明 与 示例

 

一.DDL 说明

1.1 说明

这部分的说明，在系列四和系列六中已经说明：

Oracle Golden Gate 系列六 --11gR2 Ora2Ora 单向复制GG 示例

http://blog.csdn.net/tianlesoftware/article/details/6950018

 

Oracle Golden Gate 系列四--GG 安装 与 卸载 理论知识

http://blog.csdn.net/tianlesoftware/article/details/6937183

 

不过为了便于整理与查阅，还是将DDL 同步这部分内容单独整理出来。

 

GoldenGate的DDL同步只支持两边一致的数据库，限制条件较多（如不能进行字段映射、转换等）。这些限制可以参考：

Oracle Gloden Gate 系列三--GG 支持与不支持的对象类型与操作 说明

http://blog.csdn.net/tianlesoftware/article/details/6933969

 

GG 支持DDL 也是通过创建一些table 来保存这些DDL 的信息，关于这些table 的具体说明，在系列四的第二小节：启用GG 对DDL 操作的支持有详细说明:

Oracle Golden Gate 系列四--GG 安装 与 卸载 理论知识

http://blog.csdn.net/tianlesoftware/article/details/6937183

 

对于这些存放DDL 信息表的管理的理论支持，参考系列五的第四小结：Managing theOracle DDL replication environment。

Oracle Golden Gate 系列五--GG 使用配置 说明

http://blog.csdn.net/tianlesoftware/article/details/6947973

 

这里要注意：DDL的抓取不是通过日志抓取来捕获的，而是通过触发器来实现，所以对源数据库的性能影响要比单纯的数据抓取要大很多。

所以尽量不要使用GoldenGate的DDL复制功能，在大多数业务系统中，实际上不会有频繁的数据库结构变动，完全可以通过手工的方式进行维护。

 

1.2 DDL 参数语法说明

       在配置DDL 复制时，需要在Extract 和Replicat 进程的配置文件里添加ddl属性。 现在看一下这个属性的具体说明。

这个部分内容在官方文档的的ConfiguringDDL synchronization for an Oracle database 小结有详细说明。

DDL语句包含的主要语法如下：

DDL [

{INCLUDE | EXCLUDE}
[, MAPPED | UNMAPPED | OTHER | ALL]
[, OPTYPE <type>]
[, OBJTYPE ‘<type>’]
[, OBJNAME “<name>”]
[, INSTR ‘<string>’]
[, INSTRCOMMENTS ‘<comment_string>’]
]
[...]

 

1.2.1 {INCLUDE | EXCLUDE}

Use INCLUDE and EXCLUDEto identify the beginning of an inclusion or exclusion clause.

(1)  An inclusion clause containsfiltering criteria that identifies the DDL that this parameter will affect.

(2)  An exclusion clause containsfiltering criteria that excludes specific DDL from this parameter.

 

The inclusion orexclusion clause must consist of the INCLUDE or EXCLUDE keyword followed by anyvalid combination of other options of the parameter that is being applied.

 

If you use EXCLUDE, you must create acorresponding INCLUDE clause.

--如果要使用Exclude，那么必须先指定include。

 

For example, the following is invalid:

DDL EXCLUDEOBJNAME “hr.*”

--这种语法是错误的。

However, you can use either of thefollowing:

DDL INCLUDE ALL,EXCLUDE OBJNAME “hr.*”

DDL INCLUDE OBJNAME “fin.*” EXCLUDE “fin.ss”

       --正确语法

 

An EXCLUDE takes priority over any INCLUDEs that contain the same criteria. You can use multipleinclusion and exclusion clauses.

 

1.2.2 [, MAPPED | UNMAPPED | OTHER | ALL]

Use MAPPED, UNMAPPED,OTHER, and ALL to apply INCLUDE or EXCLUDE based on the DDL operation scope.

（1） MAPPED applies INCLUDE or EXCLUDE toDDL operations that are of MAPPED scope. MAPPED filtering is performed beforefiltering that is specified with other DDL parameter options.

（2） UNMAPPED applies INCLUDE or EXCLUDEto DDL operations that are of UNMAPPED scope.

（3） OTHER applies INCLUDE or EXCLUDE toDDL operations that are of OTHER scope.

（4） ALL applies INCLUDE or EXCLUDE toDDL operations of all scopes.

 

       关于这4种scope的不同说明，具体参考官网文档。

 

1.2.3 OPTYPE <type>

Use OPTYPE toapply INCLUDE or EXCLUDE to a specific type of DDL operation, such as CREATE, ALTER,and RENAME. For <type>, use any DDL command that is valid for thedatabase.

For example, to include ALTER operations,the correct syntax is:

DDL INCLUDEOPTYPE ALTER

 

1.2.4 OBJTYPE‘<type>’

Use OBJTYPE toapply INCLUDE or EXCLUDE to a specific type of database object. For <type>,use any object type that is valid for the database, such as TABLE, INDEX, and TRIGGER.For an Oracle materialized view and materialized views log, the correct typesare snapshot and snapshot log, respectively. Enclose the name of the objecttype within single quotes.

 

For example:

DDL INCLUDEOBJTYPE ‘INDEX’

DDL INCLUDEOBJTYPE ‘SNAPSHOT’

 

For Oracle object type USER, do not use theOBJNAME option, because OBJNAME expects “owner.object” whereas USER only has aschema.

 

1.2.5 OBJNAME“<name>”

Use OBJNAME toapply INCLUDE or EXCLUDE to the fully qualified name of an object, for example owner.table_name.This option takes a doublequoted string as input. You can use a wildcard onlyfor the object name.

 

Example:

DDL INCLUDEOBJNAME “accounts.*”

 

Do not use OBJNAME for the Oracle USER object,because OBJNAME expects “owner.object” whereas USER only has a schema.

  When using OBJNAMEwith MAPPED in a Replicat parameter file, the value for OBJNAME must refer tothe name specified with the TARGET clause of the MAP statement. For example,given the following MAP statement, the correct value is OBJNAME “fin2.*”.

MAP fin.exp_*,TARGET fin2.*;

 

In the following example, a CREATE TABLE statementexecutes like this on the source:

CREATE TABLEfin.exp_phone;

 

And like this on the target:

CREATE TABLEfin2.exp_phone;

 

If a targetowner is not specified in the MAP statement, Replicat maps it to the databaseuser that is specified with the USERID parameter.

  For DDL thatcreates triggers, synonyms, and indexes, the value for OBJNAME must be the nameof the base object, not the name of the trigger, synonym, or index.

For example, toinclude the following DDL statement, the correct value is “hr.accounts,” not “hr.insert_trig.”

CREATE TRIGGERhr.insert_trig ON hr.accounts;

 

For RENAME operations,the value for OBJNAME must be the new table name. For example, to include thefollowing DDL statement, the correct value is “hr.acct.”

ALTER TABLEhr.accounts RENAME TO acct;

 

The following is an example of how tocombine DDL parameter options.

DDL &

INCLUDE UNMAPPED &

OPTYPE alter &

OBJTYPE ‘table’ &

OBJNAME “users.tab*” &

INCLUDE MAPPED OBJNAME “*” &

EXCLUDE MAPPED OBJNAME"temporary.tab*"

 

 

1.3 DDL语句格式及注意事项

 1.3.1 同步除Oracle数据库自带的用户外所有用户的DDL操作

在Extract和 Replicat进程中加入下面的内容:

ddl include all

 

       会自动同步所有除系统用户（SYS/SYSTEM）之外的DDL 操作，但是这些对象的DML 操作是否同步则受限与Extract 和 Data Pump 进程里的table 参数控制。如：

 

GGSCI (gg1) 41> view param ext1

 extract ext1

userid ggate@gg1, password ggate

--rmthost gg2,mgrport 7809

--rmttrail /u01/ggate/dirdat/lt

exttrail /u01/ggate/dirdat/lt

ddl include all

table dave.*;

 

1.3.2 只同步一个用户的数据

使用DDL 的语法：

ddl include allobjname dave.*;

 

示例如下：

GGSCI (gg1) 51> view param ext1

 

extract ext1

userid ggate@gg1, password ggate

--rmthost gg2,mgrport 7809

--rmttrail /u01/ggate/dirdat/lt

exttrail /u01/ggate/dirdat/lt

ddl include all objname dave.*;

table dave.*;

 

1.4 启用DDL

开启DDL同步的基本配置步骤为：

（1）关闭ORACLE的回收站功能。

（2）选择一个数据库schema存放支持DDL的GoldenGate对象，运行相应创建脚本。

（3）编辑globals参数文件。

（4）修改extl和repl的配置文件

 

 

具体操作步骤：

（1）关闭数据库回收站：

SQL>alter system set recyclebin=offscope=both;

 

（2）编辑globals参数文件：

GGSCI>edit param ./globals

添加以下内容后保存：

GGSCHEMA ggate  --标明支持DDL的GG对象存放在哪个schema下

 

（3）执行创建脚本：

首先需要命令行进入GG安装目录下，然后再运行sqlplus执行脚本，如果不进入目录下脚本执行会报错（应该是由于GG脚本中子脚本嵌套使用相对路径的问题所造成）。

SQL>@marker_setup.sql    --提示输入目标schema

SQL>@ddl_setup.sql       --提示输入目标schema,输入initialsetup最后输入yes

SQL>@role_setup.sql

SQL>grant GGS_GGSUSER_ROLE to ggate; --不进行该步赋权后面起进程会报错

SQL>@ddl_enable.sql      --使触发器生效

 

（4）修改提取进程和复制进程的配置文件。

       在Extract 里添加：

ddl include all

 

在Replicat 里添加：

ddl include all

ddlerror default ignore retryop       

此时repl必须指定assumetargetdefs属性，这表明只有两边数据库结构一致的情况下才可以启用DDL复制。另外，开启DDL同步不能再只映射单表了，对整个模式下的对象都有效。

1.5 清除DDL 同步设置

如果因为DDL 同步出现很多问题，最简单的方法就是卸载之后重建。安装目录下只提供了清除对象的脚本，可以如下操作：

 

首先要求把所有的GG进程停掉，包括mgr进程

SQL>@ddl_disable.sql  --首先使DDL触发器失效

SQL>@ddl_remove.sql

SQL>@marker_remove.sql

 

role_setup.sql没有对应的清除脚本，但是这块不影响配置信息的清除然后重新再创建脚本。

 

二.  清除DDL 同步示例

因为我之前的环境中已经启用了DDL，所以我们先清除DDL，在启用。

 

2.1 停GG 进程

GGSCI (gg1) 85> info all

Program    Status      Group       Lag           Time Since Chkpt

 

MANAGER    RUNNING                                          

EXTRACT    RUNNING     DPUMP       00:00:00      00:00:04   

EXTRACT    RUNNING     EXT1        00:00:00      00:00:02   

 

GGSCI (gg1) 86> stop ext1

 

Sending STOP request to EXTRACT EXT1 ...

Request processed.

 

 

GGSCI (gg1) 87> stop dpump

 

Sending STOP request to EXTRACT DPUMP ...

Request processed.

 

GGSCI (gg1) 88> stop mgr

Manager process is required by other GGSprocesses.

Are you sure you want to stop it (y/n)? y

 

Sending STOP request to MANAGER ...

Request processed.

Manager stopped.

 

GGSCI (gg1) 89> info all

Program    Status      Group       Lag           Time Since Chkpt

 

MANAGER    STOPPED                                          

EXTRACT    STOPPED     DPUMP       00:00:00      00:00:27   

EXTRACT    STOPPED     EXT1        00:00:00      00:00:33   

 

GGSCI (gg1) 90>

 

2.2 进入GG 根目录，执行脚本

       这里一定要进入根目录，不然执行脚本时会报错。

 

gg1:/home/oracle> cd $GGATE

gg1:/u01/ggate> sqlplus / as sysdba;

SQL*Plus: Release 11.2.0.3.0 Production onWed Nov 16 21:23:28 2011

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

 

Connected to:

Oracle Database 11g Enterprise EditionRelease 11.2.0.3.0 - 64bit Production

With the Partitioning, OLAP, Data Miningand Real Application Testing options

 

SQL> @ddl_disable.sql

Trigger altered.

 

SQL> @ddl_remove.sql

DDL replication removal script.

WARNING: this script removes all DDLreplication objects and data.

 

You will be prompted for the name of aschema for the GoldenGate database objects.

NOTE: The schema must be created prior torunning this script.

 

Enter GoldenGate schema name:ggate

Working, please wait ...

Spooling to file ddl_remove_spool.txt

Script complete.

 

SQL> @marker_remove.sql

 

Marker removal script.

WARNING: this script removes all markerobjects and data.

 

You will be prompted for the name of aschema for the GoldenGate database objects.

NOTE: The schema must be created prior torunning this script.

 

Enter GoldenGate schemaname:ggate

 

PL/SQL procedure successfully completed.

 

Sequence dropped.

Table dropped.

Script complete.

 

三. 启用DDL同步示例

 3.1 禁用数据库回收站

SQL>alter system set recyclebin=off scope=both;

 

如果启用DDL 支持，必须关闭recyclebin。官网的解释如下：

If therecyclebin is enabled, the Oracle GoldenGate DDL trigger session receivesimplicitrecycle bin DDL operations that cause the trigger to fail.

  --注意这在10g里，必须关闭recycle bin，在11g以后的版本，可以不用关闭。

 

3.2 创建存放DDL 信息的user并赋权

SQL> create user ggate identified by ggate default tablespace users temporary tablespace temp;

User created.

SQL> grant connect,resource to ggate;

Grant succeeded.

SQL> grant execute on utl_file to ggate;

Grant succeeded.

 

 

3.3 编辑globals参数文件：

GGSCI>edit param ./globals

添加以下内容后保存：

GGSCHEMA ggate 

--标明支持DDL的GG对象存放在哪个schema下，这个schema就是我们之前创建的schema。

 

3.4 执行安装脚本

注意：退出所有使用Oracle 的session，然后进入GG的安装目录，最后执行sqlplus，使用SYSDBA权限的用户执行如下脚本：

 

SQL>@marker_setup.sql    --提示输入目标schema

SQL>@ddl_setup.sql       --提示输入目标schema,输入initialsetup最后输入yes

SQL>@role_setup.sql

SQL>grant GGS_GGSUSER_ROLE to ddw; --不进行该步赋权后面起进程会报错

SQL>@ddl_enable.sql      --使触发器生效

 

gg1:/u01/ggate> echo $GGATE

/u01/ggate

gg1:/u01/ggate> sqlplus / as sysdba;

 

SQL*Plus: Release 11.2.0.3.0 Production onWed Nov 16 21:39:18 2011

 

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

 

 

Connected to:

Oracle Database 11g Enterprise EditionRelease 11.2.0.3.0 - 64bit Production

With the Partitioning, OLAP, Data Miningand Real Application Testing options

--脚本1

SQL> @marker_setup.sql

 

Marker setup script

 

You will be prompted for the name of aschema for the GoldenGate database objects.

NOTE: The schema must be created prior torunning this script.

NOTE: Stop all DDL replication beforestarting this installation.

 

Enter GoldenGate schema name:ggate

 

Marker setup table script complete, runningverification script...

Please enter the name of a schema for theGoldenGate database objects:

Setting schema name to GGATE

 

MARKER TABLE

-------------------------------

OK

 

MARKER SEQUENCE

-------------------------------

OK

 

Script complete.

 

--脚本2：

SQL> @ddl_setup.sql;

 GoldenGate DDL Replication setupscript

 

Verifying that current user hasprivilegesto install DDL Replication...

 

You will be prompted for the name ofaschema for the GoldenGate database objects.

NOTE: For an Oracle 10g source, thesystemrecycle bin must be disabled. For Oracle 11g and later, it can beenabled.

--注意这里提示我们在10g里，必须关闭recycle bin，在11g以后的版本，可以不用关闭。

NOTE: The schema must be created priortorunning this script.

NOTE: Stop all DDL replicationbeforestarting this installation.

 

--提示输入GG的用户：

Enter GoldenGate schema name:ggate

 

You will be prompted for the modeofinstallation.

To install or reinstall DDLreplication,enter INITIALSETUP

To upgrade DDL replication, enter NORMAL

--这里让我们选择安装模式： install 和 reinstall 选择INITIALSETUP

Enter mode of installation:INITIALSETUP

 

Working, please wait ...

Spooling to file ddl_setup_spool.txt

 

Checking for sessions that are holdinglockson Oracle Golden Gate metadata tables ...

 

Check complete.

 

Using GGATE as a GoldenGate schemaname,INITIALSETUP as a mode of installation.

 

Working, please wait ...

 

DDL replication setup scriptcomplete,running verification script...

Please enter the name of a schema fortheGoldenGate database objects:

Setting schema name to GGATE

 

DDLORA_GETTABLESPACESIZE STATUS:

……

 

STATUS OF DDL REPLICATION

-------------------------------------------------------------------------------------------------------

SUCCESSFUL installation of DDLReplicationsoftware components

 

Script complete.

 

--脚本3：

SQL> @role_setup.sql;

 

GGS Role setup script

 

This script will drop and recreate the roleGGS_GGSUSER_ROLE

To use a different role name, quitthisscript and then edit the params.sql script to change the gg_role parameterto the preferred name. (Do not run the script.)

 

You will be prompted for the name of a schemafor the GoldenGate database objects.

NOTE: The schema must be created priortorunning this script.

NOTE: Stop all DDL replication before startingthis installation.

--同样输入GG用户名：

Enter GoldenGate schema name: ggate

Wrote file role_setup_set.txt

 

PL/SQL procedure successfully completed.

 

Role setup script complete

 

Grant this role to each user assigned totheExtract, GGSCI, and Manager processes, by using the following SQL command:

--这里提示我们赋权给相关的用户：

GRANT GGS_GGSUSER_ROLE TO<loggedUser>

where <loggedUser> is theuserassigned to the GoldenGate processes.

 

--脚本4：赋权

SQL> grant GGS_GGSUSER_ROLE to ggate;

Grant succeeded.

 

--脚本5：

SQL> @ddl_enable.sql;

Trigger altered.

 

注意这里脚本创建的table都是使用默认的名称，当然也可以修改这些table的默认名，具体这块参考之前的文档中的说明。

 

3.5 配置Extract和 Replicat 参数

修改提取进程和复制进程的配置文件，分别加入ddl include all属性。如果只对某个用户，则使用如下格式：

ddl include all objname dave.*;

 

此时repl必须指定assumetargetdefs属性，这表明只有两边数据库结构一致的情况下才可以启用DDL复制。

另外，开启DDL同步不能再只映射单表了，对整个模式下的对象都有效。加入DDL复制之后，数据复制的lag明显增加了。

 

Extract 参数：

GGSCI (gg1) 35> view param ext1

 

extract ext1

userid ggate@gg1, password ggate

--rmthost gg2,mgrport 7809

--rmttrail /u01/ggate/dirdat/lt

exttrail /u01/ggate/dirdat/lt

ddl include all

table dave.*;

 

Data Pump 参数：

GGSCI (gg1) 15> view param dpump

 

extract dpump

userid ggate@gg1, password ggate

rmthost gg2, mgrport 7809

rmttrail /u01/ggate/dirdat/lt

passthru

table dave.*;

 

修改Replicat 参数：

GGSCI (gg2) 170> view param rep1

 

replicat rep1

ASSUMETARGETDEFS

userid ggate@gg2,password ggate

discardfile/u01/ggate/dirdat/rep1_discard.txt, append, megabytes 10

--HANDLECOLLISIONS

ddl include all

ddlerror default ignore retryop       

map dave.*, target dave.*;

 

 

3.6 验证DDL 同步

GGSCI (gg1) 3> start mgr

Manager started.

 

GGSCI (gg1) 4> start ext1

Sending START request to MANAGER ...

EXTRACT EXT1 starting

 

GGSCI (gg1) 5> start dpump

Sending START request to MANAGER ...

EXTRACT DPUMP starting

 

GGSCI (gg1) 11> info all

 

Program    Status      Group       Lag           Time Since Chkpt

 

MANAGER    RUNNING                                          

EXTRACT    RUNNING     DPUMP       00:00:00      00:36:04   

EXTRACT    RUNNING     EXT1        00:00:00      00:00:09   

 

这里要注意，如果启用了Data Pump，那么必须等Target 端的GG 进程启动以后才能启动，不然会报：

2011-11-1621:57:09  WARNING OGG-01223  TCP/IP error 111 (Connection refused).

 

在Source DB 上创建一张测试表：

 SQL> conn dave/dave;

Connected.

SQL> create table anqing as select *from sys.all_users;

Table created.

 SQL>

 

到Target DB上验证：

SQL> select count(1) from anqing;

  COUNT(1)

----------

       33

 

DDL 同步验证成功。

 

Source 在DML操作：

SQL> insert into anqing select * fromsys.all_users;

33 rows created.

SQL> commit;

Commit complete.

 

Target 上验证DML：

SQL> select count(*) from anqing;

 COUNT(*)

----------

       66

 

同步成功，至此DDL的测试结束。

 

 

 

 

-------------------------------------------------------------------------------------------------------

版权所有，文章允许转载，但必须以链接方式注明源地址，否则追究法律责任!

Blog:     http://blog.csdn.net/tianlesoftware

Weibo: http://weibo.com/tianlesoftware

Email:   [email protected]

Skype: tianlesoftware

 

-------加群需要在备注说明Oracle表空间和数据文件的关系，否则拒绝申请----

DBA1 群：62697716(满);   DBA2 群：62697977(满)  DBA3 群：62697850(满)  

DBA 超级群：63306533(满);  DBA4 群：83829929(满) DBA5群： 142216823(满) 

DBA6 群：158654907(满)   DBA7 群：69087192(满)  DBA8 群：172855474

DBA 超级群2：151508914  DBA9群：102954821     聊天 群：40132017(满)