ogg源端mysql添加trandata_oracle ogg golden gate 安装与配置

环境

oracle12c 12.1.0.2.0

ogg 19.1

linux Ubuntu 14.04.3 LTS

172.18.0.2 源数据库地址

172.18.0.3 目标数据库地址html

ogg根据scn同步数据，源库零停机时间

以下是主要步骤：linux

配置好ogg源端的mgr、抓取和传送进程，并启动。

配置好ogg目标端的mgr、复制进程，仅启动mgr。

源端可自由进行交易，此时观察源与目标的trail文件是否都正常。

查询源端此时的scn，并按照参数flashbask_scn进行expdp。

目标端impdp导入。

目标端start replicat xxx,aftercsn xxxxx

配置和使用GoldenGate的步骤

1 在源端和目标端配置数据库支持GoldenGate 2 在源端和目标端建立和配置GoldenGate实例 3 在源端建立和配置主抽取进程(Primary Extract) 4 在源端建立和配置Data Pump进程(Secondly Extract) 5 在目标端建立和配置Replicat进程sql

安装步骤

源数据库配置步骤

开启归档，--如未开启，从新开启须要重启实例，在mount状态下数据库

Alter database archivelog;

修改oracle日志模式：

sqlplus /nolog

conn / as sysdba

1，关闭数据库：shutdown immediate;

2，启动数据库实例到mount状态：startup mount;

3，切换数据库日志模式：

alter database archivelog;(设置数据库为归档日志模式)

alter database noarchivelog;(设置数据库为非归档日志模式)

4，打开数据库：alter database open;

5，确认数据库处于归档模式：archive log list;

开启Force loggingvim

Alter database force logging;

开启supplemental loggingbash

Alter database add supplemental log data;

alter system switch logfile;

alter database force logging;

设置数据库GoldenGate参数网络

show parameter enable_goldengate_replication;

alter system set enable_goldengate_replication=true scope=both ; --RAC的全部实例也须要设置

alter system set enable_goldengate_replication=true scope=both sid='*';

建立OGG表空间oracle

create tablespace ogg_tbs datafile '/u01/app/oracle/oradata/xe/ogg_tbs.dbf' size 5G AUTOEXTEND on extent management local segment space management auto ;

--如要启用DDL功能，OGG用户须要独立的表空间。

建立OGG帐号app

create user ggadm identified by "oracle" default tablespace ogg_tbs;

ALTER USER GGADM QUOTA UNLIMITED ON OGG_TBS;

给OGG帐号赋权

--源端：无DDL，不使用DBA权限的状况： ---For 集成模式，db 12.0.1.0

--The following privileges apply to Oracle versions 11.2.0.4 or later.

--源端：Extract Classic Mode ：ide

grant connect, resource,CREATE SESSION to ggadm;

exec dbms_goldengate_auth.grant_admin_privilege('ggadm');

exec dbms_goldengate_auth.grant_admin_privilege(grantee=>'ggadm');

grant select any dictionary to ggadm;

commit;

配置tnsnames.ora

vi $ORACLE_HOME/network/admin/tnsnames.ora

OGGSource =

(DESCRIPTION =

(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.56.101)(PORT = 1521))

(CONNECT_DATA =

(SERVER = DEDICATED)

(SERVICE_NAME = mydb)

)

)

检测tnsping 和相关网络是否联通

OGG上传目录：telnet 192.168.56.101 7809

tnsping oggsource

$ sqlplus ggadm/oracle@oggsource

目录端数据库配置

设置ogg参数

alter system set enable_goldengate_replication=true scope=both sid='*';

create tablespace ogg_tbs datafile '/u01/app/oracle/oradata/xe/ogg_tbs.dbf' size 5G AUTOEXTEND on extent management local segment space management auto ;

create user ggadm identified by "oracle" default tablespace ogg_tbs;

ALTER USER ggadm QUOTA UNLIMITED ON ogg_tbs;

grant

CREATE SESSION,

ALTER SESSION,

ALTER SYSTEM,

RESOURCE,

SELECT ANY DICTIONARY,

FLASHBACK ANY TABLE,

SELECT ANY TABLE,

SELECT ANY TRANSACTION,

insert any table,

update any table,

delete any table,

drop any table,

CREATE TABLE

to ggadm;

grant SELECT on dba_clusters to ggadm;

grant SELECT on V_$DATABASE to ggadm;

grant select on sys.logmnr_buildlog to ggadm;

grant EXECUTE on DBMS_FLASHBACK to ggadm;

grant execute on DBMS_CAPTURE_ADM to ggadm;

grant execute on DBMS_STREAMS to ggadm;

grant EXECUTE_CATALOG_ROLE to ggadm;

grant connect, resource,CREATE SESSION to ggadm;

exec dbms_goldengate_auth.grant_admin_privilege('ggadm');

exec dbms_goldengate_auth.grant_admin_privilege(grantee=>'ggadm');

grant select any dictionary to ggadm;

配置tnsnames.ora

oggtarget =

(DESCRIPTION =

(ADDRESS_LIST =

(ADDRESS = (PROTOCOL = TCP)(HOST = oracle12c-target)(PORT = 1521))

)

(CONNECT_DATA =

(SERVICE_NAME = xe)

)

)

数据库附加日志

附加日志级别分为:

数据库级别的附加日志(必须附加日志)

用户级别的附加日志(当使用GoldenGate DDL抽取功能时要启用该级别)

表级别的附加日志(必须附加日志，除非你启用了用户级别的附加日志)

数据库级别的附加日志

由于GoldenGate须要抽取捕获变动数据信息和元数据信息，这些信息须要记录日志，而且由于日志老是持续增加的，这些日志增加的信息对于GoldenGate是必须记录的，因此必须在GoldenGate进程启动前开启数据库级别的附加日志； 检查数据库是否开启附加日志:

SELECT supplemental_log_data_min, force_logging FROM v$database;

SUPPLEME FOR

-------- ---

NO NO

开启数据库级别的附加日志:

SYS@sydb>alter database add supplemental log data;

SYS@sydb>ALTER DATABASE FORCE LOGGING;

SYS@sydb>SELECT supplemental_log_data_min, force_logging FROM v$database;

SUPPLEME FOR

-------- ---

YES YES

Elapsed: 00:00:00.00

SYS@sydb>alter system switch logfile;

System altered.

Elapsed: 00:00:00.09

用户级别的附加日志

开启用户级别的附加日志的管理用户必须具备执行DBMS_CAPTURE_ADM包的权限；能够在GoldenGate GGSCI命令行下登陆到数据库管理用户为其它的用户开启附加日志；

在GGSCI命令行下使用DBLOGIN命令登陆到数据库管理用户

GGSCI (sywu) 1> dblogin userid ogg_owner,password ogg_owner

Successfully logged into database.

为用户启用附加日志

GGSCI (sywu as ogg_owner@sydb) 2> add schematrandata scott

2015-08-24 14:56:26 INFO OGG-01788 SCHEMATRANDATA has been added on schema scott.

2015-08-24 14:56:27 INFO OGG-01976 SCHEMATRANDATA for scheduling columns has been added on schema scott.

取消用户附加日志

GGSCI (sywu as ogg_owner@sydb) 6> delete schematrandata scott

2015-08-25 20:14:49 INFO OGG-01792 SCHEMATRANDATA has been deleted on schema scott.

2015-08-25 20:14:49 INFO OGG-01979 SCHEMATRANDATA for scheduling columns has been deleted on schema scott.

表级别的附加日志

在没有启用用户级别的附加日志的状况下表级别的主键或惟一索引附加日志对于GoldenGate是必须的；在某些状况下即便你启用了用户级别的附加日志，你也能够启用表级别的附加日志使主键附加日志替代每个在用户级别为GoldenGate指定的键；必须在表没有主键或惟一索引的状况下启用表级别的附加日志。

在GGSCI命令行下使用dblogin登陆到数据库管理用户

GGSCI (sywu) 1> dblogin userid ogg_owner,password ogg_owner

Successfully logged into database.

为表启用附加日志

GGSCI (sywu as ogg_owner@sydb) 3> add trandata ogg_owner.togg

Logging of supplemental redo data enabled for table OGG_OWNER.TOGG.

TRANDATA for scheduling columns has been added on table 'OGG_OWNER.TOGG'.

注意:仅当该表有主键或惟一索引时使用上面的命令启用表级别的附加日志，若是表中没有主键或惟一索引时则必须指定一个或多个或者所有列作为主键，在GoldenGate这些键的做用是过滤重复的数据。

SYS@sydb>create table ogg_owner.togg_nokey(id,name,type,CREATED,update_date)

as

select object_id,object_name,object_type,CREATED,sysdate from dba_objects where rownum<1001

Table created.

Elapsed: 00:00:00.11

在没有主键而且没有指定主键时启用表级别附加日志，默认全部列作为主键:

GGSCI (sywu as ogg_owner@sydb) 4> add trandata ogg_owner.togg_nokey

2015-08-24 16:05:01 WARNING OGG-06439 No unique key is defined for table TOGG_NOKEY. All viable columns will be used to represent the key, but may not guarantee uniqueness. KEYCOLS may be used to define the key.

Logging of supplemental redo data enabled for table OGG_OWNER.TOGG_NOKEY.

TRANDATA for scheduling columns has been added on table 'OGG_OWNER.TOGG_NOKEY'.

在没有主键而且指定主键列的状况下启用表级别附加日志:

GGSCI (sywu as ogg_owner@sydb) 26> add trandata ogg_owner.togg_nokey,cols (ID),nokey

2015-08-24 16:33:19 WARNING OGG-06439 No unique key is defined for table TOGG_NOKEY. All viable columns will be used to represent the key, but may not guarantee uniqueness. KEYCOLS may be used to define the key.

Logging of supplemental redo data enabled for table OGG_OWNER.TOGG_NOKEY.

TRANDATA for scheduling columns has been added on table 'OGG_OWNER.TOGG_NOKEY'.

安装ogg,源端与目录端安装方法相同

将ogg压缩文件放到源数据库与目录数据库的/ogg目录，这个目录根据实际状况设定

由于操做系统没有图形化，因此采用静默安装　　 先复制一份配置文件

cd /ogg/fbo_ggs_Linux_x64_shiphome/Disk1/response/

cp oggcore.rsp oggcore-setup.rsp

vim oggcore-12c.rsp

修改这几项

INSTALL_OPTION=ORA12c

SOFTWARE_LOCATION=/u01/app/ogg

START_MANAGER=false

开始安装

mkdir /u01/app/ogg

chown -R oracle:dba /u01/app/ogg

chown -R oracle:dba /ogg

su - oracle

cd /ogg/fbo_ggs_Linux_x64_shiphome/Disk1

./runInstaller -silent -responseFile /ogg/fbo_ggs_Linux_x64_shiphome/Disk1/response/oggcore-12c.rsp

安装成功

配置.bash_profile

PATH=$PATH:$HOME/.local/bin:$HOME/bin

export PATH

export ORACLE_BASE=/data/oracle

export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1

export ORACLE_SID=orcl

export ORACLE_TERM=xterm

export PATH=$ORACLE_HOME/bin:/usr/sbin:$PATH

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib

export LANG=C

export NLS_LANG=AMERICAN_AMERICA.AL32UTF8

export OGG_HOME=/data/oracle/product/ogg_src

export PATH=$OGG_HOME:$PATH

export LD_LIBRARY_PATH=$OGG_HOME:$LD_LIBRARY_PATH

更新.bash_profile

source .bash_profile

ggsci

ogg安装成功

检查依赖包

ldd ggsci #检查依赖包

建立参数文件、数据文件、日志文件等的子目录

ggsci

create subdirs

create subdirs

Creating subdirectories under current directory /home/oracle

Parameter file /u01/app/ogg/dirprm: created.

Report file /u01/app/ogg/dirrpt: created.

Checkpoint file /u01/app/ogg/dirchk: created.

Process status files /u01/app/ogg/dirpcs: created.

SQL script files /u01/app/ogg/dirsql: created.

Database definitions files /u01/app/ogg/dirdef: created.

Extract data files /u01/app/ogg/dirdat: created.

Temporary files /u01/app/ogg/dirtmp: created.

Credential store files /u01/app/ogg/dircrd: created.

Masterkey wallet files /u01/app/ogg/dirwlt: created.

Dump files /u01/app/ogg/dirdmp: created.

安装注意事项

安装过程当中注意如下几点：

1) 安装过程当中若是可使用图形界面建议使用图形界面安装；

2) 安装过程当中注意配置好环境变量，使用oracle用户安装；

3) 若是用户没法提供oracle用户安装，建议安装用户必需要有oracle的全部组权限；

4) 静默安装时注意仅须要编辑两个参数，MGR能够在安装完毕后配置，若是须要配置DATABASE_LOCATION，该参数应为$ORACLE_HOME；

5) 建议安装过程当中若有报错须要及时处理，若是静默模式安装失败，建议彻底卸载或者更换安装目录。

OGG 配置过程(源端)

建立credential store,建立OGG认证证书文件用于隐藏口令登陆

GGSCI > add credentialstore --把用户凭证加入Credentialstore里

把用户凭证加入Credentialstore

alter credentialstore add user ggadm@oggsource password oracle alias oggsourceadm

测试是否能够免密登陆

dblogin useridalias oggsourceadm

Successfully logged into database.

配置抽取表

登陆成功后，将要抽取的表加入trandata

--add trandata schema.tablename

--可经过schema.*表明添加该用户下全部表

add trandata scott.T1,allcols

查看配置状况

info trandata schema.tablename

建立checkpoint表

add checkpointtable ggadm.checkpoint

管理进程配置(源端mgr)

配置全局参数

edit params ./GLOBALS--注意：对于UNIX/Linux 平台，此步骤作完后，须要到目录下检查,文件名必定要大写，不能是globals

--For Performance Monitor:

--ENABLEMONITORING

CHECKPOINTTABLE ggadm.checkpoint

-- For DDL :

GGSCHEMA ggadm

配置MGR进程

edit param mgr

PORT 7809

DYNAMICPORTLIST 7810-7820

--AUTOSTART ER *

--AUTORESTART ER *,RETRIES 5,WAITMINUTES 7, RESETMINUTES 60

PURGEOLDEXTRACTS ./dirdat/*,usecheckpoints, minkeepdays 7

--------------------

--For ddl MGR

--userid ggadm, password goldengate123

--useridalias oggsourceadm

--PURGEDDLHISTORY MINKEEPDAYS 7, MAXKEEPDAYS 10

--PURGEMARKERHISTORY MINKEEPDAYS 7, MAXKEEPDAYS 10

--------------------

LAGREPORTHOURS 1

LAGINFOMINUTES 30

LAGCRITICALMINUTES 45

管理MGR进程

START MGR

info MGR

stop MGR

Extract脚本配置(源端)

管理Extract进程和Extract目录

添加

ADD EXTRACT e1, TRANLOG,BEGIN now

ADD EXTRACT e1, TRANLOG,BEGIN now,threads 2 —RAC为节点数

ADD EXTTRAIL ./dirdat/e1, EXTRACT e1

查看

info EXTTRAIL

info EXTRACT e1

删除

delete EXTTRAIL ./dirdat/e1, EXTRACT e1

delete EXTRACT e1

编辑Extract参数

edit params e1

EXTRACT e1

setenv (NLS_LANG=SIMPLIFIED CHINESE_CHINA.AL32UTF8)

USERIDALIAS oggsourceadm

--USERIDALIAS pmsprod

TRANLOGOPTIONS DBLOGREADER --rac时使用

FETCHOPTIONS NOUSESNAPSHOT

GETTRUNCATES

EXTTRAIL ./dirdat/e1

DISCARDFILE ./dirrpt/e1.dsc, APPEND, MEGABYTES 4000

WARNLONGTRANS 1H, CHECKINTERVAL 5M

CACHEMGR CACHESIZE 1024MB, CACHEDIRECTORY ./dirtmp

LOGALLSUPCOLS

NOCOMPRESSUPDATES

UPDATERECORDFORMAT FULL

REPORTCOUNT EVERY 2 SECONDS, RATE

-- table scott.*;能够*代替全部表

table scott.T1;

管理Extract进程

GGSCI > start extract e1 –启动e1进程

GGSCI > stop extract e1 –关闭e1进程

GGSCI > kill extract e1 --强制中止e1进程

GGSCI > info e1 --查看进程参数信息

GGSCI > stats e1 --查看进程运行信息

GGSCI > view report e1 ---查看运行报告！

问题处理

ERROR OGG-00717 Oracle GoldenGate Capture for Oracle, EXTRACT3.prm: Found unsupported in-memory undo record in sequence 11566, at RBA 1473040, with SCN 3330.2812655651

Possible Cause:

Supplemental logging is not enabled on the database level or the extract is positioned to a point where the supplemental logging has not been enabled i.e an attempt was made to start the extract when the supplemental logging was not enabled on the database.

Solution:

Check if Supplemental logging is enabled

SQL> SELECT SUPPLEMENTAL_LOG_DATA_MIN FROM V$DATABASE;

SUPPLEME

--------

NO

Enable Supplemental Logging

SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;

Database altered.

Switch Logfile

SQL> ALTER SYSTEM SWITCH LOGFILE;

System altered.

And attempt to start the extract with begin now option

GGSCI (sev274) 2> alter extract EXTRACT3, begin now

EXTRACT altered.

GGSCI (sev274) 3> start EXTRACT3

Sending START request to MANAGER ...

EXTRACT EXTRACT3 starting

GGSCI (sev274) 4> info EXTRACT3

EXTRACT EXTRACT3 Last Started 2015-11-09 16:29 Status RUNNING

Checkpoint Lag 00:00:00 (updated 00:00:05 ago)

Log Read Checkpoint Oracle Redo Logs

2015-11-09 16:29:20 Seqno 0, RBA 0

SCN 0.0 (0)

PUMP脚本配置(源端)

管理pump进程和pump目录 添加

ADD EXTRACT p1 EXTTRAILSOURCE ./dirdat/e1, BEGIN NOW

ADD RMTTRAIL /u01/app/ogg/dirdat/tx, EXTRACT p1

查看

GGSCI>info RMTTRAIL

GGSCI>info EXTRACT p1

删除

GGSCI>delete RMTTRAIL /u01/app/ogg/dirdat/tx, EXTRACT p1

GGSCI> delete EXTRACT p1

编辑PUMP参数 edit params p1;

EXTRACT p1

--setenv (NLS_LANG="SIMPLIFIED CHINESE_CHINA.AL32UTF8")

--For ogg 12.2 DYNAMICRESOLUTION is deprecated

--DYNAMICRESOLUTION

--USERIDALIAS ggadm

--USERIDALIAS oggsourceadm

discardfile ./dirrpt/p1.dsc,append,megabytes 4000

rmthost oracle12c-target mgrport 7809

rmttrail /u01/app/ogg/dirdat/tx

PASSTHRU

TABLE SCOTT.*;

管理PUMP进程(源端)

GGSCI > start extract p1 –启动e1进程

GGSCI > stop extract p1 –关闭e1进程

GGSCI > kill extract p1 --强制中止e1进程

GGSCI > info p1 --查看进程参数信息

GGSCI > stats p1 --查看进程运行信息

GGSCI > view report p1 ---查看运行报告

目标端配置证书(目标) 建立OGG认证证书文件用于隐藏口令登陆

GGSCI > add credentialstore

GGSCI > alter credentialstore add user ggadm@oggtarget,password "oracle" alias ggadmtarget

测试登陆

GGSCI (b28a89583b2e) 3> dblogin useridalias ggadmtarget

Successfully logged into database.

配置globals参数(目标)

--For Performance Monitor:

ENABLEMONITORING

CHECKPOINTTABLE ggadm.checkpoint

-- For DDL :

GGSCHEMA ggadm

添加检查点表(目标)

add checkpointtable ggadm.checkpoint

添加replicat应用进程及配置参数

add replicat rep,exttrail ./dirdat/tx checkpointtable ggadm.checkpoint

edit params rep

replicat rep

setenv (NLS_LANG="SIMPLIFIED CHINESE_CHINA.AL32UTF8")

assumetargetdefs

discardfile ./dirout/RAMER.DSC, purge

USERIDALIAS ggadmtarget

map SCOTT.*, target SCOTT.*;

replicat

alter REPLICAT rep1, begin now

配置mgr进程(目标)

GGSCI>edit param mgr

PORT 7809

DYNAMICPORTLIST 7810-7820

管理mgr进程命令

GGSCI> START MGR

GGSCI> info MGR

GGSCI> stop MGR

启动,查看replicat

start replicat rep,aftercsn 402721**

ddl支持

oracle 12c如下须要执行如下语句,重启数据库才能生效,oracle 12c不须要执行

ALTER SYSTEM SET recyclebin=off SCOPE=SPFILE;

在goldengate目录下，执行'maker_setup.sql and ddl_setup.sql.'脚本

建立GGS_GGSUSER_ROLE角色，并受权给goldengate schema

@role_setup.sql

启用ddl支持

@ddl_enable.sql

@ddl_pin ggadm

相关连接

https://www.cnblogs.com/lanston/p/ogg_configAndUse.html

https://www.cnblogs.com/VoiceOfDreams/p/8576973.html

https://blog.csdn.net/anzhen0429/article/details/78637122

https://blog.csdn.net/orabing/article/details/79932093

https://blog.csdn.net/moscot_wu/article/details/78957572

https://blog.csdn.net/tangkailyy/article/details/48026205?utm_source=blogxgwz5

https://blog.csdn.net/orabing/article/details/79932093

http://www.cnblogs.com/junnor/p/4307228.html

https://www.cnblogs.com/kkterry/p/3816793.html

http://www.cnxdug.org/?p=1961

http://blog.itpub.net/7204674/viewspace-713290/

https://docs.oracle.com/goldengate/1212/gg-winux/GIORA.pdf

https://docs.oracle.com/goldengate/1212/gg-winux/GIORA/install.htm#GIORA976

https://blog.yannickjaquier.com/oracle/goldengate-12c-tutorial.html

https://www.cnblogs.com/chinas/p/7339465.html