oracle ogg golden gate 安装与配置

2019独角兽企业重金招聘Python工程师标准>>>

环境

oracle12c 12.1.0.2.0
ogg 19.1
linux Ubuntu 14.04.3 LTS
172.18.0.2 源数据库地址
172.18.0.3 目标数据库地址

ogg根据scn同步数据，源库零停机时间

如下是主要步骤：

配置好ogg源端的mgr、抓取和传送进程，并启动。
配置好ogg目标端的mgr、复制进程，仅启动mgr。
源端可自由进行交易，此时观察源与目标的trail文件是否都正常。
查询源端此时的scn，并按照参数flashbask_scn进行expdp。
目标端impdp导入。
目标端start replicat xxx,aftercsn xxxxx

配置和使用GoldenGate的步骤

1 在源端和目标端配置数据库支持GoldenGate 2 在源端和目标端创建和配置GoldenGate实例 3 在源端创建和配置主抽取进程(Primary Extract) 4 在源端创建和配置Data Pump进程(Secondly Extract) 5 在目标端创建和配置Replicat进程

安装步骤

源数据库配置步骤

开启归档，--如未开启，重新开启需要重启实例，在mount状态下

Alter database archivelog;
修改oracle日志模式：
sqlplus /nolog
conn / as sysdba
1，关闭数据库：shutdown immediate;
2，启动数据库实例到mount状态：startup mount;
3，切换数据库日志模式：
      alter database archivelog;（设置数据库为归档日志模式）
      alter database noarchivelog;（设置数据库为非归档日志模式）
4，打开数据库：alter database open;
5，确认数据库处于归档模式：archive log list;

开启Force logging

Alter database force logging;

开启supplemental logging

Alter database add supplemental log data;
alter system switch logfile;
alter database force logging;

设置数据库GoldenGate参数

show parameter enable_goldengate_replication;
alter system set enable_goldengate_replication=true scope=both ; --RAC的所有实例也需要设置
alter system set enable_goldengate_replication=true scope=both sid='*';

创建OGG表空间

create tablespace ogg_tbs datafile '/u01/app/oracle/oradata/xe/ogg_tbs.dbf' size 5G AUTOEXTEND on extent management local segment space management auto ;

--如要启用DDL功能，OGG用户需要独立的表空间。
创建OGG账号

create user ggadm identified by "oracle" default tablespace ogg_tbs;
ALTER USER   GGADM QUOTA UNLIMITED ON  OGG_TBS;

给OGG账号赋权
--源端：无DDL，不使用DBA权限的情况： ---For 集成模式，db 12.0.1.0
--The following privileges apply to Oracle versions 11.2.0.4 or later.
--源端：Extract Classic Mode ：

grant connect, resource,CREATE SESSION to ggadm;
exec dbms_goldengate_auth.grant_admin_privilege('ggadm');
exec dbms_goldengate_auth.grant_admin_privilege(grantee=>'ggadm');
grant select any dictionary to ggadm;
commit;

配置tnsnames.ora

vi $ORACLE_HOME/network/admin/tnsnames.ora 
OGGSource =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.56.101)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = mydb)
)
)

检测tnsping 和相关网络是否联通

OGG上传目录：telnet 192.168.56.101 7809
tnsping oggsource
$ sqlplus ggadm/oracle@oggsource

目录端数据库配置

设置ogg参数

alter system set enable_goldengate_replication=true scope=both sid='*';
create tablespace ogg_tbs datafile '/u01/app/oracle/oradata/xe/ogg_tbs.dbf' size 5G AUTOEXTEND on extent management local segment space management auto ;
create user ggadm identified by "oracle" default tablespace ogg_tbs;
ALTER USER  ggadm QUOTA UNLIMITED ON ogg_tbs;
grant 
CREATE SESSION,
ALTER SESSION,
ALTER SYSTEM,
RESOURCE,
SELECT ANY DICTIONARY,
FLASHBACK ANY TABLE,
SELECT ANY TABLE,
SELECT ANY TRANSACTION,
insert any table,
update any table,
delete any table,
drop any table,
CREATE TABLE
to ggadm;
grant SELECT on dba_clusters to ggadm;
grant SELECT on V_$DATABASE to ggadm;
grant select on sys.logmnr_buildlog to ggadm;
grant EXECUTE on DBMS_FLASHBACK to ggadm;
grant execute on DBMS_CAPTURE_ADM to ggadm;
grant execute on DBMS_STREAMS to ggadm;  
grant EXECUTE_CATALOG_ROLE to ggadm;
grant connect, resource,CREATE SESSION to ggadm;
exec dbms_goldengate_auth.grant_admin_privilege('ggadm');
exec dbms_goldengate_auth.grant_admin_privilege(grantee=>'ggadm');
grant select any dictionary to ggadm;

配置tnsnames.ora

oggtarget =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = oracle12c-target)(PORT = 1521))
    )
    (CONNECT_DATA =
      (SERVICE_NAME = xe)
    )
  )

数据库附加日志

附加日志级别分为:

数据库级别的附加日志(必须附加日志)
用户级别的附加日志(当使用GoldenGate DDL抽取功能时要启用该级别)
表级别的附加日志(必须附加日志，除非你启用了用户级别的附加日志)

数据库级别的附加日志
因为GoldenGate需要抽取捕获变更数据信息和元数据信息，这些信息需要记录日志，并且因为日志总是持续增长的，这些日志增长的信息对于GoldenGate是必须记录的，所以必须在GoldenGate进程启动前开启数据库级别的附加日志；检查数据库是否开启附加日志:

SELECT supplemental_log_data_min, force_logging FROM v$database;
SUPPLEME FOR
-------- ---
NO       NO

开启数据库级别的附加日志:

SYS@sydb>alter database add supplemental log data;
SYS@sydb>ALTER DATABASE FORCE LOGGING;

SYS@sydb>SELECT supplemental_log_data_min, force_logging FROM v$database;

SUPPLEME FOR
-------- ---
YES      YES
Elapsed: 00:00:00.00
SYS@sydb>alter system switch logfile;

System altered.
Elapsed: 00:00:00.09

用户级别的附加日志

开启用户级别的附加日志的管理用户必须具有执行DBMS_CAPTURE_ADM包的权限；可以在GoldenGate GGSCI命令行下登录到数据库管理用户为其它的用户开启附加日志；
在GGSCI命令行下使用DBLOGIN命令登录到数据库管理用户

GGSCI (sywu) 1> dblogin userid ogg_owner,password ogg_owner
Successfully logged into database.

为用户启用附加日志

GGSCI (sywu as ogg_owner@sydb) 2> add schematrandata scott
2015-08-24 14:56:26  INFO    OGG-01788  SCHEMATRANDATA has been added on schema scott.
2015-08-24 14:56:27  INFO    OGG-01976  SCHEMATRANDATA for scheduling columns has been added on schema scott.

取消用户附加日志

GGSCI (sywu as ogg_owner@sydb) 6> delete schematrandata scott
2015-08-25 20:14:49  INFO    OGG-01792  SCHEMATRANDATA has been deleted on schema scott.
2015-08-25 20:14:49  INFO    OGG-01979  SCHEMATRANDATA for scheduling columns has been deleted on schema scott.

表级别的附加日志
在没有启用用户级别的附加日志的情况下表级别的主键或唯一索引附加日志对于GoldenGate是必须的；在某些情况下即使你启用了用户级别的附加日志，你也可以启用表级别的附加日志使主键附加日志替代每一个在用户级别为GoldenGate指定的键；必须在表没有主键或唯一索引的情况下启用表级别的附加日志。
在GGSCI命令行下使用dblogin登录到数据库管理用户

GGSCI (sywu) 1> dblogin userid ogg_owner,password ogg_owner
Successfully logged into database.

为表启用附加日志

GGSCI (sywu as ogg_owner@sydb) 3> add trandata ogg_owner.togg
Logging of supplemental redo data enabled for table OGG_OWNER.TOGG.
TRANDATA for scheduling columns has been added on table 'OGG_OWNER.TOGG'.

注意:仅当该表有主键或唯一索引时使用上面的命令启用表级别的附加日志，如果表中没有主键或唯一索引时则必须指定一个或多个或者全部列做为主键，在GoldenGate这些键的作用是过滤重复的数据。

SYS@sydb>create table ogg_owner.togg_nokey(id,name,type,CREATED,update_date) 
      as 
 select object_id,object_name,object_type,CREATED,sysdate from dba_objects  where rownum<1001
Table created.
Elapsed: 00:00:00.11

在没有主键并且没有指定主键时启用表级别附加日志，默认所有列做为主键:

GGSCI (sywu as ogg_owner@sydb) 4> add trandata ogg_owner.togg_nokey

2015-08-24 16:05:01  WARNING OGG-06439  No unique key is defined for table TOGG_NOKEY. All viable columns will be used to represent the key, but may not guarantee uniqueness. KEYCOLS may be used to define the key.

Logging of supplemental redo data enabled for table OGG_OWNER.TOGG_NOKEY.
TRANDATA for scheduling columns has been added on table 'OGG_OWNER.TOGG_NOKEY'.

在没有主键并且指定主键列的情况下启用表级别附加日志:

GGSCI (sywu as ogg_owner@sydb) 26>  add trandata ogg_owner.togg_nokey,cols (ID),nokey

2015-08-24 16:33:19  WARNING OGG-06439  No unique key is defined for table TOGG_NOKEY. All viable columns will be used to represent the key, but may not guarantee uniqueness. KEYCOLS may be used to define the key.

Logging of supplemental redo data enabled for table OGG_OWNER.TOGG_NOKEY.
TRANDATA for scheduling columns has been added on table 'OGG_OWNER.TOGG_NOKEY'.

安装ogg,源端与目录端安装方法相同

将ogg压缩文件放到源数据库与目录数据库的/ogg目录，这个目录根据实际情况设定

因为操作系统没有图形化，所以采用静默安装　　先复制一份配置文件

cd /ogg/fbo_ggs_Linux_x64_shiphome/Disk1/response/
cp oggcore.rsp oggcore-setup.rsp
vim oggcore-12c.rsp
修改这几项
INSTALL_OPTION=ORA12c
SOFTWARE_LOCATION=/u01/app/ogg
START_MANAGER=false

开始安装　 
mkdir /u01/app/ogg
chown -R oracle:dba /u01/app/ogg
chown -R oracle:dba /ogg
su - oracle
cd /ogg/fbo_ggs_Linux_x64_shiphome/Disk1
./runInstaller -silent -responseFile /ogg/fbo_ggs_Linux_x64_shiphome/Disk1/response/oggcore-12c.rsp

安装成功

配置.bash_profile

PATH=$PATH:$HOME/.local/bin:$HOME/bin
export PATH
export ORACLE_BASE=/data/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1
export ORACLE_SID=orcl
export ORACLE_TERM=xterm
export PATH=$ORACLE_HOME/bin:/usr/sbin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export LANG=C
export NLS_LANG=AMERICAN_AMERICA.AL32UTF8
export OGG_HOME=/data/oracle/product/ogg_src
export PATH=$OGG_HOME:$PATH
export LD_LIBRARY_PATH=$OGG_HOME:$LD_LIBRARY_PATH

更新.bash_profile

source .bash_profile
ggsci

ogg安装成功

检查依赖包

ldd ggsci #检查依赖包

创建参数文件、数据文件、日志文件等的子目录

ggsci
create subdirs
create subdirs

Creating subdirectories under current directory /home/oracle

Parameter file                 /u01/app/ogg/dirprm: created.
Report file                    /u01/app/ogg/dirrpt: created.
Checkpoint file                /u01/app/ogg/dirchk: created.
Process status files           /u01/app/ogg/dirpcs: created.
SQL script files               /u01/app/ogg/dirsql: created.
Database definitions files     /u01/app/ogg/dirdef: created.
Extract data files             /u01/app/ogg/dirdat: created.
Temporary files                /u01/app/ogg/dirtmp: created.
Credential store files         /u01/app/ogg/dircrd: created.
Masterkey wallet files         /u01/app/ogg/dirwlt: created.
Dump files                     /u01/app/ogg/dirdmp: created.

安装注意事项
安装过程中注意以下几点：
1）安装过程中如果可以使用图形界面建议使用图形界面安装；
2）安装过程中注意配置好环境变量，使用oracle用户安装；
3）如果用户无法提供oracle用户安装，建议安装用户必须要有oracle的所有组权限；
4）静默安装时注意仅需要编辑两个参数，MGR可以在安装完毕后配置，如果需要配置DATABASE_LOCATION，该参数应为$ORACLE_HOME；
5）建议安装过程中如有报错需要及时处理，如果静默模式安装失败，建议完全卸载或者更换安装目录。

OGG 配置过程(源端)

创建credential store,创建OGG认证证书文件用于隐藏口令登录

GGSCI > add credentialstore --把用户凭证加入Credentialstore里

把用户凭证加入Credentialstore

alter credentialstore add user ggadm@oggsource password oracle alias oggsourceadm

测试是否可以免密登录

dblogin useridalias oggsourceadm
Successfully logged into database.

配置抽取表
登录成功后，将要抽取的表加入trandata

--add trandata schema.tablename
--可通过schema.*代表添加该用户下所有表
add trandata scott.T1,allcols

查看配置情况

info trandata schema.tablename

创建checkpoint表

add checkpointtable ggadm.checkpoint

管理进程配置(源端mgr)

配置全局参数

edit params ./GLOBALS--注意：对于UNIX/Linux 平台，此步骤做完后，需要到目录下检查,文件名一定要大写，不能是globals
--For Performance Monitor:
--ENABLEMONITORING
CHECKPOINTTABLE ggadm.checkpoint
-- For DDL :
GGSCHEMA ggadm

配置MGR进程

edit param mgr
PORT 7809
DYNAMICPORTLIST 7810-7820
--AUTOSTART ER *
--AUTORESTART ER *,RETRIES 5,WAITMINUTES 7, RESETMINUTES 60
PURGEOLDEXTRACTS ./dirdat/*,usecheckpoints, minkeepdays 7
--------------------
--For ddl MGR
--userid ggadm, password goldengate123
--useridalias oggsourceadm
--PURGEDDLHISTORY MINKEEPDAYS 7, MAXKEEPDAYS 10
--PURGEMARKERHISTORY MINKEEPDAYS 7, MAXKEEPDAYS 10
--------------------
LAGREPORTHOURS 1
LAGINFOMINUTES 30
LAGCRITICALMINUTES 45

管理MGR进程

START MGR
info MGR
stop MGR

Extract脚本配置(源端)

管理Extract进程和Extract目录
添加

ADD EXTRACT e1, TRANLOG,BEGIN now
ADD EXTRACT e1, TRANLOG,BEGIN now,threads 2 —RAC为节点数
ADD EXTTRAIL ./dirdat/e1, EXTRACT e1

查看　　

info EXTTRAIL　　 
info EXTRACT e1

删除

delete EXTTRAIL ./dirdat/e1, EXTRACT e1　　 
delete EXTRACT e1

编辑Extract参数

 edit params e1
EXTRACT e1
setenv (NLS_LANG=SIMPLIFIED CHINESE_CHINA.AL32UTF8)
USERIDALIAS oggsourceadm
--USERIDALIAS pmsprod
TRANLOGOPTIONS DBLOGREADER  --rac时使用
FETCHOPTIONS NOUSESNAPSHOT
GETTRUNCATES
EXTTRAIL ./dirdat/e1
DISCARDFILE ./dirrpt/e1.dsc, APPEND, MEGABYTES 4000
WARNLONGTRANS 1H, CHECKINTERVAL 5M
CACHEMGR CACHESIZE 1024MB, CACHEDIRECTORY ./dirtmp
 
LOGALLSUPCOLS
NOCOMPRESSUPDATES
UPDATERECORDFORMAT FULL
 
REPORTCOUNT EVERY 2 SECONDS, RATE
-- table scott.*;可以*代替所有表
table scott.T1;

管理Extract进程

GGSCI > start extract e1 –启动e1进程
GGSCI > stop extract e1 –关闭e1进程
GGSCI > kill extract e1 --强制停止e1进程
GGSCI > info e1 --查看进程参数信息
GGSCI > stats e1 --查看进程运行信息
GGSCI > view report e1 ---查看运行报告！

问题处理

ERROR   OGG-00717  Oracle GoldenGate Capture for Oracle, EXTRACT3.prm:  Found unsupported in-memory undo record in sequence 11566, at RBA 1473040, with SCN 3330.2812655651

Possible Cause:
Supplemental logging is not enabled on the database level or the extract is positioned to a point where the supplemental logging has not been enabled i.e an attempt was made to start the extract when the supplemental logging was not enabled on the database.

Solution:

Check if Supplemental logging is enabled

SQL> SELECT SUPPLEMENTAL_LOG_DATA_MIN FROM V$DATABASE;

SUPPLEME
--------
NO

Enable Supplemental Logging

SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;

Database altered.

Switch Logfile

SQL> ALTER SYSTEM SWITCH LOGFILE;

System altered.

And attempt to start the extract with begin now option

GGSCI (sev274) 2> alter extract EXTRACT3, begin now
EXTRACT altered.


GGSCI (sev274) 3> start EXTRACT3

Sending START request to MANAGER ...
EXTRACT EXTRACT3 starting


GGSCI (sev274) 4> info EXTRACT3

EXTRACT    EXTRACT3  Last Started 2015-11-09 16:29   Status RUNNING
Checkpoint Lag       00:00:00 (updated 00:00:05 ago)
Log Read Checkpoint  Oracle Redo Logs
                     2015-11-09 16:29:20  Seqno 0, RBA 0
                     SCN 0.0 (0)

PUMP脚本配置(源端)

管理pump进程和pump目录添加

ADD EXTRACT p1 EXTTRAILSOURCE ./dirdat/e1, BEGIN NOW
ADD RMTTRAIL /u01/app/ogg/dirdat/tx, EXTRACT p1

查看

GGSCI>info RMTTRAIL
GGSCI>info EXTRACT p1

删除

GGSCI>delete RMTTRAIL /u01/app/ogg/dirdat/tx, EXTRACT p1
GGSCI> delete EXTRACT p1

编辑PUMP参数 edit params p1;

EXTRACT p1
--setenv (NLS_LANG="SIMPLIFIED CHINESE_CHINA.AL32UTF8")
--For ogg 12.2 DYNAMICRESOLUTION is deprecated
--DYNAMICRESOLUTION
--USERIDALIAS ggadm
--USERIDALIAS oggsourceadm
discardfile ./dirrpt/p1.dsc,append,megabytes 4000
rmthost oracle12c-target  mgrport 7809
rmttrail /u01/app/ogg/dirdat/tx
 
PASSTHRU 
TABLE SCOTT.*;

管理PUMP进程(源端)

GGSCI > start extract p1 –启动e1进程
GGSCI > stop extract p1 –关闭e1进程
GGSCI > kill extract p1 --强制停止e1进程
GGSCI > info p1 --查看进程参数信息
GGSCI > stats p1 --查看进程运行信息
GGSCI > view report p1 ---查看运行报告

目标端配置证书（目标）创建OGG认证证书文件用于隐藏口令登录

GGSCI > add credentialstore
GGSCI > alter credentialstore add user ggadm@oggtarget,password "oracle" alias ggadmtarget

测试登录

GGSCI (b28a89583b2e) 3> dblogin useridalias ggadmtarget
Successfully logged into database.

配置globals参数（目标）

--For Performance Monitor:
ENABLEMONITORING
CHECKPOINTTABLE ggadm.checkpoint
-- For DDL :
GGSCHEMA ggadm

添加检查点表（目标）

add checkpointtable ggadm.checkpoint

添加replicat应用进程及配置参数

add replicat rep,exttrail ./dirdat/tx checkpointtable ggadm.checkpoint
edit params rep
replicat rep
setenv (NLS_LANG="SIMPLIFIED CHINESE_CHINA.AL32UTF8")
assumetargetdefs
discardfile ./dirout/RAMER.DSC, purge
USERIDALIAS ggadmtarget
map SCOTT.*, target SCOTT.*;

replicat

alter REPLICAT rep1, begin now

配置mgr进程(目标)

GGSCI>edit param mgr  
PORT 7809  
DYNAMICPORTLIST 7810-7820 
管理mgr进程命令
GGSCI> START MGR  
GGSCI> info MGR  
GGSCI> stop MGR

启动,查看replicat

start replicat rep,aftercsn 402721**

ddl支持

oracle 12c以下需要执行以下语句,重启数据库才能生效,oracle 12c不需要执行

ALTER SYSTEM SET recyclebin=off SCOPE=SPFILE;

在goldengate目录下，执行'maker_setup.sql and ddl_setup.sql.'脚本
创建GGS_GGSUSER_ROLE角色，并授权给goldengate schema

@role_setup.sql

启用ddl支持

@ddl_enable.sql
@ddl_pin ggadm