AlfredZhao
AlfredZhao

OGG实验:喂奶间隔数据表通过OGG配置同步

我之前在《使用SQL计算宝宝每次吃奶的时间间隔(数据保障篇)》中提到数据实时同步的方案，其中有一种是数据表通过OGG进行同步，当时没有详细展开测试，只给了之前学习OGG时的配置示例。由于之前很少接触OGG的管理运维，最近在进行实际测试时遇到些问题，虽然不难，但也是初学者常遇到的问题，所以记录下这个过程。

需求：将TEST用户下的数据表T_BABY通过OGG进行同步。
环境：
源端：RHEL6.5, IP地址：192.168.1.61
Oracle 11.2.0.4 RAC（2 nodes）+ OGG 12.2.0.2
待同步数据表test.t_baby已有数据

目标端：OEL5.7, IP地址：192.168.1.11
Oracle 11.2.0.3 单实例 + OGG 12.2.0.2

数据表通过OGG配置同步：

  • 1.源端和目标端软件安装部署
  • 2.源端配置
  • 3.目标端配置
  • 4.实验中遇到的问题处理

1.源端和目标端软件安装部署

在源端和目标端都部署OGG 12.2.0.2，使用图形界面安装，默认选择安装完成后自动启动mgr进程，安装截图类似如下：
OGG实验:喂奶间隔数据表通过OGG配置同步_第1张图片
OGG实验:喂奶间隔数据表通过OGG配置同步_第2张图片
OGG实验:喂奶间隔数据表通过OGG配置同步_第3张图片
OGG实验:喂奶间隔数据表通过OGG配置同步_第4张图片
OGG实验:喂奶间隔数据表通过OGG配置同步_第5张图片

2.源端配置

首先确认数据库是否处于归档模式，是否开启force logging和数据库最小附加日志：

select log_mode,supplemental_log_data_min,force_logging from v$database;

特殊参数enable_goldengate_replication设置：

alter system set enable_goldengate_replication=true scope=both;

2.1 数据库开启最小附加日志、归档模式、force logging

--1) RAC开启归档模式
srvctl stop database -d demo
sqlplus / as sysdba
startup mount
alter database archivelog;
alter database open;
srvctl start database -d demo

--2）开启force logging
alter database force logging;

--3）开启数据库最小附加日志
alter database add supplemental log data;

开启后再次检查确认数据库已经处于归档模式、开启了force logging和数据库最小附加日志：

sys@DEMO> select log_mode,supplemental_log_data_min,force_logging from v$database;

LOG_MODE     SUPPLEME FOR
------------ -------- ---
ARCHIVELOG   YES      YES

2.2 需要同步的表开启详细附加日志
在GGSCI命令行下，使用"add trandata user.table_name"开启表的详细附加日志。

GGSCI (jystdrac1) 1> add trandata test.t_baby;
ERROR: Not logged into database, use DBLOGIN.

GGSCI (jystdrac1) 2> dblogin userid ggs_admin
Password: 
Successfully logged into database.

GGSCI (jystdrac1 as ggs_admin@demo1) 3> add trandata test.t_baby;
ERROR: No viable tables matched specification.

GGSCI (jystdrac1 as ggs_admin@demo1) 4> add trandata test.t_baby   

Logging of supplemental redo data enabled for table TEST.T_BABY.
TRANDATA for scheduling columns has been added on table 'TEST.T_BABY'.
TRANDATA for instantiation CSN has been added on table 'TEST.T_BABY'.

注意上述命令3是因为结尾多写了";"，而OGG并不需要像SQL那样以";"结尾，所以多写了";"反而会报错。

2.3 源端配置extract和datapump进程
源端配置extract和datapump进程:

GGSCI (jystdrac1) 1> edit param extbaby
--Local Extract extbaby
--Author: Alfred Zhao
--
EXTRACT extbaby
SETENV(NLS_LANG=american_america.ZHS16GBK)
SETENV(ORACLE_SID=demo1)
USERID ggs_admin, PASSWORD ggs_admin
TRANLOGOPTIONS ASMUSER sys@ASM,ASMPASSWORD oracle
EXTTRAIL ./dirdat/st
TABLE TEST.T_BABY;


GGSCI (jystdrac1) 2> edit param dpbaby
--Local datapump dpbaby
--Author: Alfred Zhao
--
EXTRACT dpbaby
PASSTHRU
RMTHOST 192.168.1.11, MGRPORT 7809
RMTTRAIL ./dirdat/tt
TABLE TEST.T_BABY;


--添加extbaby(注意这里RAC2个节点，所以指定threads 2)
GGSCI (jystdrac1) 3> add extract extbaby, tranlog, begin now, threads 2
GGSCI (jystdrac1) 4> add exttrail ./dirdat/st, extract extbaby, megabytes 50

--添加dpbaby
GGSCI (jystdrac1) 5> add extract dpbaby, exttrailsource ./dirdat/st, begin now
GGSCI (jystdrac1) 6> add rmttrail ./dirdat/tt, EXTRACT dpbaby, MEGABYTES 50

--启动extract和datapump进程
GGSCI (jystdrac1) 7> start *

确认抽取进程正常后，初始化数据做准备，备份当前的t_baby表：

[oracle@jystdrac1 ggs]$ /bin/sh /home/oracle/baby/backup_exp_t_baby.sh

Export: Release 11.2.0.4.0 - Production on Tue Feb 11 17:36:20 2020

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.


Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Tes
Export done in ZHS16GBK character set and AL16UTF16 NCHAR character set

About to export specified tables via Conventional Path ...
. . exporting table                         T_BABY        462 rows exported
Export terminated successfully without warnings.

导出文件：/public/backup/t_baby_20200211.dmp

将源端这个导出文件传输到目标端，给后续导入使用，我这里的/public目录是NAS，源端目标端都可以访问，所以无需拷贝。

3.目标端配置

3.1 初始化数据，导入目标端数据库
导入之前确认已创建用户：test
imp test/test file=/public/backup/t_baby_20200211.dmp full=y

[oracle@OEL-ASM dirdat]$ imp test/test file=/public/backup/t_baby_20200211.dmp full=y

Import: Release 11.2.0.3.0 - Production on Tue Feb 11 17:38:09 2020

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.


Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Automatic Storage Management, OLAP, Data Mining
and Real Application Testing options

Export file created by EXPORT:V11.02.00 via conventional path
import done in ZHS16GBK character set and AL16UTF16 NCHAR character set
import server uses WE8MSWIN1252 character set (possible charset conversion)
. importing TEST's objects into TEST
. importing TEST's objects into TEST
. . importing table                       "T_BABY"        462 rows imported
Import terminated successfully without warnings.

3.2 目标端配置checkpointtable

view params ./GLOBALS
checkpointtable ggt_admin.chkpt
dblogin userid ggt_admin, password ggt_admin
add checkpointtable ggt_admin.chkpt

3.3 目标端配置replicat进程

edit param repbaby
--Local datapump repbaby
--Author: Alfred Zhao
--
REPLICAT repbaby
SETENV(NLS_LANG=american_america.ZHS16GBK)
SETENV (ORACLE_SID=test)
USERID ggt_admin, PASSWORD ggt_admin
DISCARDFILE ./dirrpt/repbaby.dsc, PURGE
HandleCollisions
AssumeTargetDefs
Map test.*,Target test.*;

--添加repbaby进程
add replicat repbaby, exttrail ./dirdat/tt

--确认初始化数据完成后，开启repbaby进程
GGSCI (OEL-ASM) 4> start *

Sending START request to MANAGER ...
REPLICAT REPBABY starting


GGSCI (OEL-ASM) 5> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                           
REPLICAT    RUNNING     REPBABY     00:00:00      00:01:48    

GGSCI (OEL-ASM) 6> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                           
REPLICAT    RUNNING     REPBABY     00:00:00      00:00:00

4.实验中遇到的问题处理

4.1 extract进程extbaby启动不成功，报错OGG-02091

2020-02-11 16:34:28  ERROR   OGG-02091  Oracle GoldenGate Capture for Oracle, extbaby.prm:  Operation not supported because enable_goldengate_replication is not set to true.
2020-02-11 16:34:28  ERROR   OGG-01668  Oracle GoldenGate Capture for Oracle, extbaby.prm:  PROCESS ABENDING.

这个报错很明显，按要求修改这个参数即可：

alter system set enable_goldengate_replication=true scope=both;

4.2 extract进程extbaby启动不成功，报错OGG-00446
extbaby启动不成功，日志显示：


2020-02-11 16:36:55  ERROR   OGG-00446  Oracle GoldenGate Capture for Oracle, extbaby.prm:  No valid log files for current redo sequence 859, thread 1, error retrieving redo file name for sequence 859, archived = 0, use_alternate = 0Not able to establish initial position for begin time 2020-02-11 16:33:22.000000.
2020-02-11 16:36:55  ERROR   OGG-01668  Oracle GoldenGate Capture for Oracle, extbaby.prm:  PROCESS ABENDING.

这是因为RAC的redo日志在ASM中无法访问到，extbaby配置文件中加入ASM实例的访问配置，并配置好对应的tnsnames.ora即可：

TRANLOGOPTIONS ASMUSER sys@ASM,ASMPASSWORD oracle

tnsnames.ora添加：

ASM =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.61)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = +ASM)
      (SID = +ASM1)
    )
  )

4.3 extract进程extbaby启动不成功，依然报错OGG-00446，但内容有区别
具体表现为启动后，还是运行一会儿后abended，具体报错如下：

2020-02-11 16:58:34  ERROR   OGG-00446  Oracle GoldenGate Capture for Oracle, extbaby.prm:  Could not find archived log for sequence 2157 thread 2 under default destinations SQL , error retrieving redo file name for sequence 2157, archived = 1, use_alternate = 0Not able to establish initial position for begin time 2020-02-11 16:33:22.000000.
2020-02-11 17:15:25  ERROR   OGG-01668  Oracle GoldenGate Capture for Oracle, extbaby.prm:  PROCESS ABENDING.

--切换日志后，再启动ogg进程：
sys@DEMO> alter system archive log current;

System altered.

sys@DEMO> select * from v$log;

    GROUP#    THREAD#  SEQUENCE#      BYTES  BLOCKSIZE    MEMBERS ARC STATUS           FIRST_CHANGE# FIRST_TIME          NEXT_CHANGE# NEXT_TIME
---------- ---------- ---------- ---------- ---------- ---------- --- ---------------- ------------- ------------------- ------------ -------------------
         1          1        861   52428800        512          1 NO  CURRENT               32759599 2020-02-11 17:17:18   2.8147E+14
         2          1        860   52428800        512          1 YES INACTIVE              32755401 2020-02-11 17:00:53     32759599 2020-02-11 17:17:18
         3          2       2159   52428800        512          2 NO  CURRENT               32759603 2020-02-11 17:17:18   2.8147E+14
         4          2       2158   52428800        512          2 YES INACTIVE              32756440 2020-02-11 17:12:36     32759603 2020-02-11 17:17:18


GGSCI (jystdrac1) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                           
EXTRACT     RUNNING     DPBABY      00:00:00      00:00:02    
EXTRACT     STOPPED     EXTBABY     00:00:00      00:44:42    


GGSCI (jystdrac1) 2> start ext*

Sending START request to MANAGER ...
EXTRACT EXTBABY starting


GGSCI (jystdrac1) 3> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                           
EXTRACT     RUNNING     DPBABY      00:00:00      00:00:00    
EXTRACT     RUNNING     EXTBABY     00:00:00      00:44:50    

--依然报错：sequence 2157 thread 2
2020-02-11 17:19:31  ERROR   OGG-00446  Oracle GoldenGate Capture for Oracle, extbaby.prm:  Could not find archived log for sequence 2157 thread 2 under default destinations SQL