数据守护集群の实时主备_故障自动转移

2.1 数据守护集群の实时主备_故障自动转移

1. 前期规划

建议：MAL子系统尽量单独走一个网络（内部），以免引起对外提供服务的公共网络，因为归档日志传输很频繁的。

2. 安装数据库基础软件

(略)

3.初始化数据库（主库）

[root@DMDB1 network-scripts]# su - dmdba
## 3.1 初始化主库：库名 dmdb  实例名：GRP1_RT_01
[dmdba@DMDB1 ~]$ dminit PATH=/DMDB/data DB_NAME=dmdb INSTANCE_NAME=GRP1_RT_01
initdb V8
db version: 0x7000b
file dm.key not found, use default license!
License will expire on 2021-09-04
 log file path: /DMDB/data/dmdb/dmdb01.log
 log file path: /DMDB/data/dmdb/dmdb02.log
write to dir [/DMDB/data/dmdb].
create dm database success. 2020-12-02 10:41:27

## 3.2 检查数据库的数据文件初始情况
[dmdba@DMDB1 ~]$ ls -lh /DMDB/data/dmdb
总用量 512K
drwxr-xr-x 2 dmdba dinstall 4.0K 12月  2 10:41 bak
drwxr-xr-x 2 dmdba dinstall 4.0K 12月  2 10:41 ctl_bak
-rw-r--r-- 1 dmdba dinstall 5.0K 12月  2 10:41 dm.ctl
-rw-r--r-- 1 dmdba dinstall 256M 12月  2 10:41 dmdb01.log
-rw-r--r-- 1 dmdba dinstall 256M 12月  2 10:41 dmdb02.log
-rw-r--r-- 1 dmdba dinstall  49K 12月  2 10:41 dm.ini
-rw-r--r-- 1 dmdba dinstall  807 12月  2 10:41 dminit20201202104124.log
-rw-r--r-- 1 dmdba dinstall  633 12月  2 10:41 dm_service.prikey
drwxr-xr-x 2 dmdba dinstall 4.0K 12月  2 10:41 HMAIN
-rw-r--r-- 1 dmdba dinstall 128M 12月  2 10:41 MAIN.DBF
-rw-r--r-- 1 dmdba dinstall 128M 12月  2 10:41 ROLL.DBF
-rw-r--r-- 1 dmdba dinstall  479 12月  2 10:41 sqllog.ini
-rw-r--r-- 1 dmdba dinstall  10M 12月  2 10:41 SYSTEM.DBF

## 3.3 注册数据库实例的服务
[root@DMDB1 ~]# /DMDB/app/DM8/script/root/dm_service_installer.sh -t dmserver -dm_ini /DMDB/data/dmdb/dm.ini -p GRP1_RT_01 
移动服务脚本文件(/DMDB/app/DM8/bin/DmServiceGRP1_RT_01 到 /etc/rc.d/init.d/DmServiceGRP1_RT_01)
创建服务(DmServiceGRP1_RT_01)完成
[root@DMDB1 ~]# chkconfig --list |grep -i dm
DmAPService     0:关闭  1:关闭  2:启用  3:启用  4:启用  5:启用  6:关闭
DmServiceGRP1_RT_01     0:关闭  1:关闭  2:启用  3:启用  4:启用  5:启用  6:关闭
mdmonitor       0:关闭  1:关闭  2:启用  3:启用  4:启用  5:启用  6:关闭
nfs-rdma        0:关闭  1:关闭  2:关闭  3:关闭  4:关闭  5:关闭  6:关闭
rdma            0:关闭  1:关闭  2:关闭  3:关闭  4:关闭  5:关闭  6:关闭

[root@DMDB1 ~]# service DmServiceGRP1_RT_01 start
Starting DmServiceGRP1_RT_01:                              [ OK ]

## 3.4 检查下数据库是否能正常登录
[root@DMDB1 ~]# su - dmdba
[dmdba@DMDB1 ~]$ disql SYSDBA/SYSDBA
服务器[LOCALHOST:5236]:处于普通打开状态
登录使用时间 : 5.957(ms)
disql V8
SQL> select name,instance_name,host_name,STATUS$ from v$instance;
行号     NAME       INSTANCE_NAME HOST_NAME STATUS$
---------- ---------- ------------- --------- -------
1          GRP1_RT_01 GRP1_RT_01    DMDB1     OPEN
已用时间: 7.716(毫秒). 执行号:500.

SQL> select name,status$,arch_mode from v$database;
行号     NAME STATUS$     ARCH_MODE
---------- ---- ----------- ---------
1          dmdb 4           N
已用时间: 8.636(毫秒). 执行号:502.

4. 脱机备份主库

由于没有开启归档，所以选择dmrman脱机备份方式。

## 4.1 停库
[root@DMDB1 ~]# su - dmdba
[dmdba@DMDB1 ~]$ service DmServiceGRP1_RT_01 stop
Stopping DmServiceGRP1_RT_01:                              [ OK ]

[dmdba@DMDB1 ~]$ ps -ef |grep dmserver
dmdba      3785   3708  0 10:56 pts/0    00:00:00 grep dmserver

## 4.2 dmrman冷备主库
[dmdba@DMDB1 ~]$ dmrman CTLSTMT="BACKUP DATABASE '/DMDB/data/dmdb/dm.ini' FULL TO BACKUP_FILE1 BACKUPSET '/DMDB/backup/BACKUP_FILE_01'"
dmrman V8
BACKUP DATABASE '/DMDB/data/dmdb/dm.ini' FULL TO BACKUP_FILE1 BACKUPSET '/DMDB/backup/BACKUP_FILE_01'
file dm.key not found, use default license!
Database mode = 0, oguid = 0
EP[0]'s cur_lsn[37090]
BACKUP DATABASE [dmdb],execute......
CMD CHECK LSN......
BACKUP DATABASE [dmdb],collect dbf......
CMD CHECK ......
DBF BACKUP SUBS......
total 1 packages processed...
total 2 packages processed...
total 3 packages processed...
DBF BACKUP MAIN......
BACKUPSET [/DMDB/backup/BACKUP_FILE_01] END, CODE [0]......
META GENERATING......
total 7 packages processed...
total 7 packages processed...
total 7 packages processed!
CMD END.CODE:[0]
backup successfully!
time used: 00:00:01.502

## 4.3 确认备份成功
[dmdba@DMDB1 ~]$ cd /DMDB/backup/
[dmdba@DMDB1 backup]$ ls -l
总用量 4

drwxr-xr-x 2 dmdba dinstall 4096 12月  2 11:00 BACKUP_FILE_01
[dmdba@DMDB1 backup]$ du -sh BACKUP_FILE_01
6.2M    BACKUP_FILE_01

## 4.4 远程拷贝备份到DMDB2机器
[dmdba@DMDB1 backup]$ scp -rp BACKUP_FILE_01/ 192.168.0.12:/DMDB/backup
The authenticity of host '192.168.0.12 (192.168.0.12)' can't be established.
RSA key fingerprint is db:a7:63:d0:23:cd:29:1f:1b:ab:53:8f:04:9a:d4:d9.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.0.12' (RSA) to the list of known hosts.
[email protected]'s password: 
BACKUP_FILE_01.meta                                                                            100%   73KB  72.5KB/s   00:00    
BACKUP_FILE_01.bak                                                                             100% 6246KB   6.1MB/s   00:00

---

5. 初始化数据库备库

## 5.1 初始化备库：库名 dmdb  实例名：GRP1_RT_02
[dmdba@DMDB2 ~]$ dminit PATH=/DMDB/data DB_NAME=dmdb INSTANCE_NAME=GRP1_RT_02
initdb V8
db version: 0x7000b
License will expire on 2021-06-25
 log file path: /DMDB/data/dmdb/dmdb01.log
 log file path: /DMDB/data/dmdb/dmdb02.log
write to dir [/DMDB/data/dmdb].
create dm database success. 2020-12-02 11:15:13

## 5.2 注册数据库实例的服务
[root@DMDB2 ~]# /DMDB/app/DM8/script/root/dm_service_installer.sh -t dmserver -dm_ini /DMDB/data/dmdb/dm.ini -p GRP1_RT_02
移动服务脚本文件(/DMDB/app/DM8/bin/DmServiceGRP1_RT_02 到 /etc/rc.d/init.d/DmServiceGRP1_RT_02)
创建服务(DmServiceGRP1_RT_02)完成
[root@DMDB2 ~]# chkconfig --list | grep -i dm
DmAPService     0:关闭  1:关闭  2:启用  3:启用  4:启用  5:启用  6:关闭
DmServiceGRP1_RT_02     0:关闭  1:关闭  2:启用  3:启用  4:启用  5:启用  6:关闭
mdmonitor       0:关闭  1:关闭  2:启用  3:启用  4:启用  5:启用  6:关闭
nfs-rdma        0:关闭  1:关闭  2:关闭  3:关闭  4:关闭  5:关闭  6:关闭
rdma            0:关闭  1:关闭  2:关闭  3:关闭  4:关闭  5:关闭  6:关闭
[root@DMDB2 ~]# service DmServiceGRP1_RT_02 start
Starting DmServiceGRP1_RT_02:                              [ OK ]

---

6. 备库脱机还原主库数据

### 6.1 停备库
[root@DMDB2 ~]# service DmServiceGRP1_RT_02 stop
Stopping DmServiceGRP1_RT_02:                              [ OK ]

### 6.2 开始脱机数据库还原和恢复
# 还原
[dmdba@DMDB2 ~]$ dmrman CTLSTMT="RESTORE DATABASE '/DMDB/data/dmdb/dm.ini' FROM BACKUPSET '/DMDB/backup/BACKUP_FILE_01'"
dmrman V8
RESTORE DATABASE '/DMDB/data/dmdb/dm.ini' FROM BACKUPSET '/DMDB/backup/BACKUP_FILE_01'
RESTORE DATABASE CHECK......
RESTORE DATABASE,data collect......
RESTORE DATABASE,database refresh ......
RESTORE BACKUPSET [/DMDB/backup/BACKUP_FILE_01] START......
total 3 packages processed...
total 7 packages processed...
RESTORE DATABASE,UPDATE ctl file......
RESTORE DATABASE,REBUILD key file......
RESTORE DATABASE,CHECK db info......
RESTORE DATABASE,UPDATE db info......
total 7 packages processed...
total 7 packages processed!
CMD END.CODE:[0]
restore successfully.
time used: 00:00:01.216


# 恢复
[dmdba@DMDB2 ~]$ dmrman CTLSTMT="RECOVER DATABASE '/DMDB/data/dmdb/dm.ini' FROM BACKUPSET '/DMDB/backup/BACKUP_FILE_01'"
dmrman V8
RECOVER DATABASE '/DMDB/data/dmdb/dm.ini' FROM BACKUPSET '/DMDB/backup/BACKUP_FILE_01'
Database mode = 0, oguid = 0
EP[0]'s cur_lsn[37090]
RESTORE RLOG CHECK......
CMD END.CODE:[603],DESC:[备份集[/DMDB/backup/BACKUP_FILE_01]备份过程中未产生日志]
备份集[/DMDB/backup/BACKUP_FILE_01]备份过程中未产生日志
recover successfully!
time used: 475.993(ms)


# 更新同步魔数
[dmdba@DMDB2 ~]$ dmrman CTLSTMT="RECOVER DATABASE '/DMDB/data/dmdb/dm.ini' UPDATE DB_MAGIC"
dmrman V8
RECOVER DATABASE '/DMDB/data/dmdb/dm.ini' UPDATE DB_MAGIC
Database mode = 0, oguid = 0
EP[0]'s cur_lsn[37090]
EP[0]'s apply_lsn[37090] >= end_lsn[37090]
recover successfully!
time used: 00:00:01.024


### 6.3 检查主库与备库之间的数据一致性
## 6.3.1 主备库两个实例分别以mount方式启动
[dmdba@DMDB1 dmdb]$ dmserver /DMDB/data/dmdb/dm.ini mount
file dm.key not found, use default license!
version info: develop
Use normal os_malloc instead of HugeTLB
Use normal os_malloc instead of HugeTLB
DM Database Server x64 V8 1-1-126-20.09.04-126608-ENT  startup...
Database mode = 0, oguid = 0
License will expire on 2021-09-04
file lsn: 37090
ndct db load finished
ndct fill fast pool finished
nsvr_startup end.
aud sys init success.
aud rt sys init success.
systables desc init success.
ndct_db_load_info success.
SYSTEM IS READY.

[dmdba@DMDB2 ~]$ dmserver /DMDB/data/dmdb/dm.ini mount
version info: security
Use normal os_malloc instead of HugeTLB
Use normal os_malloc instead of HugeTLB
DM Database Server x64 V8 1-1-126-20.09.04-126608-ENT  startup...
Database mode = 0, oguid = 0
License will expire on 2021-06-25
file lsn: 37090
ndct db load finished
ndct fill fast pool finished
nsvr_startup end.
aud sys init success.
aud rt sys init success.
systables desc init success.
ndct_db_load_info success.
SYSTEM IS READY.


## 6.3.2 对比主库与备库之间的魔数
[dmdba@DMDB1 ~]$ disql SYSDBA/SYSDBA
服务器[LOCALHOST:5236]:处于普通配置状态
登录使用时间 : 8.820(ms)
disql V8
SQL> select file_LSN, cur_LSN from v$rlog;
行号     FILE_LSN             CUR_LSN             
---------- -------------------- --------------------
1          37090                37090
已用时间: 6.816(毫秒). 执行号:0.

SQL> select permanent_magic;
行号     PERMANENT_MAGIC
---------- ---------------
1          738069168
已用时间: 2.936(毫秒). 执行号:1.


[root@DMDB1 ~]# su - dmdba
[dmdba@DMDB1 ~]$ disql SYSDBA/SYSDBA
服务器[LOCALHOST:5236]:处于普通配置状态
登录使用时间 : 6.145(ms)
disql V8
SQL> select file_lsn,cur_lsn from v$rlog;
行号     FILE_LSN             CUR_LSN             
---------- -------------------- --------------------
1          37090                37090
已用时间: 4.530(毫秒). 执行号:100.

SQL> select permanent_magic;
行号     PERMANENT_MAGIC
---------- ---------------
1          738069168
已用时间: 0.684(毫秒). 执行号:10

---

7. 配置主备库（参数文件）

7.1 配置dm.ini

主库

INSTANCE_NAME = GRP1_RT_01 # 实例名 "组名_守护环境_序号", 字符总长度不超过16个字符
PORT_NUM = 5236            #数据库实例监听端口
DW_PORT = 5336             #守护环境下，监听守护进程连接请求端口
DW_INACTIVE_INTERVAL = 60  #接收守护进程消息超时时间
ALTER_MODE_STATUS = 0      #不允许手工方式修改实例模式/状态/OGUID
ENABLE_OFFLINE_TS = 2      #不允许备库OFFLINE表空间
MAL_INI = 1                #打开MAL系统
ARCH_INI = 1               #打开归档配置
#HA_INST_CHECK_FLAG = 1     #检测是否多个实例进程同时启动
RLOG_SEND_APPLY_MON = 64   #统计最近64次的日志发送信息  

备库

INSTANCE_NAME  = GRP1_RT_02
PORT_NUM  = 5237             #数据库实例监听端口
DW_PORT =  5337              #守护环境下，监听守护进程连接端口
DW_INACTIVE_INTERVAL  = 60   #接收守护进程消息超时时间
ALTER_MODE_STATUS  = 0       #不允许手工方式修改实例模式/状态/OGUID
ENABLE_OFFLINE_TS  = 2       #不允许备库OFFLINE表空间
MAL_INI = 1                  #打开MAL系统
ARCH_INI  = 1                #打开归档配置
#HA_INST_CHECK_FLAG = 1       #检测是否多个实例进程同时启动
RLOG_SEND_APPLY_MON = 64     #统计最近64次的日志重演信息

7.2 配置dmmal.ini

主库/备库

MAL_CHECK_INTERVAL = 5         #MAL链路检测的时间间隔
MAL_CONN_FAIL_INTERVAL = 5     #判定MAL链路断开的时间

[MAL_INST1]
MAL_INST_NAME = GRP1_RT_01     #实例名，和dm.ini中的INSTANCE_NAME一致
MAL_HOST = 10.10.10.11         #MAL系统监听TCP连接的IP地址
MAL_PORT = 5436                #MAL系统监听TCP连接的端口
MAL_INST_HOST = 192.168.0.11   #实例的对外服务IP地址
MAL_INST_PORT = 5236           #实例的对外服务端口，和dm.ini中的PORT_NUM一致
MAL_DW_PORT  = 5536           #实例本地守护进程监听TCP连接的端口
MAL_INST_DW_PORT = 5336       #节点实例监听守护进程TCP连接的端口. 各个实例守护进程与监视器之间通信端口.

[MAL_INST2]
MAL_INST_NAME = GRP1_RT_02
MAL_HOST = 10.10.10.12
MAL_PORT = 5437
MAL_INST_HOST = 192.168.0.12
MAL_INST_PORT = 5237
MAL_DW_PORT  = 5537
MAL_INST_DW_PORT = 5337

7.3 配置dmarch.ini

主库

[ARCHIVE_REALTIME]
ARCH_TYPE  = REALTIME      #实时归档类型
ARCH_DEST  = GRP1_RT_02    #实时归档目标实例名，即远程归档到备库，向GRP1_RT_02实例同步数据

[ARCHIVE_LOCAL1]
ARCH_TYPE  = LOCAL         #本地归档类型
ARCH_DEST  = /DMDB/arch    #本地归档文件存放路径
ARCH_FILE_SIZE = 128       #单位 Mb，本地单个归档文件最大值
ARCH_SPACE_LIMIT = 0       #单位 Mb，0表示无限制，范围 1024~4294967294M

备库

[ARCHIVE_REALTIME]
ARCH_TYPE  = REALTIME      #实时归档类型
ARCH_DEST  = GRP1_RT_01    #实时归档目标实例名，即远程归档到备库，向GRP1_RT_01实例同步数据。两者互为主备库，在故障出现时。

[ARCHIVE_LOCAL1]
ARCH_TYPE  = LOCAL         #本地归档类型
ARCH_DEST  = /DMDB/arch    #本地归档文件存放路径
ARCH_FILE_SIZE = 128       #单位 Mb，本地单个归档文件最大值
ARCH_SPACE_LIMIT = 0       #单位 Mb，0表示无限制，范围 1024~4294967294M

7.4 配置dmwatcher.ini

全局守护类型，自动切换模式

主库

[GRP1]
DW_TYPE = GLOBAL               #全局守护类型
DW_MODE = AUTO                 #自动切换模式，切记：这个开启一定要配置确认监视器
DW_ERROR_TIME = 10             #远程守护进程故障认定时间
INST_RECOVER_TIME = 60         #主库守护进程启动恢复的间隔时间
INST_ERROR_TIME = 10           #本地实例故障认定时间
INST_OGUID = 453331            #守护系统唯一OGUID值
INST_INI = /DMDB/data/dmdb/dm.ini  #dm.ini配置文件路径
INST_AUTO_RESTART = 1          #打开实例的自动启动功能
INST_STARTUP_CMD = /DMDB/app/DM8/bin/dmserver  #命令行方式启动
RLOG_SEND_THRESHOLD = 0       #指定主库发送日志到备库的时间阈值，默认关闭
RLOG_APPLY_THRESHOLD = 0      #指定备库重演日志的时间阈值，默认关闭

备库

[GRP1]
DW_TYPE = GLOBAL               #全局守护类型
DW_MODE = AUTO                 #自动切换模式，切记：这个开启一定要配置确认监视器
DW_ERROR_TIME = 10             #远程守护进程故障认定时间
INST_RECOVER_TIME = 60         #主库守护进程启动恢复的间隔时间
INST_ERROR_TIME = 10           #本地实例故障认定时间
INST_OGUID = 453331            #守护系统唯一OGUID值
INST_INI = /DMDB/data/dmdb/dm.ini  #dm.ini配置文件路径
INST_AUTO_RESTART = 1          #打开实例的自动启动功能
INST_STARTUP_CMD = /DMDB/app/DM8/bin/dmserver  #命令行方式启动
RLOG_SEND_THRESHOLD = 0       #指定主库发送日志到备库的时间阈值，默认关闭
RLOG_APPLY_THRESHOLD = 0      #指定备库重演日志的时间阈值，默认关闭

配置dmwatcher.ctl （可选）

同一个守护进程组，必须使用同一份 dmwatcher.ctl。拷贝到各个数据库的数据目录下。

dmctlcvt TYPE=3 SRC=/DMDB/data/dmdb/dmwatcher.ini DEST=/DMDB/data/dmdb
cp /DMDB/data/dmdb/GRP1/dmwatcher.ctl /DMDB/data/dmdb

7.5 配置监视器 dmmonitor.ini (监视器主机)

确认监视器：确认监视器除了相同的命令支持外，在主库发生故障时，能够自动通知实时备库接管为新的主库，具有自动故障处理的功能。
故障自动切换模式下，必须配置确认监视器，并且确认监视器最多仅能配置一个。

MON_DW_CONFIRM = 1              #确认监视器模式
MON_LOG_PATH = /DMDB/data/log   #监视器日志文件存放路径
MON_LOG_INTERVAL  = 60          #每隔60s定时记录系统信息到日志文件
MON_LOG_FILE_SIZE = 32          #每个日志文件最大32M
MON_LOG_SPACE_LIMIT = 0         #不限定日志文件总占用空间

[GRP1]
MON_INST_OGUID = 453331         #组GRP1的唯一OGUID值
#以下配置为监视器到组GRP1的守护进程的连接信息，以"IP:PORT"的形式配置
#IP对应dmmal.ini中的MAL_HOST，PORT对应dmmal.ini中的MAL_DW_PORT
MON_DW_IP = 10.10.10.11:5536
MON_DW_IP = 10.10.10.12:5537

---

8. 启动主备库设置OGUID

### 8.1 主库
# mount方式启动
[dmdba@DMDB1 dmdb]$ dmserver /DMDB/data/dmdb/dm.ini mount
file dm.key not found, use default license!
version info: develop
Use normal os_malloc instead of HugeTLB
Use normal os_malloc instead of HugeTLB
DM Database Server x64 V8 1-1-126-20.09.04-126608-ENT  startup...
Database mode = 0, oguid = 0
License will expire on 2021-09-04
begin redo pwr log collect, last ckpt lsn: 37090 ...
redo pwr log collect finished
main rfil[/DMDB/data/dmdb/dmdb01.log]'s grp collect 0 valid pwr record, discard 0 invalid pwr record
EP[0]'s cur_lsn[37090]
begin redo log recover, last ckpt lsn: 37090 ...
redo log recover finished
ndct db load finished
ndct fill fast pool finished
nsvr_startup end.
aud sys init success.
aud rt sys init success.
systables desc init success.
ndct_db_load_info success.
SYSTEM IS READY.

#系统通过OGUID 值确定一个守护进程组，由用户保证 OGUID 值的唯一性，
#并确保数据守护系统中，数据库、守护进程和监视器配置相同的OGUID值。
[root@DMDB1 ~]# su - dmdba
[dmdba@DMDB1 ~]$ disql SYSDBA/SYSDBA
服务器[LOCALHOST:5236]:处于普通配置状态
登录使用时间 : 5.938(ms)
disql V8
SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 1);
DMSQL 过程已成功完成
已用时间: 25.411(毫秒). 执行号:0.

SQL> sp_set_oguid(453331);
DMSQL 过程已成功完成
已用时间: 43.392(毫秒). 执行号:1.

SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 0);
DMSQL 过程已成功完成
已用时间: 12.588(毫秒). 执行号:2.


### 8.2 备库
# mount方式启动
#一定要以mount方式启动数据库实例，否则系统启动时会重构回滚表空间，生成Redo日志；
#并且，启动后应用可能连接到数据库实例进行操作，破坏主备库的数据一致性。数据守护配置结束后，守护进程会自动Open数据库。
[dmdba@DMDB2 dmdb]$ dmserver /DMDB/data/dmdb/dm.ini mount
version info: security
Use normal os_malloc instead of HugeTLB
Use normal os_malloc instead of HugeTLB
DM Database Server x64 V8 1-1-126-20.09.04-126608-ENT  startup...
Database mode = 0, oguid = 0
License will expire on 2021-06-25
begin redo pwr log collect, last ckpt lsn: 37090 ...
redo pwr log collect finished
main rfil[/DMDB/data/dmdb/dmdb01.log]'s grp collect 0 valid pwr record, discard 0 invalid pwr record
EP[0]'s cur_lsn[37090]
begin redo log recover, last ckpt lsn: 37090 ...
redo log recover finished
ndct db load finished
ndct fill fast pool finished
nsvr_startup end.
aud sys init success.
aud rt sys init success.
systables desc init success.
ndct_db_load_info success.
SYSTEM IS READY

[dmdba@DMDB2 ~]$ disql SYSDBA/SYSDBA@localhost:5237
notBefore.
notAfter error.
notAfter error.
服务器[localhost:5237]:处于普通配置状态
登录使用时间 : 16.602(ms)
上次登录ip       : ::1
上次登录时间   : 2020-12-02 10:50:49
登录失败次数   : 0
口令是否过期   : 未过期
disql V8
SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 1);
DMSQL 过程已成功完成
已用时间: 19.704(毫秒). 执行号:0.

SQL> sp_set_oguid(453331);
DMSQL 过程已成功完成
已用时间: 8.817(毫秒). 执行号:1.

SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 0);
DMSQL 过程已成功完成
已用时间: 13.194(毫秒). 执行号:2

---

9. 修改数据库模式 (可与8步骤浓缩为一个步骤)

### 9.1 主库
SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 1);
SQL> ALTER DATABASE PRIMARY; 
SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 0);


### 9.2 备库
SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 1);
SQL> ALTER DATABASE STANDBY; 
SQL> SP_SET_PARA_VALUE(1, 'ALTER_MODE_STATUS', 0);

---

10. 启动守护进程

## 主库
#守护进程启动后，进入Startup状态，此时实例都处于Mount状态。
#守护进程开始广播自身和其监控实例的状态信息，结合自身信息和远程守护进程的广播信息，
#守护进程将本地实例Open，并切换为Open状态。
[dmdba@DMDB1 dmdb]$ dmwatcher /DMDB/data/dmdb/dmwatcher.ini 
DMWATCHER[4.0] V8
DMWATCHER[4.0] IS READY

## 备库
[dmdba@DMDB2 ~]$ dmwatcher /DMDB/data/dmdb/dmwatcher.ini
DMWATCHER[4.0] V8
DMWATCHER[4.0] IS READY


# 查看守护进程实例的状态
SQL> select GROUP_NAME,INST_NAME,DW_TYPE,DW_MODE,DW_STATUS from V$DMWATCHER;
行号     GROUP_NAME INST_NAME  DW_TYPE DW_MODE DW_STATUS
---------- ---------- ---------- ------- ------- ---------
1          GRP1       GRP1_RT_01 GLOBAL  AUTO    OPEN

SQL> select GROUP_NAME,INST_NAME,DW_TYPE,DW_MODE,DW_STATUS from  V$DMWATCHER;
行号     GROUP_NAME INST_NAME  DW_TYPE DW_MODE DW_STATUS
---------- ---------- ---------- ------- ------- ---------
1          GRP1       GRP1_RT_02 GLOBAL  AUTO    OPEN

---

11. 启动监视器

[dmdba@DMDB3 DAMENG]$ dmmonitor /DMDB/data/DAMENG/dmmonitor.ini
[monitor]         2020-12-02 15:34:09: DMMONITOR[4.0] V8
[monitor]         2020-12-02 15:34:09: DMMONITOR[4.0] IS READY.

[monitor]         2020-12-02 15:34:09: 收到守护进程(GRP1_RT_01)消息
                  WTIME                WSTATUS        INST_OK   INAME            ISTATUS     IMODE     RSTAT    N_OPEN   FLSN            CLSN            
                  2020-12-02 15:34:08  OPEN           OK        GRP1_RT_01       OPEN        PRIMARY   VALID    2        38352           38352           

[monitor]         2020-12-02 15:34:09: 收到守护进程(GRP1_RT_02)消息
                  WTIME                WSTATUS        INST_OK   INAME            ISTATUS     IMODE     RSTAT    N_OPEN   FLSN            CLSN            
                  2020-12-02 15:34:10  OPEN           OK        GRP1_RT_02       OPEN        STANDBY   VALID    2        38352           38352           
                  
help
达梦守护监视器当前支持以下命令:
用法说明: 
命令中的[group_name]和[db_name]，在组名或库名不唯一的情况下，需要指定名称，组名和库名之间用'.'分隔. 
对于show monitor命令，如果db_name未指定，则按照监视器ini的配置顺序，使用组中第一个活动的守护进程获取连接信息, 
#=============================================================================================#
#-----------------------------------系统全局命令----------------------------------------------#
1.help                                                            --显示帮助信息
2.exit                                                            --退出监视器
3.show version                                                    --显示监视器自身版本信息
4.show global info                                                --显示所有组的全局信息
5.show database [group_name.]db_name                              --显示指定库的详细信息
6.show [group_name]                                               --显示指定组的详细信息，如果未指定组名，则显示所有组的详细信息
7.show i[nterval] n                                               --每隔n秒自动显示所有组的详细信息
8.q                                                               --取消自动显示
9.list [[group_name.]db_name]                                     --列出指定组的库对应的守护进程配置信息，如果都未指定，则列出所有守护进程配置信息
10.show open info [group_name.]db_name                            --显示指定库的OPEN历史信息
11.show arch send info [group_name.]db_name                       --查看源库到指定组的指定库的归档同步信息(包含恢复间隔信息)
12.show apply stat [group_name.]db_name                           --查看指定组的指定库的日志重演信息
13.show monitor [group_name[.]] [db_name]                         --列出连接到指定守护进程的所有监视器信息

14.tip                                                            --查看系统当前运行状态

15.login                                                          --登录监视器
16.logout                                                         --退出登录

17.get takeover time                                              --获取备库开始自动接管需要延迟等待的时间

---输入q/Q退出或者回车键继续---    

show
2020-12-02 15:37:20 
#================================================================================#
GROUP            OGUID       MON_CONFIRM     MODE            MPP_FLAG  
GRP1             453331      TRUE            AUTO            FALSE     


<<DATABASE GLOBAL INFO:>>
IP              MAL_DW_PORT  WTIME                WTYPE     WCTLSTAT  WSTATUS        INAME            INST_OK   N_EP  N_OK  ISTATUS     IMODE     DSC_STATUS     RTYPE     RSTAT    
10.10.10.11     5536         2020-12-02 15:37:18  GLOBAL    VALID     OPEN           GRP1_RT_01       OK        1     1     OPEN        PRIMARY   DSC_OPEN       REALTIME  VALID    

EP INFO:
INST_PORT  INST_OK   INAME            ISTATUS     IMODE     DSC_SEQNO  DSC_CTL_NODE RTYPE     RSTAT    FSEQ            FLSN            CSEQ            CLSN            DW_STAT_FLAG          
5236       OK        GRP1_RT_01       OPEN        PRIMARY   0          0            REALTIME  VALID    3933            38352           3933            38352           NONE                  

<<DATABASE GLOBAL INFO:>>
IP              MAL_DW_PORT  WTIME                WTYPE     WCTLSTAT  WSTATUS        INAME            INST_OK   N_EP  N_OK  ISTATUS     IMODE     DSC_STATUS     RTYPE     RSTAT    
10.10.10.12     5537         2020-12-02 15:37:21  GLOBAL    VALID     OPEN           GRP1_RT_02       OK        1     1     OPEN        STANDBY   DSC_OPEN       REALTIME  VALID    

EP INFO:
INST_PORT  INST_OK   INAME            ISTATUS     IMODE     DSC_SEQNO  DSC_CTL_NODE RTYPE     RSTAT    FSEQ            FLSN            CSEQ            CLSN            DW_STAT_FLAG          
5237       OK        GRP1_RT_02       OPEN        STANDBY   0          0            REALTIME  VALID    3927            38352           3927            38352           NONE                  

DATABASE(GRP1_RT_02) APPLY INFO FROM (GRP1_RT_01):
DSC_SEQNO[0], (ASEQ, SSEQ, KSEQ)[3933, 3933, 3933], (ALSN, SLSN, KLSN)[38352, 38352, 38352], N_TSK[0], TSK_MEM_USE[0]


#================================================================================#    

---

12. 注册服务(root用户操作)

### 12.1 守护进程服务
# 主库
/DMDB/app/DM8/script/root/dm_service_installer.sh -t dmwatcher -watcher_ini /DMDB/data/dmdb/dmwatcher.ini -p DW1

# 备库
/DMDB/app/DM8/script/root/dm_service_installer.sh -t dmwatcher -watcher_ini /DMDB/data/dmdb/dmwatcher.ini -p DW2


### 12.2 监视器服务
# 监视器主机
/DMDB/app/DM8/script/root/dm_service_installer.sh -t dmmonitor -monitor_ini /DMDB/data/DAMENG/dmmonitor.ini -p MON1

---

13. 测试主库插入数据后是否同步到备库

## 主库准备测试数据
[dmdba@DMDB1 ~]$ disql SYSDBA/SYSDBA
服务器[LOCALHOST:5236]:处于主库打开状态
登录使用时间 : 6.784(ms)
disql V8
SQL> create table dmfs as select * from sysobjects;     
操作已执行
已用时间: 23.862(毫秒). 执行号:705.
SQL> select count(1) from dmfs;

行号     COUNT(1)            
---------- --------------------
1          1340
用时间: 3.975(毫秒). 执行号:706.


## 备库查询，是否应用主库最新的日志
[dmdba@DMDB2 ~]$ disql SYSDBA/SYSDBA@LOCALHOST:5237
notBefore.
notAfter error.
notAfter error.
服务器[LOCALHOST:5237]:处于备库打开状态
登录使用时间 : 17.063(ms)
上次登录ip       : ::1
上次登录时间   : 2020-12-02 15:56:15
登录失败次数   : 0
口令是否过期   : 未过期
disql V8
SQL> select count(1) from dmfs;
行号     COUNT(1)            
---------- --------------------
1          1340
已用时间: 8.217(毫秒). 执行号:600


## 验证备库不能做数据修改
SQL> create table dmfs2 as select * from sysobjects; 
create table dmfs2 as select * from sysobjects; 
[-710]:试图在STANDBY模式下，修改用户库.
已用时间: 5.986(毫秒). 执行号:0.

---

14. 测试数据库服务持续可用性

### 14.1 模拟服务器断电
# 主库节点DMDB1
shutdown -h now

### 14.2 检查监视器状态
show global info
2020-12-02 16:05:29 
#================================================================================#
GROUP            OGUID       MON_CONFIRM     MODE            MPP_FLAG  
GRP1             453331      TRUE            AUTO            FALSE     


<<DATABASE GLOBAL INFO:>>
IP              MAL_DW_PORT  WTIME                WTYPE     WCTLSTAT  WSTATUS        INAME            INST_OK   N_EP  N_OK  ISTATUS     IMODE     DSC_STATUS     RTYPE     RSTAT    
10.10.10.12     5537         2020-12-02 16:05:31  GLOBAL    VALID     OPEN           GRP1_RT_02       OK        1     1     OPEN        STANDBY   DSC_OPEN       REALTIME  VALID    


ERROR DATABASE:
IP              MAL_DW_PORT  WTIME                WTYPE     WCTLSTAT  WSTATUS        INAME            INST_OK   N_EP  N_OK  ISTATUS     IMODE     DSC_STATUS     RTYPE     RSTAT    
10.10.10.11     5536         2020-12-02 16:05:16  GLOBAL    VALID     ERROR          GRP1_RT_01       OK        1     1     OPEN        PRIMARY   DSC_OPEN       REALTIME  VALID    


#================================================================================#

[monitor]         2020-12-02 16:05:38: 检测到PRIMARY实例故障，开始对组(GRP1)执行自动接管

[monitor]         2020-12-02 16:05:38: 通知组(GRP1)当前活动的守护进程设置MID
[monitor]         2020-12-02 16:05:38: 通知组(GRP1)当前活动的守护进程设置MID成功
[monitor]         2020-12-02 16:05:38: 开始使用实例GRP1_RT_02接管
[monitor]         2020-12-02 16:05:38: 通知守护进程GRP1_RT_02切换TAKEOVER状态
[monitor]         2020-12-02 16:05:38: 守护进程(GRP1_RT_02)状态切换 [OPEN-->TAKEOVER]
[monitor]         2020-12-02 16:05:38: 切换守护进程GRP1_RT_02为TAKEOVER状态成功
[monitor]         2020-12-02 16:05:38: 实例GRP1_RT_02开始执行SP_SET_GLOBAL_DW_STATUS(0, 7)语句
[monitor]         2020-12-02 16:05:38: 实例GRP1_RT_02执行SP_SET_GLOBAL_DW_STATUS(0, 7)语句成功
[monitor]         2020-12-02 16:05:38: 实例GRP1_RT_02开始执行SP_APPLY_KEEP_PKG()语句
[monitor]         2020-12-02 16:05:39: 实例GRP1_RT_02执行SP_APPLY_KEEP_PKG()语句成功
[monitor]         2020-12-02 16:05:39: 实例GRP1_RT_02开始执行ALTER DATABASE MOUNT语句
[monitor]         2020-12-02 16:05:40: 实例GRP1_RT_02执行ALTER DATABASE MOUNT语句成功
[monitor]         2020-12-02 16:05:40: 实例GRP1_RT_02开始执行ALTER DATABASE PRIMARY语句
[monitor]         2020-12-02 16:05:41: 实例GRP1_RT_02执行ALTER DATABASE PRIMARY语句成功
[monitor]         2020-12-02 16:05:44: 通知实例GRP1_RT_02修改所有归档状态无效
[monitor]         2020-12-02 16:05:44: 修改所有实例归档为无效状态成功
[monitor]         2020-12-02 16:05:44: 实例GRP1_RT_02开始执行ALTER DATABASE OPEN FORCE语句
[monitor]         2020-12-02 16:05:46: 实例GRP1_RT_02执行ALTER DATABASE OPEN FORCE语句成功
[monitor]         2020-12-02 16:05:46: 实例GRP1_RT_02开始执行SP_SET_GLOBAL_DW_STATUS(7, 0)语句
[monitor]         2020-12-02 16:05:46: 实例GRP1_RT_02执行SP_SET_GLOBAL_DW_STATUS(7, 0)语句成功
[monitor]         2020-12-02 16:05:46: 通知守护进程GRP1_RT_02切换OPEN状态
[monitor]         2020-12-02 16:05:46: 守护进程(GRP1_RT_02)状态切换 [TAKEOVER-->OPEN]
[monitor]         2020-12-02 16:05:46: 切换守护进程GRP1_RT_02为OPEN状态成功
[monitor]         2020-12-02 16:05:46: 通知组(GRP1)的守护进程执行清理操作
[monitor]         2020-12-02 16:05:46: 清理守护进程(GRP1_RT_02)请求成功
[monitor]         2020-12-02 16:05:46: 使用实例GRP1_RT_02接管成功
[monitor]         2020-12-02 16:05:46: 组(GRP1)使用实例GRP1_RT_02自动接管成功


show
2020-12-02 16:09:57 
#================================================================================#
GROUP            OGUID       MON_CONFIRM     MODE            MPP_FLAG  
GRP1             453331      TRUE            AUTO            FALSE     


<<DATABASE GLOBAL INFO:>>
IP              MAL_DW_PORT  WTIME                WTYPE     WCTLSTAT  WSTATUS        INAME            INST_OK   N_EP  N_OK  ISTATUS     IMODE     DSC_STATUS     RTYPE     RSTAT    
10.10.10.12     5537         2020-12-02 16:09:58  GLOBAL    VALID     OPEN           GRP1_RT_02       OK        1     1     OPEN        PRIMARY   DSC_OPEN       REALTIME  VALID    

EP INFO:
INST_PORT  INST_OK   INAME            ISTATUS     IMODE     DSC_SEQNO  DSC_CTL_NODE RTYPE     RSTAT    FSEQ            FLSN            CSEQ            CLSN            DW_STAT_FLAG          
5237       OK        GRP1_RT_02       OPEN        PRIMARY   0          0            REALTIME  VALID    3945            39941           3945            39941           NONE                  

ERROR DATABASE:

<<DATABASE GLOBAL INFO:>>
IP              MAL_DW_PORT  WTIME                WTYPE     WCTLSTAT  WSTATUS        INAME            INST_OK   N_EP  N_OK  ISTATUS     IMODE     DSC_STATUS     RTYPE     RSTAT    
10.10.10.11     5536         2020-12-02 16:05:16  GLOBAL    VALID     ERROR          GRP1_RT_01       OK        1     1     OPEN        PRIMARY   DSC_OPEN       REALTIME  VALID    

EP INFO:
INST_PORT  INST_OK   INAME            ISTATUS     IMODE     DSC_SEQNO  DSC_CTL_NODE RTYPE     RSTAT    FSEQ            FLSN            CSEQ            CLSN            DW_STAT_FLAG          
5236       OK        GRP1_RT_01       OPEN        PRIMARY   0          0            REALTIME  VALID    3943            38583           3943            38583           NONE                  

#================================================================================#

开机DMDB1，让其自动恢复加入数据守护集群。

开机自启动数据库各服务：辅助服务、数据库实例服务、数据守护服务

[root@DMDB1 ~]# ps -fu dmdba
UID         PID   PPID  C STIME TTY          TIME CMD
dmdba      2332      1  0 16:15 ?        00:00:00 /DMDB/app/DM8/bin/dmap
dmdba      2389      1  1 16:15 ?        00:00:08 /DMDB/app/DM8/bin/dmserver /DMDB/data/dmdb/dm.ini -noconsole
dmdba      2512      1  0 16:15 ?        00:00:02 /DMDB/app/DM8/bin/dmwatcher /DMDB/data/dmdb/dmwatcher.ini

# 监视器情况
[monitor]         2020-12-02 16:16:02: 守护进程(GRP1_RT_01)状态切换 [NONE-->STARTUP]
[monitor]         2020-12-02 16:16:02: 守护进程(GRP1_RT_01)状态切换 [STARTUP-->UNIFY EP]
                  WTIME                WSTATUS        INST_OK   INAME            ISTATUS     IMODE     RSTAT    N_OPEN   FLSN            CLSN            
                  2020-12-02 16:16:01  UNIFY EP       OK        GRP1_RT_01       MOUNT       PRIMARY   VALID    2        38583           38583           

[monitor]         2020-12-02 16:16:02: 守护进程(GRP1_RT_01)状态切换 [UNIFY EP-->STARTUP]
                  WTIME                WSTATUS        INST_OK   INAME            ISTATUS     IMODE     RSTAT    N_OPEN   FLSN            CLSN            
                  2020-12-02 16:16:01  STARTUP        OK        GRP1_RT_01       MOUNT       STANDBY   INVALID  2        38583           38583           

[monitor]         2020-12-02 16:16:02: 守护进程(GRP1_RT_01)状态切换 [STARTUP-->UNIFY EP]
                  WTIME                WSTATUS        INST_OK   INAME            ISTATUS     IMODE     RSTAT    N_OPEN   FLSN            CLSN            
                  2020-12-02 16:16:01  UNIFY EP       OK        GRP1_RT_01       MOUNT       STANDBY   INVALID  2        38583           38583           

[monitor]         2020-12-02 16:16:02: 守护进程(GRP1_RT_01)状态切换 [UNIFY EP-->STARTUP]
                  WTIME                WSTATUS        INST_OK   INAME            ISTATUS     IMODE     RSTAT    N_OPEN   FLSN            CLSN            
                  2020-12-02 16:16:01  STARTUP        OK        GRP1_RT_01       OPEN        STANDBY   INVALID  2        38583           38583           

[monitor]         2020-12-02 16:16:02: 守护进程(GRP1_RT_01)状态切换 [STARTUP-->OPEN]
                  WTIME                WSTATUS        INST_OK   INAME            ISTATUS     IMODE     RSTAT    N_OPEN   FLSN            CLSN            
                  2020-12-02 16:16:02  OPEN           OK        GRP1_RT_01       OPEN        STANDBY   INVALID  2        38583           38583           

[monitor]         2020-12-02 16:16:02: 守护进程(GRP1_RT_02)状态切换 [OPEN-->RECOVERY]
                  WTIME                WSTATUS        INST_OK   INAME            ISTATUS     IMODE     RSTAT    N_OPEN   FLSN            CLSN            
                  2020-12-02 16:16:04  RECOVERY       OK        GRP1_RT_02       OPEN        PRIMARY   VALID    3        39941           39941           

[monitor]         2020-12-02 16:16:06: 守护进程(GRP1_RT_02)状态切换 [RECOVERY-->OPEN]
                  WTIME                WSTATUS        INST_OK   INAME            ISTATUS     IMODE     RSTAT    N_OPEN   FLSN            CLSN            
                  2020-12-02 16:16:08  OPEN           OK        GRP1_RT_02       OPEN        PRIMARY   VALID    3        39941           39941  


show
2020-12-02 16:26:02 
#================================================================================#
GROUP            OGUID       MON_CONFIRM     MODE            MPP_FLAG  
GRP1             453331      TRUE            AUTO            FALSE     


<<DATABASE GLOBAL INFO:>>
IP              MAL_DW_PORT  WTIME                WTYPE     WCTLSTAT  WSTATUS        INAME            INST_OK   N_EP  N_OK  ISTATUS     IMODE     DSC_STATUS     RTYPE     RSTAT    
10.10.10.12     5537         2020-12-02 16:26:03  GLOBAL    VALID     OPEN           GRP1_RT_02       OK        1     1     OPEN        PRIMARY   DSC_OPEN       REALTIME  VALID    

EP INFO:
INST_PORT  INST_OK   INAME            ISTATUS     IMODE     DSC_SEQNO  DSC_CTL_NODE RTYPE     RSTAT    FSEQ            FLSN            CSEQ            CLSN            DW_STAT_FLAG          
5237       OK        GRP1_RT_02       OPEN        PRIMARY   0          0            REALTIME  VALID    3945            39941           3945            39941           NONE                  

<<DATABASE GLOBAL INFO:>>
IP              MAL_DW_PORT  WTIME                WTYPE     WCTLSTAT  WSTATUS        INAME            INST_OK   N_EP  N_OK  ISTATUS     IMODE     DSC_STATUS     RTYPE     RSTAT    
10.10.10.11     5536         2020-12-02 16:26:00  GLOBAL    VALID     OPEN           GRP1_RT_01       OK        1     1     OPEN        STANDBY   DSC_OPEN       REALTIME  VALID    

EP INFO:
INST_PORT  INST_OK   INAME            ISTATUS     IMODE     DSC_SEQNO  DSC_CTL_NODE RTYPE     RSTAT    FSEQ            FLSN            CSEQ            CLSN            DW_STAT_FLAG          
5236       OK        GRP1_RT_01       OPEN        STANDBY   0          0            REALTIME  VALID    3943            39941           3943            39941           NONE                  

DATABASE(GRP1_RT_01) APPLY INFO FROM (GRP1_RT_02):
DSC_SEQNO[0], (ASEQ, SSEQ, KSEQ)[3945, 3945, 3945], (ALSN, SLSN, KLSN)[39941, 39941, 39941], N_TSK[0], TSK_MEM_USE[0]


show global info
2020-12-02 16:26:17 
#================================================================================#
GROUP            OGUID       MON_CONFIRM     MODE            MPP_FLAG  
GRP1             453331      TRUE            AUTO            FALSE     


<<DATABASE GLOBAL INFO:>>
IP              MAL_DW_PORT  WTIME                WTYPE     WCTLSTAT  WSTATUS        INAME            INST_OK   N_EP  N_OK  ISTATUS     IMODE     DSC_STATUS     RTYPE     RSTAT    
10.10.10.12     5537         2020-12-02 16:26:18  GLOBAL    VALID     OPEN           GRP1_RT_02       OK        1     1     OPEN        PRIMARY   DSC_OPEN       REALTIME  VALID    

IP              MAL_DW_PORT  WTIME                WTYPE     WCTLSTAT  WSTATUS        INAME            INST_OK   N_EP  N_OK  ISTATUS     IMODE     DSC_STATUS     RTYPE     RSTAT    
10.10.10.11     5536         2020-12-02 16:26:16  GLOBAL    VALID     OPEN           GRP1_RT_01       OK        1     1     OPEN        STANDBY   DSC_OPEN       REALTIME  VALID    


#================================================================================#

SQL> select decode(role$,1,'PRI',role$),decode(STATUS$,4,'open',3,'mount',5,'supend')  from v$database;
行号     DECODE("ROLE$",1,'PRI',"ROLE$") DECODE("STATUS$",4,'open',3,'mount',5,'supend')
---------- ------------------------------- -----------------------------------------------
1          PRI                             open

---

15. 手动切换主备集群

恢复GRP1_RT_01实例为主库

## 在监视器主机环境
#================================================================================#

switchover
[monitor]         2020-12-02 16:34:29: 开始切换实例GRP1_RT_01
[monitor]         2020-12-02 16:34:29: 通知守护进程GRP1_RT_02切换SWITCHOVER状态
[monitor]         2020-12-02 16:34:29: 守护进程(GRP1_RT_02)状态切换 [OPEN-->SWITCHOVER]
[monitor]         2020-12-02 16:34:29: 切换守护进程GRP1_RT_02为SWITCHOVER状态成功
[monitor]         2020-12-02 16:34:29: 通知守护进程GRP1_RT_01切换SWITCHOVER状态
[monitor]         2020-12-02 16:34:29: 守护进程(GRP1_RT_01)状态切换 [OPEN-->SWITCHOVER]
[monitor]         2020-12-02 16:34:29: 切换守护进程GRP1_RT_01为SWITCHOVER状态成功
[monitor]         2020-12-02 16:34:29: 实例GRP1_RT_02开始执行SP_SET_GLOBAL_DW_STATUS(0, 6)语句
[monitor]         2020-12-02 16:34:29: 实例GRP1_RT_02执行SP_SET_GLOBAL_DW_STATUS(0, 6)语句成功
[monitor]         2020-12-02 16:34:29: 实例GRP1_RT_01开始执行SP_SET_GLOBAL_DW_STATUS(0, 6)语句
[monitor]         2020-12-02 16:34:30: 实例GRP1_RT_01执行SP_SET_GLOBAL_DW_STATUS(0, 6)语句成功
[monitor]         2020-12-02 16:34:30: 实例GRP1_RT_02开始执行ALTER DATABASE MOUNT语句
[monitor]         2020-12-02 16:34:31: 实例GRP1_RT_02执行ALTER DATABASE MOUNT语句成功
[monitor]         2020-12-02 16:34:31: 实例GRP1_RT_01开始执行SP_APPLY_KEEP_PKG()语句
[monitor]         2020-12-02 16:34:32: 实例GRP1_RT_01执行SP_APPLY_KEEP_PKG()语句成功
[monitor]         2020-12-02 16:34:32: 实例GRP1_RT_01开始执行ALTER DATABASE MOUNT语句
[monitor]         2020-12-02 16:34:33: 实例GRP1_RT_01执行ALTER DATABASE MOUNT语句成功
[monitor]         2020-12-02 16:34:33: 实例GRP1_RT_02开始执行ALTER DATABASE STANDBY语句
[monitor]         2020-12-02 16:34:33: 实例GRP1_RT_02执行ALTER DATABASE STANDBY语句成功
[monitor]         2020-12-02 16:34:33: 实例GRP1_RT_01开始执行ALTER DATABASE PRIMARY语句
[monitor]         2020-12-02 16:34:34: 实例GRP1_RT_01执行ALTER DATABASE PRIMARY语句成功
[monitor]         2020-12-02 16:34:34: 通知实例GRP1_RT_01修改所有归档状态无效
[monitor]         2020-12-02 16:34:34: 修改所有实例归档为无效状态成功
[monitor]         2020-12-02 16:34:34: 实例GRP1_RT_02开始执行ALTER DATABASE OPEN FORCE语句
[monitor]         2020-12-02 16:34:34: 实例GRP1_RT_02执行ALTER DATABASE OPEN FORCE语句成功
[monitor]         2020-12-02 16:34:34: 实例GRP1_RT_01开始执行ALTER DATABASE OPEN FORCE语句
[monitor]         2020-12-02 16:34:36: 实例GRP1_RT_01执行ALTER DATABASE OPEN FORCE语句成功
[monitor]         2020-12-02 16:34:36: 实例GRP1_RT_02开始执行SP_SET_GLOBAL_DW_STATUS(6, 0)语句
[monitor]         2020-12-02 16:34:36: 实例GRP1_RT_02执行SP_SET_GLOBAL_DW_STATUS(6, 0)语句成功
[monitor]         2020-12-02 16:34:36: 实例GRP1_RT_01开始执行SP_SET_GLOBAL_DW_STATUS(6, 0)语句
[monitor]         2020-12-02 16:34:36: 实例GRP1_RT_01执行SP_SET_GLOBAL_DW_STATUS(6, 0)语句成功
[monitor]         2020-12-02 16:34:36: 通知守护进程GRP1_RT_02切换OPEN状态
[monitor]         2020-12-02 16:34:36: 守护进程(GRP1_RT_02)状态切换 [SWITCHOVER-->OPEN]
[monitor]         2020-12-02 16:34:36: 切换守护进程GRP1_RT_02为OPEN状态成功
[monitor]         2020-12-02 16:34:36: 通知守护进程GRP1_RT_01切换OPEN状态
[monitor]         2020-12-02 16:34:36: 守护进程(GRP1_RT_01)状态切换 [SWITCHOVER-->OPEN]
[monitor]         2020-12-02 16:34:36: 切换守护进程GRP1_RT_01为OPEN状态成功
[monitor]         2020-12-02 16:34:36: 通知组(GRP1)的守护进程执行清理操作
[monitor]         2020-12-02 16:34:36: 清理守护进程(GRP1_RT_01)请求成功
[monitor]         2020-12-02 16:34:36: 清理守护进程(GRP1_RT_02)请求成功
[monitor]         2020-12-02 16:34:36: 实例GRP1_RT_01切换成功

2020-12-02 16:34:36 
#================================================================================#
GROUP            OGUID       MON_CONFIRM     MODE            MPP_FLAG  
GRP1             453331      TRUE            AUTO            FALSE     


<<DATABASE GLOBAL INFO:>>
IP              MAL_DW_PORT  WTIME                WTYPE     WCTLSTAT  WSTATUS        INAME            INST_OK   N_EP  N_OK  ISTATUS     IMODE     DSC_STATUS     RTYPE     RSTAT    
10.10.10.11     5536         2020-12-02 16:34:35  GLOBAL    VALID     OPEN           GRP1_RT_01       OK        1     1     OPEN        PRIMARY   DSC_OPEN       REALTIME  VALID    

EP INFO:
INST_PORT  INST_OK   INAME            ISTATUS     IMODE     DSC_SEQNO  DSC_CTL_NODE RTYPE     RSTAT    FSEQ            FLSN            CSEQ            CLSN            DW_STAT_FLAG          
5236       OK        GRP1_RT_01       OPEN        PRIMARY   0          0            REALTIME  VALID    3946            41299           3946            41299           NONE                  

<<DATABASE GLOBAL INFO:>>
IP              MAL_DW_PORT  WTIME                WTYPE     WCTLSTAT  WSTATUS        INAME            INST_OK   N_EP  N_OK  ISTATUS     IMODE     DSC_STATUS     RTYPE     RSTAT    
10.10.10.12     5537         2020-12-02 16:34:38  GLOBAL    VALID     OPEN           GRP1_RT_02       OK        1     1     OPEN        STANDBY   DSC_OPEN       REALTIME  INVALID  

EP INFO:
INST_PORT  INST_OK   INAME            ISTATUS     IMODE     DSC_SEQNO  DSC_CTL_NODE RTYPE     RSTAT    FSEQ            FLSN            CSEQ            CLSN            DW_STAT_FLAG          
5237       OK        GRP1_RT_02       OPEN        STANDBY   0          0            REALTIME  INVALID  3945            39941           3945            39941           NONE                  

DATABASE(GRP1_RT_02) APPLY INFO FROM (GRP1_RT_01):
DSC_SEQNO[0], (ASEQ, SSEQ, KSEQ)[3945, 3945, 3945], (ALSN, SLSN, KLSN)[39941, 39941, 39941], N_TSK[0], TSK_MEM_USE[0]


#================================================================================#

[monitor]         2020-12-02 16:34:39: 守护进程(GRP1_RT_01)状态切换 [OPEN-->RECOVERY]
                  WTIME                WSTATUS        INST_OK   INAME            ISTATUS     IMODE     RSTAT    N_OPEN   FLSN            CLSN            
                  2020-12-02 16:34:38  RECOVERY       OK        GRP1_RT_01       OPEN        PRIMARY   VALID    4        41299           41299           

[monitor]         2020-12-02 16:34:42: 守护进程(GRP1_RT_01)状态切换 [RECOVERY-->OPEN]
                  WTIME                WSTATUS        INST_OK   INAME            ISTATUS     IMODE     RSTAT    N_OPEN   FLSN            CLSN            
                  2020-12-02 16:34:41  OPEN           OK        GRP1_RT_01       OPEN        PRIMARY   VALID    4        41299           41299           


show
2020-12-02 16:35:17 
#================================================================================#
GROUP            OGUID       MON_CONFIRM     MODE            MPP_FLAG  
GRP1             453331      TRUE            AUTO            FALSE     


<<DATABASE GLOBAL INFO:>>
IP              MAL_DW_PORT  WTIME                WTYPE     WCTLSTAT  WSTATUS        INAME            INST_OK   N_EP  N_OK  ISTATUS     IMODE     DSC_STATUS     RTYPE     RSTAT    
10.10.10.11     5536         2020-12-02 16:35:16  GLOBAL    VALID     OPEN           GRP1_RT_01       OK        1     1     OPEN        PRIMARY   DSC_OPEN       REALTIME  VALID    

EP INFO:
INST_PORT  INST_OK   INAME            ISTATUS     IMODE     DSC_SEQNO  DSC_CTL_NODE RTYPE     RSTAT    FSEQ            FLSN            CSEQ            CLSN            DW_STAT_FLAG          
5236       OK        GRP1_RT_01       OPEN        PRIMARY   0          0            REALTIME  VALID    3946            41299           3946            41299           NONE                  

<<DATABASE GLOBAL INFO:>>
IP              MAL_DW_PORT  WTIME                WTYPE     WCTLSTAT  WSTATUS        INAME            INST_OK   N_EP  N_OK  ISTATUS     IMODE     DSC_STATUS     RTYPE     RSTAT    
10.10.10.12     5537         2020-12-02 16:35:18  GLOBAL    VALID     OPEN           GRP1_RT_02       OK        1     1     OPEN        STANDBY   DSC_OPEN       REALTIME  VALID    

EP INFO:
INST_PORT  INST_OK   INAME            ISTATUS     IMODE     DSC_SEQNO  DSC_CTL_NODE RTYPE     RSTAT    FSEQ            FLSN            CSEQ            CLSN            DW_STAT_FLAG          
5237       OK        GRP1_RT_02       OPEN        STANDBY   0          0            REALTIME  VALID    3945            41299           3945            41299           NONE                  

DATABASE(GRP1_RT_02) APPLY INFO FROM (GRP1_RT_01):
DSC_SEQNO[0], (ASEQ, SSEQ, KSEQ)[3946, 3946, 3946], (ALSN, SLSN, KLSN)[41299, 41299, 41299], N_TSK[0], TSK_MEM_USE[0]

---

总结：

启动数据守护集群步骤

启动主库实例 ==> 启动主库的守护进程 ==> 启动备库实例 ==> 启动备库的守护进程 ==> 启动确认监视器

关闭数据守护集群步骤

关闭确认监视器 ==> 关闭备库的守护进程 ==> 关闭主库的守护进程 ==> 关闭主库实例服务 ==> 关闭备库实例服务