oracle 12.1.0.2 使用 MAA 架构之四 Global Data Services(GDS)
os: centos 7.4
db: oracle 12.1.0.2
个人感觉这个东西有点类似 haproxy.不过看起来比 haproxy 高大上许多.
个人感觉如果oracle数据库不多的话,使用上一篇<
GDS 依赖一个 oracle 数据库,用来存放 GDS Catalog.需要先安装一个 12.1.0.2 的数据库.
• Oracle Global Data Services (GDS)
Oracle Global Data Services (GDS) applies the Oracle RAC service model to
pools of globally distributed databases, providing dynamic load balancing, failover,
and centralized service management for a set of replicated databases that offer
common services. The set of databases can include Oracle RAC and single
instance Oracle databases interconnected through Oracle Data Guard, Oracle
GoldenGate, or any other replication technology.
GDS is integrated with Oracle Data Guard broker. This allows role-specific global
services to be automatically started and stopped as appropriate when role
transitions occur within an Oracle Data Guard broker configuration.
GDS allows the specification of a replication lag limit for a global service. If the lag
limit is exceeded at a given replica, the global service is temporarily stopped at
that replica and new client requests are routed to a replica that satisfies the lag
limit. The global service is automatically restarted at the original replica when the
replication lag becomes less than the lag limit
maa 信息如下
primary standby1 standby2
hosts node1,node2 nodea,nodeb nodeasm
db_name orcl0 orcl0 orcl0
db_unique_name orclp orcls1 orcls2
instance_name orclp1,orclp2 orcls11,orcls12 orcls21
gds 信息如下
gdsdb 是必须的，是用来存放 GDS Catalog。暂时设置为单实例,但是按照 adg 的标准去规划.
primary
hosts nodegdsdb
db_name gdsdb
db_unique_name gdsdbp
instance_name gdsdbp1
gsm 信息如下,先配置一台
hosts nodegsm1
gdsdb os 设置
参考<
<
gdsdb rdbms 安装
参考<
<
这个是最基本的 oracle 数据库安装,没啥好说的.
简单起见,就安装个最简单的非pdb ,使用dbf存储.
gsm 安装
gsm 全称是 Global Service Manager，即 全局服务管理器，是gds的主要组成部分。gsm的职责如下：
- 提供客户端连接时的负载均衡
- 提供service级别的负载均衡，故障转移
- 创建 run time LBA(load balancing advisory),并分发到所有的客户端
- 监控数据库实例的可用性和全局服务的可用性，并在失败的时候通知客户端
- 作为区域的监听，用户客户端连接 global service
# vi /home/oracle/.bash_profile
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/12.1.0/gsm_1
export TNS_ADMIN=$ORACLE_HOME/network/admin
export NLS_LANG=AMERICAN_AMERICA.AL32UTF8
export NLS_DATE_FORMAT='yyyy-mm-dd hh24:mi:ss'
export PATH=.:${PATH}:$HOME/bin:$ORACLE_HOME/bin
export PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
export PATH=${PATH}:$ORACLE_BASE/common/oracle/bin
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export CLASSPATH=$ORACLE_HOME/JRE
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export TEMP=/tmp
export TMP=/tmp
export TMPDIR=/tmp
# xhost +
# su - oracle
$ export DISPLAY=:0.0
$ xclock
$ rz
$ ls -l
-rw-r--r-- 1 oracle oinstall 567375160 Apr 24 17:12 linuxamd64_12102_gsm.zip
$ unzip linuxamd64_12102_gsm.zip
$ cd gsm
$ ./runInstaller
gsm 配置
需要登录 gdsdb 创建用户
# su - oracle
$ sqlplus / as sysdba;
SQL> set lines 200;
col username format a20;
col account_status format a20;
SQL> select username,account_status from dba_users where username like '%GSM%';
USERNAME ACCOUNT_STATUS
-------------------- --------------------
GSMCATUSER EXPIRED & LOCKED
GSMADMIN_INTERNAL EXPIRED & LOCKED
GSMUSER EXPIRED & LOCKED
SQL> create user mygsm identified by mygsm default tablespace users;
SQL> grant gsmadmin_role to mygsm;
SQL> alter user gsmcatuser identified by gsmcatuser account unlock;
创建连接
# su - oracle
$ which gdsctl
/u01/app/oracle/product/12.1.0/gsm_1/bin/gdsctl
$ gdsctl
GDSCTL: Version 12.1.0.2.0 - Production on Mon May 06 11:38:01 CST 2019
Copyright (c) 2011, 2014, Oracle. All rights reserved.
Welcome to GDSCTL, type "help" for information.
Current GSM is set to GSMORA
GDSCTL>
GDSCTL> connect mygsm/[email protected]:1521/gdsdbp
GDSCTL> exit;
创建 catalog
$ gdsctl
GDSCTL> create catalog -database 192.168.56.31:1521/gdsdbp -user mygsm/mygsm
创建 gsm
增加 gsm,需要事先修改 nodegdsdb 的 /etc/hosts
GDSCTL> add gsm -gsm gsm1 -listener 12000 -catalog 192.168.56.31:1521/gdsdbp
GDSCTL> set gsm -gsm gsm1
GDSCTL> config gsm -gsm gsm1
GDSCTL> start gsm -gsm gsm1
GDSCTL> status gsm
此时查看 nodegsm1 节点的 tnslsnr 信息
$ ps -ef|grep -i tnslsnr |grep -v grep
oracle 8794 1 0 12:12 ? 00:00:00 /u01/app/oracle/product/12.1.0/gsm_1/bin/tnslsnr ifile=/u01/app/oracle/product/12.1.0/gsm_1/network/admin/gsm.ora GSM1 SNLSM:8ec90000 -inherit -mode gsm
$ netstat -lntp
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:6123 0.0.0.0:* LISTEN 8811/ons
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:6234 0.0.0.0:* LISTEN 8811/ons
tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:6011 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:12000 0.0.0.0:* LISTEN 8794/tnslsnr
增加 gds pool,region
GDSCTL>
GDSCTL> add gdspool -gdspool pay
GDSCTL> add region -region shenzhen,guangzhou
GDSCTL> config gdspool
Name Broker
---- ------
dbpoolora No
pay No
GDSCTL> config region
Name Buddy
---- -----
regionora
shenzhen
guangzhou
可看出 gdspool 有默认值 dbpoolora, region 有默认值 regionora.
GDSCTL> modify region -region shenzhen -buddy guangzhou
GDSCTL> modify region -region guangzhou -buddy shenzhen
GDSCTL> config region
Name Buddy
---- -----
regionora
shenzhen guangzhou
guangzhou shenzhen
增加 database/brokerconfig 到 gds pool
在 maa 的 primary 上 unlock 数据库用户 gsmuser
$ sqlplus / as sysdba;
SQL> alter user gsmuser identified by gsmuser account unlock;
User altered.
SQL> set lines 200;
col username format a20;
col account_status format a20;
SQL> select username,account_status from dba_users where username like '%GSM%';
USERNAME ACCOUNT_STATUS
-------------------- --------------------
GSMUSER OPEN
GSMADMIN_INTERNAL EXPIRED & LOCKED
GSMCATUSER EXPIRED & LOCKED
然后在 gdsctl 里添加 database/brokerconfig
使用 dataguard broker 配置的就使用 add brokerconfig，否则就使用 add database
GDSCTL> add brokerconfig -connect testcluster1-scan:1521/orclp -region shenzhen -gdspool pay
"gsmuser" password:
DB Unique Name: orclp
GDSCTL> config database;
Name Pool Status Region
---- ---- ------ ------
orcls1 pay Ok
orcls2 pay Ok
orclp pay Ok shenzhen
GDSCTL> modify database -database orcls1 -region shenzhen -gdspool pay
GDSCTL> modify database -database orcls2 -region guangzhou -gdspool pay
GDSCTL> config database;
Name Pool Status Region
---- ---- ------ ------
orcls1 pay Ok shenzhen
orcls2 pay Ok guangzhou
orclp pay Ok shenzhen
GDSCTL> status database
Database: "orclp" Registered: Y State: Ok ONS: Y. Role: PRIMARY Instances: 1 Region: shenzhen
Registered instances:
pay%1
Database: "orcls1" Registered: Y State: Ok ONS: Y. Role: PH_STNDBY Instances: 1 Region: shenzhen
Registered instances:
pay%11
Database: "orcls2" Registered: Y State: Ok ONS: Y. Role: PH_STNDBY Instances: 1 Region: guangzhou
Registered instances:
pay%21
gdspool 创建 service
GDSCTL> help add service
GDSCTL> add service -gdspool pay -service srv_pay_workload_pdb1 -preferred_all -role PRIMARY -notification TRUE -pdbname orclpdb
GDSCTL> config service
GDSCTL> start service -service srv_pay_workload_pdb1 -gdspool pay
client tns
rac_withdg_workload=
(DESCRIPTION=
(FAILOVER=ON)
(ADDRESS_LIST=
(LOAD_BALANCE=ON)
(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.56.32)(PORT=1521))
)
(CONNECT_DATA=
(SERVER = DEDICATED)
(SERVICE_NAME=srv_pay_workload_pdb1.pay.oradbcloud)
(REGION=shenzhen)
)
)
一些错误
ORA-12154: TNS:could not resolve the connect identifier specified (ngsmoci_connect)
然后在 gdsctl 里添加 database/brokerconfig
使用 dataguard broker 配置的就使用 add brokerconfig，否则就使用 add database
GDSCTL> add brokerconfig -connect testcluster1-scan:1521/orclp -region shenzhen -gdspool pay
"gsmuser" password:
DB Unique Name: orclp
GSM Warnings:
orcls1:NET-40052: Database connection is lost
orcls2:ORA-12154: TNS:could not resolve the connect identifier specified (ngsmoci_connect)
貌似在 DB Unique Name = orclp 上执行成功了，但是在 DB Unique Name = orcls1,orcls2 上出错了。
再次执行
GDSCTL> add brokerconfig -connect testcluster1-scan:1521/orclp -region shenzhen -gdspool pay
"gsmuser" password:
DB Unique Name: orclp
GSM-45029: SQL error
ORA-44881: GDS pool is not empty
ORA-06512: at "GSMADMIN_INTERNAL.DBMS_GSM_POOLADMIN", line 1055
ORA-06512: at line 1
此时需要执行下 remove 操作
GDSCTL> remove brokerconfig -gdspool pay
移除后,在MOS 查找后发现，是 v$dg_broker_config 的 tns串需要在所有节点的 tnsnames.ora 都配置上。
再次执行成功,
GDSCTL> add brokerconfig -connect testcluster1-scan:1521/orclp -region shenzhen -gdspool pay
"gsmuser" password:
DB Unique Name: orclp
GDSCTL> config database;
Name Pool Status Region
---- ---- ------ ------
orcls1 pay Ok
orcls2 pay Ok
orclp pay Ok shenzhen
GDSCTL> modify database -database orcls1 -region shenzhen -gdspool pay
GDSCTL> modify database -database orcls2 -region guangzhou -gdspool pay
GDSCTL> config database;
Name Pool Status Region
---- ---- ------ ------
orcls1 pay Ok shenzhen
orcls2 pay Ok guangzhou
orclp pay Ok shenzhen
GDSCTL> status database
Database: "orclp" Registered: Y State: Ok ONS: Y. Role: PRIMARY Instances: 1 Region: shenzhen
Registered instances:
pay%1
Database: "orcls1" Registered: Y State: Ok ONS: Y. Role: PH_STNDBY Instances: 1 Region: shenzhen
Registered instances:
pay%11
Database: "orcls2" Registered: Y State: Ok ONS: Y. Role: PH_STNDBY Instances: 1 Region: guangzhou
Registered instances:
pay%21
Current user “grid” is not the oracle owner user “oracle” of oracle home
GDSCTL>add service -gdspool pay -service srv_pay_workload_pdb1 -preferred_all -role PRIMARY -notification TRUE -pdbname orclpdb
GSM Errors:
NET-40063: Failed to create global service. Transaction rollbacked
orclp:ORA-45540: Call to SRVCTL failed with status 1, errors: PRCD-1288 : User is not authorized to create service srv_pay_workload_pdb1 for database orclp
PRKH-1014 : Current user "grid" is not the oracle owner user "oracle" of oracle home "/u01/app/oracle/product/12.1.0/db_1"
.
ORA-06512: at "GSMADMIN_INTERNAL.DBMS_GSM_DBADMIN", line 1371
ORA-06512: at line 1 (ngsmoci_execute)
orcls1:ORA-45540: Call to SRVCTL failed with status 1, errors: PRCD-1288 : User is not authorized to create service srv_pay_workload_pdb1 for database orcls1
PRKH-1014 : Current user "grid" is not the oracle owner user "oracle" of oracle home "/u01/app/oracle/product/12.1.0/db_1"
.
ORA-06512: at "GSMADMIN_INTERNAL.DBMS_GSM_DBADMIN", line 1371
ORA-06512: at line 1 (ngsmoci_execute)
上面这个错误参考 (Doc ID 2009820.1)
参考:
<
https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=543941216010749&id=1617946.1&_afrWindowMode=0&_adf.ctrl-state=xu5b1ncuh_4
<
https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=550950190401110&parent=DOCUMENT&sourceId=1617946.1&id=2123709.1&_afrWindowMode=0&_adf.ctrl-state=wn8ajh422_53
<
https://www.oracle.com/technetwork/database/availability/client-failover-2280805.pdf
<
https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=99561832954567&id=1593712.1&_afrWindowMode=0&_adf.ctrl-state=extiidnlx_4
<
https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=25283628713118&parent=DOCUMENT&sourceId=1593712.1&id=1617163.1&_afrWindowMode=0&_adf.ctrl-state=6t53m6k09_295
<
https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=25390052547282&parent=DOCUMENT&sourceId=1617163.1&id=1593786.1&_afrWindowMode=0&_adf.ctrl-state=6t53m6k09_352
<
https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=25575593038720&parent=DOCUMENT&sourceId=1593786.1&id=759208.1&_afrWindowMode=0&_adf.ctrl-state=6t53m6k09_401
<>
https://docs.oracle.com/database/121/HABPT/toc.htm
<
https://www.oracle.com/technetwork/database/features/availability/ha-casestudies-098033.html