oracle11gRAC升级到oracleRAC19C
升级oracle11g集群到19.3
前述环境检查：
[root@qhdb1 ~]# crsctl status res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA1.dg
ONLINE ONLINE qhdb1
ONLINE ONLINE qhdb2
ora.LISTENER.lsnr
ONLINE ONLINE qhdb1
ONLINE ONLINE qhdb2
ora.OCR.dg
ONLINE ONLINE qhdb1
ONLINE ONLINE qhdb2
ora.asm
ONLINE ONLINE qhdb1 Started
ONLINE ONLINE qhdb2 Started
ora.gsd
OFFLINE OFFLINE qhdb1
OFFLINE OFFLINE qhdb2
ora.net1.network
ONLINE ONLINE qhdb1
ONLINE ONLINE qhdb2
ora.ons
ONLINE ONLINE qhdb1
ONLINE ONLINE qhdb2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE qhdb1
ora.andb.db
1 ONLINE ONLINE qhdb1 Open
2 ONLINE ONLINE qhdb2 Open
ora.cvu
1 ONLINE ONLINE qhdb1
ora.oc4j
1 ONLINE OFFLINE
ora.qhdb1.vip
1 ONLINE ONLINE qhdb1
ora.qhdb2.vip
1 ONLINE ONLINE qhdb2
ora.scan1.vip
1 ONLINE ONLINE qhdb1
-------------------------------------------------------------------------------
CRS版本检查：
[root@qhdb1 ~]# crsctl query crs releaseversion
Oracle High Availability Services release version on the local node is [11.2.0.4.0]
[root@qhdb1 ~]# crsctl query crs softwareversion
Oracle Clusterware version on node [qhdb1] is [11.2.0.4.0]
You have mail in /var/spool/mail/root
[root@qhdb1 ~]# crsctl query crs softwareversion -all
Oracle Clusterware version on node [qhdb2] is [11.2.0.4.0]
Oracle Clusterware version on node [qhdb1] is [11.2.0.4.0]
[root@qhdb1 ~]# crsctl check cluster -all
**************************************************************
qhdb1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
qhdb2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
ASM兼容性检查：
SQL> set lines 300
SQL> col name for a30
col COMPATIBILITY for a30
col DATABASE_COMPATIBILITY for a40
select GROUP_NUMBER,NAME,TYPE,COMPATIBILITY,DATABASE_COMPATIBILITY from v$asm_diskgroup;
SQL> SQL> SQL>
GROUP_NUMBER NAME TYPE COMPATIBILITY DATABASE_COMPATIBILITY
------------ ------------------------------ ------ ------------------------------ ----------------------------------------
1 OCR NORMAL 11.2.0.0.0 10.1.0.0.0
2 DATA1 EXTERN 11.2.0.0.0 10.1.0.0.0
设置ASM兼容性：
alter diskgroup DATA1 set attribute 'compatible.rdbms'='x';
alter diskgroup DATA1 set attribute 'compatible.asm'='x';
补丁应用情况检查：
SQL> set lines 300
SQL> col COMP_NAME for a50
SQL> select COMP_ID,comp_name,version,status from dba_registry order by 4;
COMP_ID COMP_NAME VERSION STATUS
------------------------------ -------------------------------------------------- ------------------------------ --------------------------------------------
OWB OWB 11.2.0.4.0 VALID
RAC Oracle Real Application Clusters 11.2.0.4.0 VALID
EM Oracle Enterprise Manager 11.2.0.4.0 VALID
AMD OLAP Catalog 11.2.0.4.0 VALID
SDO Spatial 11.2.0.4.0 VALID
ORDIM Oracle Multimedia 11.2.0.4.0 VALID
XDB Oracle XML Database 11.2.0.4.0 VALID
CONTEXT Oracle Text 11.2.0.4.0 VALID
EXF Oracle Expression Filter 11.2.0.4.0 VALID
RUL Oracle Rules Manager 11.2.0.4.0 VALID
OWM Oracle Workspace Manager 11.2.0.4.0 VALID
COMP_ID COMP_NAME VERSION STATUS
------------------------------ -------------------------------------------------- ------------------------------ --------------------------------------------
CATALOG Oracle Database Catalog Views 11.2.0.4.0 VALID
CATPROC Oracle Database Packages and Types 11.2.0.4.0 VALID
JAVAVM JServer JAVA Virtual Machine 11.2.0.4.0 VALID
XML Oracle XDK 11.2.0.4.0 VALID
CATJAVA Oracle Database Java Packages 11.2.0.4.0 VALID
APS OLAP Analytic Workspace 11.2.0.4.0 VALID
XOQ Oracle OLAP API 11.2.0.4.0 VALID
APEX Oracle Application Express 3.2.1.00.12 VALID
19 rows selected.
SQL>
SQL> col ACTION_TIME for a30
SQL> col ACTION for a15
SQL> col NAMESPACE for a15
SQL> col VERSION for a25
SQL> col COMMENTS for a60
SQL> col BUNDLE_SERIES for a15
set lines 400
select * from dba_registry_history order by 1;
SQL> SQL>
ACTION_TIME ACTION NAMESPACE VERSION ID BUNDLE_SERIES COMMENTS
------------------------------ --------------- --------------- ------------------------- ---------- --------------- ------------------------------------------------------------
24-AUG-13 12.03.45.119862 PM APPLY SERVER 11.2.0.4 0 PSU Patchset 11.2.0.2.0
10-AUG-21 01.45.44.791636 PM APPLY SERVER 11.2.0.4 0 PSU Patchset 11.2.0.2.0
10-AUG-21 03.18.19.828867 PM APPLY SERVER 11.2.0.4 190416 PSU PSU 11.2.0.4.190416
[grid@qhdb1 ~]$ opatch lsinv
Oracle Interim Patch Installer version 11.2.0.3.21
Copyright (c) 2021, Oracle Corporation. All rights reserved.
Oracle Home : /u01/app/11.2.4/grid
Central Inventory : /u01/app/oraInventory
from : /u01/app/11.2.4/grid/oraInst.loc
OPatch version : 11.2.0.3.21
OUI version : 11.2.0.4.0
Log file location : /u01/app/11.2.4/grid/cfgtoollogs/opatch/opatch2021-08-11_10-30-33AM_1.log
Lsinventory Output file location : /u01/app/11.2.4/grid/cfgtoollogs/opatch/lsinv/lsinventory2021-08-11_10-30-33AM.txt
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: qhdb1.zpepc.com.cn
ARU platform id: 226
ARU platform description:: Linux x86-64
Installed Top-level Products (1):
Oracle Grid Infrastructure 11g 11.2.0.4.0
There are 1 products installed in this Oracle Home.
Interim patches (3) :
Patch 28729245 : applied on Tue Aug 10 14:34:05 CST 2021
Unique Patch ID: 22724549
Patch description: "ACFS Patch Set Update : 11.2.0.4.190115 (28729245)"
Created on 2 Nov 2018, 04:47:07 hrs PST8PDT
Bugs fixed:
28069955, 26247490, 21369858, 16318126, 19690653, 17503605, 17203009
17359415, 20140148, 17611362, 25098392, 17270418, 17164243, 19053182
17696547, 17488768, 25078431, 18168684, 21519796, 18143006, 21208140
23321114, 17428148, 17070158, 20438706, 17510275, 17172303, 28142134
18610307, 17376318, 17721778, 22198405, 17699423, 22810422, 18915417
21233961, 18155334, 25381434, 19919907, 18321597, 18185024, 17363999
17636008, 27593263, 20681968, 17475946, 27463879
Patch 29141056 : applied on Tue Aug 10 14:33:07 CST 2021
Unique Patch ID: 22768427
Patch description: "Database Patch Set Update : 11.2.0.4.190416 (29141056)"
Created on 17 Feb 2019, 23:52:26 hrs PST8PDT
Sub-patch 28729262; "Database Patch Set Update : 11.2.0.4.190115 (28729262)"
1.配置19 软件安装目录
grid:节点qhdb1
umask 022
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/11.2.4/grid
####19c ORACLE_HOME
##export ORACLE_HOME=/u01/app/19.0.0/grid
export ORACLE_SID=+ASM2
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:$ORACLE_HOME/rdbms/lib
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:/usr/bin:/usr/sbin:/usr/local/bin:$PATH
grid:节点qhdb2
umask 022
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/11.2.4/grid
####19c ORACLE_HOME
##export ORACLE_HOME=/u01/app/19.0.0/grid
export ORACLE_SID=+ASM1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:$ORACLE_HOME/rdbms/lib
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:/usr/bin:/usr/sbin:/usr/local/bin:$PATH
oracle：节点qhdb1
umask 022
export ORACLE_BASE=/u01/app/oracle
export ORACLE_SID=andb2
export ORACLE_HOME=/u01/app/oracle/product/11.2.4/db_home1
#####19c oracle_home
##export ORACLE_HOME=/u01/app/oracle/product/19.0.0/db_home1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:$ORACLE_HOME/rdbms/lib
export PATH=$ORACLE_HOME/bin:/u01/app/11.2.4/grid/bin:/usr/bin:/usr/sbin:/usr/local/bin:$PATH
oracle：节点qhdb2
umask 022
export ORACLE_BASE=/u01/app/oracle
export ORACLE_SID=andb1
export ORACLE_HOME=/u01/app/oracle/product/11.2.4/db_home1
#####19c oracle_home
##export ORACLE_HOME=/u01/app/oracle/product/19.0.0/db_home1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:$ORACLE_HOME/rdbms/lib
export PATH=$ORACLE_HOME/bin:/u01/app/11.2.4/grid/bin:/usr/bin:/usr/sbin:/usr/local/bin:$PATH
2.拷贝软件到ORACLE_HOME
grid:
cp LINUX.X64_193000_grid_home.zip /u01/app/19.0.0/grid/
oracle:
cp LINUX.X64_193000_db_home.zip /u01/app/oracle/product/19.0.0/db_home1/
修改软件包权限，并解压：
grid:
[grid@qhdb1 grid]$ logout
You have mail in /var/spool/mail/root
[root@qhdb1 app]# cd /u01/app/19.0.0/grid
[root@qhdb1 grid]# ls -lrt
total 2821472
-rw-r--r-- 1 root root 2889184573 Aug 11 10:16 LINUX.X64_193000_grid_home.zip
[root@qhdb1 grid]#
[root@qhdb1 grid]# chown -R grid:oinstall LINUX.X64_193000_grid_home.zip
[root@qhdb1 grid]# su - grid
Last login: Wed Aug 11 10:30:28 CST 2021 on pts/0
[grid@qhdb1 ~]$ cd /u01/app/19.0.0/grid
[grid@qhdb1 grid]$ ls -lrt
total 2821472
-rw-r--r-- 1 grid oinstall 2889184573 Aug 11 10:16 LINUX.X64_193000_grid_home.zip
[grid@qhdb1 grid]$ unzip LINUX.X64_193000_grid_home.zip
oracle:
[oracle@qhdb1 ~]$ cd /u01/app/oracle/product/19.0.0/db_home1
[oracle@qhdb1 db_home1]$ ls -lrt
total 2987996
-rw-r--r-- 1 root root 3059705302 Aug 11 10:22 LINUX.X64_193000_db_home.zip
[oracle@qhdb1 db_home1]$
[oracle@qhdb1 db_home1]$
[oracle@qhdb1 db_home1]$ logout
[root@qhdb1 ~]# cd /u01/app/oracle/product/19.0.0/db_home1
[root@qhdb1 db_home1]# chown -R oracle:oinstall LINUX.X64_193000_db_home.zip
[root@qhdb1 db_home1]# su - oracle
Last login: Wed Aug 11 10:32:29 CST 2021 on pts/3
[oracle@qhdb1 ~]$ cd /u01/app/oracle/product/19.0.0/db_home1
[oracle@qhdb1 db_home1]$ ls -lrt
total 2987996
-rw-r--r-- 1 oracle oinstall 3059705302 Aug 11 10:22 LINUX.X64_193000_db_home.zip
[oracle@qhdb1 db_home1]$
[oracle@qhdb1 db_home1]$
[oracle@qhdb1 db_home1]$ unzip LINUX.X64_193000_db_home.zip
3.调出VNC，准备升级CRS
[grid@qhdb1 grid]$ export DISPLAY=:1
[grid@qhdb1 grid]$ pwd
/u01/app/19.0.0/grid
[grid@qhdb1 grid]$ ./gridSetup.sh
1.选择升级GI-
2.直接点击下一步--若互信有问题则会提示、重新配置互信
3.点击下一步
4.设置ORACLE_BASE 和ORACLE_HOME目录，这里ORACLE_HOME目录是灰色不让选择的，但是BASE目录是可以选择的
5.升级脚本是自动运行还是手工运行
手工运行uproot.sh脚本就不要选择
要是自动运行升级脚本，则打勾勾，输入root密码
6.需要注意：
尝试忽略，提示不能够忽略，所以，接下来 我们需要逐个问题结局：
1.提示需要安装补丁：28553832
2.设置ASM diskgroup compatibility 兼容性参数设置；
3.time oggset between nodes ：两个节点时间差太大，需要设置NTP服务；
一：首先解决：NTP
配置NTP服务器：
二：查看ASM磁盘组兼容性参数，并修改：
三：补丁更新：28553832--
注意：我这里直接下载了11.2.0.4的最新补丁包：31718723
接下来就可以忽略升级GI
接下来就是升级GI
执行升级脚本：
节点一执行升级脚本：
[root@qhdb1 ~]# /u01/app/19.0.0/grid/rootupgrade.sh
Performing root user operation.
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/19.0.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying dbhome to /usr/local/bin ...
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]: y
Copying coraenv to /usr/local/bin ...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/19.0.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/grid/crsdata/qhdb1/crsconfig/rootcrs_qhdb1_2021-08-16_08-15-33AM.log
2021/08/16 08:15:49 CLSRSC-595: Executing upgrade step 1 of 18: 'UpgradeTFA'.
2021/08/16 08:15:49 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 08:15:49 CLSRSC-595: Executing upgrade step 2 of 18: 'ValidateEnv'.
2021/08/16 08:15:53 CLSRSC-595: Executing upgrade step 3 of 18: 'GetOldConfig'.
2021/08/16 08:15:54 CLSRSC-464: Starting retrieval of the cluster configuration data
2021/08/16 08:16:01 CLSRSC-692: Checking whether CRS entities are ready for upgrade. This operation may take a few minutes.
2021/08/16 08:18:50 CLSRSC-4003: Successfully patched Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 08:18:50 CLSRSC-693: CRS entities validation completed successfully.
2021/08/16 08:18:54 CLSRSC-515: Starting OCR manual backup.
2021/08/16 08:18:59 CLSRSC-516: OCR manual backup successful.
2021/08/16 08:21:21 CLSRSC-486:
At this stage of upgrade, the OCR has changed.
Any attempt to downgrade the cluster after this point will require a complete cluster outage to restore the OCR.
2021/08/16 08:21:21 CLSRSC-541:
To downgrade the cluster:
1. All nodes that have been upgraded must be downgraded.
2021/08/16 08:21:21 CLSRSC-542:
2. Before downgrading the last node, the Grid Infrastructure stack on all other cluster nodes must be down.
2021/08/16 08:21:27 CLSRSC-465: Retrieval of the cluster configuration data has successfully completed.
2021/08/16 08:21:27 CLSRSC-595: Executing upgrade step 4 of 18: 'GenSiteGUIDs'.
2021/08/16 08:21:28 CLSRSC-595: Executing upgrade step 5 of 18: 'UpgPrechecks'.
2021/08/16 08:21:32 CLSRSC-363: User ignored prerequisites during installation
2021/08/16 08:21:38 CLSRSC-595: Executing upgrade step 6 of 18: 'SetupOSD'.
2021/08/16 08:21:38 CLSRSC-595: Executing upgrade step 7 of 18: 'PreUpgrade'.
2021/08/16 08:22:03 CLSRSC-468: Setting Oracle Clusterware and ASM to rolling migration mode
2021/08/16 08:22:03 CLSRSC-482: Running command: '/u01/app/19.0.0/grid/bin/asmca -silent -upgradeNodeASM -nonRolling false -oldCRSHome /u01/app/11.2.4/grid -oldCRSVersion 11.2.0.4.0 -firstNode true -startRolling true '
ASM configuration upgraded in local node successfully.
2021/08/16 08:22:10 CLSRSC-469: Successfully set Oracle Clusterware and ASM to rolling migration mode
2021/08/16 08:25:52 CLSRSC-466: Starting shutdown of the current Oracle Grid Infrastructure stack
2021/08/16 08:26:19 CLSRSC-467: Shutdown of the current Oracle Grid Infrastructure stack has successfully completed.
2021/08/16 08:26:24 CLSRSC-595: Executing upgrade step 8 of 18: 'CheckCRSConfig'.
2021/08/16 08:26:24 CLSRSC-595: Executing upgrade step 9 of 18: 'UpgradeOLR'.
2021/08/16 08:26:35 CLSRSC-595: Executing upgrade step 10 of 18: 'ConfigCHMOS'.
2021/08/16 08:26:36 CLSRSC-595: Executing upgrade step 11 of 18: 'UpgradeAFD'.
2021/08/16 08:26:42 CLSRSC-595: Executing upgrade step 12 of 18: 'createOHASD'.
2021/08/16 08:26:49 CLSRSC-595: Executing upgrade step 13 of 18: 'ConfigOHASD'.
2021/08/16 08:26:49 CLSRSC-329: Replacing Clusterware entries in file 'oracle-ohasd.service'
2021/08/16 08:27:51 CLSRSC-595: Executing upgrade step 14 of 18: 'InstallACFS'.
CRS-41053: checking Oracle Grid Infrastructure for file permission issues
PRVH-0116 : Path "/u01/app/19.0.0/grid/crs/install/cmdllroot.sh" with permissions "rw-r--r--" does not have execute permissions for the owner, file's group, and others on node "qhdb1".
PRVG-2031 : Owner of file "/u01/app/19.0.0/grid/crs/install/cmdllroot.sh" did not match the expected value on node "qhdb1". [Expected = "grid(1002)" ; Found = "root(0)"]
PRVG-2032 : Group of file "/u01/app/19.0.0/grid/crs/install/cmdllroot.sh" did not match the expected value on node "qhdb1". [Expected = "oinstall(1006)" ; Found = "root(0)"]
CRS-4124: Oracle High Availability Services startup failed.
CRS-4000: Command Start failed, or completed with errors.
2021/08/16 08:30:00 CLSRSC-117: Failed to start Oracle Clusterware stack
Died at /u01/app/19.0.0/grid/crs/install/crsutils.pm line 11714.
You have mail in /var/spool/mail/root
[root@qhdb1 ~]#
升级报错： 权限不足
检查："/u01/app/19.0.0/grid/crs/install/cmdllroot.sh权限
修改权限：chown -R grid:oinstall * ; chmod -R 775 *
看到这个目录有些文件权限是root:root ,所以 我们直接修改。----------------------------问题OK！
重新执行升级脚本：
[root@qhdb1 ~]# /u01/app/19.0.0/grid/rootupgrade.sh
Performing root user operation.
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/19.0.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/19.0.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/grid/crsdata/qhdb1/crsconfig/rootcrs_qhdb1_2021-08-16_09-15-01AM.log
2021/08/16 09:15:12 CLSRSC-595: Executing upgrade step 1 of 18: 'UpgradeTFA'.
2021/08/16 09:15:12 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:15:12 CLSRSC-4012: Shutting down Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:16:52 CLSRSC-4013: Successfully shut down Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:16:52 CLSRSC-595: Executing upgrade step 2 of 18: 'ValidateEnv'.
2021/08/16 09:16:53 CLSRSC-595: Executing upgrade step 3 of 18: 'GetOldConfig'.
2021/08/16 09:16:53 CLSRSC-464: Starting retrieval of the cluster configuration data
2021/08/16 09:16:57 CLSRSC-692: Checking whether CRS entities are ready for upgrade. This operation may take a few minutes.
2021/08/16 09:17:00 CLSRSC-693: CRS entities validation completed successfully.
2021/08/16 09:17:04 CLSRSC-4003: Successfully patched Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:17:06 CLSRSC-465: Retrieval of the cluster configuration data has successfully completed.
2021/08/16 09:17:06 CLSRSC-595: Executing upgrade step 4 of 18: 'GenSiteGUIDs'.
2021/08/16 09:17:07 CLSRSC-595: Executing upgrade step 5 of 18: 'UpgPrechecks'.
2021/08/16 09:17:07 CLSRSC-595: Executing upgrade step 6 of 18: 'SetupOSD'.
2021/08/16 09:17:07 CLSRSC-595: Executing upgrade step 7 of 18: 'PreUpgrade'.
2021/08/16 09:17:07 CLSRSC-595: Executing upgrade step 8 of 18: 'CheckCRSConfig'.
2021/08/16 09:17:08 CLSRSC-595: Executing upgrade step 9 of 18: 'UpgradeOLR'.
2021/08/16 09:17:09 CLSRSC-595: Executing upgrade step 10 of 18: 'ConfigCHMOS'.
2021/08/16 09:17:44 CLSRSC-595: Executing upgrade step 11 of 18: 'UpgradeAFD'.
2021/08/16 09:17:45 CLSRSC-595: Executing upgrade step 12 of 18: 'createOHASD'.
2021/08/16 09:17:46 CLSRSC-595: Executing upgrade step 13 of 18: 'ConfigOHASD'.
CSKM-1110: error while generating cluster root certificate
CRS-10403: (:CLSCRED0102:)Could not connect to credential storage: PROCL-32: Oracle High Availability Services on the local node is not running Messaging error [gipcretConnectionRefused] [29].
CRS-4000: Command Delete failed, or completed with errors.
问题：执行升级脚本失败，分析问题原因：提示HAS not running！！！
由于我们Linux版本是7，原数据库版本是11g，由于11g和Linux7版本的systemd进程服务不相匹配，
所以我们在Linux7版本上安装oracle11gRAC,需要手工创建ohas.service
以下是Linux上安装11g，需要配置的OHAS服务。
启动服务：
systemctl daemon-reload
systemctl enable ohas.service
systemctl start ohas.service
systemctl status ohas.service
/usr/lib/systemd/system/ohas.service
处理方式：ps -fle | grep d.bin 然后kill
关闭ohas服务：
systemctl disable ohas.service
systemctl stop ohas.service
然后在重新执行升级脚本：
[root@qhdb1 ~]# /u01/app/19.0.0/grid/rootupgrade.sh
Performing root user operation.
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/19.0.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/19.0.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/grid/crsdata/qhdb1/crsconfig/rootcrs_qhdb1_2021-08-16_09-40-46AM.log
2021/08/16 09:40:57 CLSRSC-595: Executing upgrade step 1 of 18: 'UpgradeTFA'.
2021/08/16 09:40:57 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:40:57 CLSRSC-4012: Shutting down Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:42:37 CLSRSC-4013: Successfully shut down Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:42:37 CLSRSC-595: Executing upgrade step 2 of 18: 'ValidateEnv'.
2021/08/16 09:42:38 CLSRSC-595: Executing upgrade step 3 of 18: 'GetOldConfig'.
2021/08/16 09:42:38 CLSRSC-464: Starting retrieval of the cluster configuration data
2021/08/16 09:42:43 CLSRSC-692: Checking whether CRS entities are ready for upgrade. This operation may take a few minutes.
2021/08/16 09:42:50 CLSRSC-4003: Successfully patched Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 09:45:40 CLSRSC-693: CRS entities validation completed successfully.
2021/08/16 09:45:46 CLSRSC-465: Retrieval of the cluster configuration data has successfully completed.
2021/08/16 09:45:46 CLSRSC-595: Executing upgrade step 4 of 18: 'GenSiteGUIDs'.
2021/08/16 09:45:47 CLSRSC-595: Executing upgrade step 5 of 18: 'UpgPrechecks'.
2021/08/16 09:45:47 CLSRSC-595: Executing upgrade step 6 of 18: 'SetupOSD'.
2021/08/16 09:45:47 CLSRSC-595: Executing upgrade step 7 of 18: 'PreUpgrade'.
2021/08/16 09:45:47 CLSRSC-595: Executing upgrade step 8 of 18: 'CheckCRSConfig'.
2021/08/16 09:45:48 CLSRSC-595: Executing upgrade step 9 of 18: 'UpgradeOLR'.
2021/08/16 09:45:49 CLSRSC-595: Executing upgrade step 10 of 18: 'ConfigCHMOS'.
2021/08/16 09:45:49 CLSRSC-595: Executing upgrade step 11 of 18: 'UpgradeAFD'.
2021/08/16 09:45:50 CLSRSC-595: Executing upgrade step 12 of 18: 'createOHASD'.
2021/08/16 09:45:51 CLSRSC-595: Executing upgrade step 13 of 18: 'ConfigOHASD'.
2021/08/16 09:45:54 CLSRSC-595: Executing upgrade step 14 of 18: 'InstallACFS'.
2021/08/16 09:48:17 CLSRSC-595: Executing upgrade step 15 of 18: 'InstallKA'.
2021/08/16 09:48:24 CLSRSC-595: Executing upgrade step 16 of 18: 'UpgradeCluster'.
clscfg: EXISTING configuration version 5 detected.
Successfully taken the backup of node specific configuration in OCR.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
2021/08/16 10:01:59 CLSRSC-343: Successfully started Oracle Clusterware stack
2021/08/16 10:02:05 CLSRSC-595: Executing upgrade step 17 of 18: 'UpgradeNode'.
2021/08/16 10:02:09 CLSRSC-474: Initiating upgrade of resource types
2021/08/16 10:02:58 CLSRSC-475: Upgrade of resource types successfully initiated.
2021/08/16 10:03:07 CLSRSC-595: Executing upgrade step 18 of 18: 'PostUpgrade'.
2021/08/16 10:03:21 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
看到软件已经升级成功了！
开始升级节点2：/u01/app/19.0.0/grid/rootupgrade.sh
CRS-1115: Oracle Clusterware has already been upgraded.
2021/08/16 10:36:54 CLSRSC-479: Successfully set Oracle Clusterware active version
2021/08/16 10:37:02 CLSRSC-476: Finishing upgrade of resource types
PRCR-1079 : Failed to start resource ora.qosmserver
TCC-0004: The container was not able to start.
Failed to start connector [Connector[HTTP/1.1-8888]]Failed to initialize component [Connector[HTTP/1.1-8888]]
Protocol handler initialization failed
Address already in use
CRS-2674: Start of 'ora.qosmserver' on 'qhdb2' failed
TCC-0004: The container was not able to start.
Failed to start connector [Connector[HTTP/1.1-8888]]Failed to initialize component [Connector[HTTP/1.1-8888]]
Protocol handler initialization failed
Address already in use
CRS-2674: Start of 'ora.qosmserver' on 'qhdb1' failed
CRS-2632: There are no more servers to try to place resource 'ora.qosmserver' on that would satisfy its placement policy
2021/08/16 10:37:37 CLSRSC-180: An error occurred while executing the command '/u01/app/19.0.0/grid/bin/srvctl upgrade model -s 11.2.0.4.0 -d 19.0.0.0.0 -p last'
2021/08/16 10:37:37 CLSRSC-284: Failed to perform last node tasks for cluster modeling upgrade
Died at /u01/app/19.0.0/grid/crs/install/orasrvm.pm line 570.
解决方式：
[root@qhdb2 ~]# vi /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
#::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 注释掉
1.查看lo的IPV6地址：
删除 ：
ip addr del ::1/128 dev lo
netstat -tunlp | grep ':8888'
ps -fle | grep d.bin ---KILL
ps -fle | grep grid ---KILL
然后重新运行rootupgrade.sh 升级脚本
=============================================
[root@qhdb2 app]# /u01/app/19.0.0/grid/rootupgrade.sh
Performing root user operation.
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/19.0.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/19.0.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/grid/crsdata/qhdb2/crsconfig/rootcrs_qhdb2_2021-08-16_02-21-22PM.log
2021/08/16 14:21:31 CLSRSC-595: Executing upgrade step 1 of 18: 'UpgradeTFA'.
2021/08/16 14:21:31 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 14:21:31 CLSRSC-4012: Shutting down Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 14:23:11 CLSRSC-4013: Successfully shut down Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 14:23:11 CLSRSC-595: Executing upgrade step 2 of 18: 'ValidateEnv'.
2021/08/16 14:23:12 CLSRSC-595: Executing upgrade step 3 of 18: 'GetOldConfig'.
2021/08/16 14:23:12 CLSRSC-464: Starting retrieval of the cluster configuration data
2021/08/16 14:23:24 CLSRSC-4003: Successfully patched Oracle Trace File Analyzer (TFA) Collector.
2021/08/16 14:23:48 CLSRSC-465: Retrieval of the cluster configuration data has successfully completed.
2021/08/16 14:23:49 CLSRSC-595: Executing upgrade step 4 of 18: 'GenSiteGUIDs'.
2021/08/16 14:23:49 CLSRSC-595: Executing upgrade step 5 of 18: 'UpgPrechecks'.
2021/08/16 14:23:49 CLSRSC-595: Executing upgrade step 6 of 18: 'SetupOSD'.
2021/08/16 14:23:49 CLSRSC-595: Executing upgrade step 7 of 18: 'PreUpgrade'.
2021/08/16 14:23:49 CLSRSC-595: Executing upgrade step 8 of 18: 'CheckCRSConfig'.
2021/08/16 14:23:50 CLSRSC-595: Executing upgrade step 9 of 18: 'UpgradeOLR'.
2021/08/16 14:23:51 CLSRSC-595: Executing upgrade step 10 of 18: 'ConfigCHMOS'.
2021/08/16 14:24:22 CLSRSC-595: Executing upgrade step 11 of 18: 'UpgradeAFD'.
2021/08/16 14:24:23 CLSRSC-595: Executing upgrade step 12 of 18: 'createOHASD'.
2021/08/16 14:24:24 CLSRSC-595: Executing upgrade step 13 of 18: 'ConfigOHASD'.
2021/08/16 14:24:24 CLSRSC-329: Replacing Clusterware entries in file 'oracle-ohasd.service'
2021/08/16 14:24:38 CLSRSC-595: Executing upgrade step 14 of 18: 'InstallACFS'.
2021/08/16 14:24:40 CLSRSC-595: Executing upgrade step 15 of 18: 'InstallKA'.
2021/08/16 14:24:41 CLSRSC-595: Executing upgrade step 16 of 18: 'UpgradeCluster'.
2021/08/16 14:25:47 CLSRSC-343: Successfully started Oracle Clusterware stack
2021/08/16 14:25:53 CLSRSC-595: Executing upgrade step 17 of 18: 'UpgradeNode'.
CRS-1115: Oracle Clusterware has already been upgraded.
2021/08/16 14:25:58 CLSRSC-478: Setting Oracle Clusterware active version on the last node to be upgraded
2021/08/16 14:25:58 CLSRSC-482: Running command: '/u01/app/19.0.0/grid/bin/crsctl set crs activeversion'
CRS-1115: Oracle Clusterware has already been upgraded.
2021/08/16 14:26:58 CLSRSC-479: Successfully set Oracle Clusterware active version
2021/08/16 14:27:06 CLSRSC-476: Finishing upgrade of resource types
2021/08/16 14:27:45 CLSRSC-477: Successfully completed upgrade of resource types
2021/08/16 14:28:43 CLSRSC-595: Executing upgrade step 18 of 18: 'PostUpgrade'.
Successfully updated XAG resources.
2021/08/16 14:29:13 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
至此GI升级完成！！！DB的升级其实就是重新安装一套数据库软件而已，这个一般来时不会出现问题的。
-----------------------接下来升级数据库软件
节点一 二创建19c数据软件目录
mkdir -pv /u01/app/oracle/product/19.0.0/db_home1
chown -R oracle:oinstall /u01/app/oracle/product/19.0.0
chmod -R 775 /u01/app/oracle/product/19.0.0/db_home1
su - oracle
unzip LINUX.X64_193000_db_home.zip
调出图形化，装备安装19c数据库软件
仅安装数据库软件
集群数据库软件
默认选择NEXT
默认选择NEXT
默认选择NEXT
默认选择NEXT
默认选择NEXT--点击忽略打勾勾
执行完毕后点击OK 软件升级完成！！！
------------------------------接下来升级数据库andb
---拷贝文件11g $ORACLE_HOME/dbs/initandb***.ora 到19c环境下
ORA-00603: ORACLE server session terminated by fatal error
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-39701: database must be mounted EXCLUSIVE for UPGRADE or DOWNGRADE
提示cluster_database 参数需要修改
[oracle@qhdb1 ~]$ sqlplus / as sysdba
SQL*Plus: Release 19.0.0.0.0 - Production on Mon Aug 16 16:42:49 2021
Version 19.3.0.0.0
Copyright (c) 1982, 2019, Oracle. All rights reserved.
Connected to an idle instance.
SQL>
SQL> startup nomount
ORACLE instance started.
Total System Global Area 8589930600 bytes
Fixed Size 12456040 bytes
Variable Size 1593835520 bytes
Database Buffers 6962544640 bytes
Redo Buffers 21094400 bytes
SQL>
SQL> show parameter cluster
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
cdb_cluster boolean FALSE
cdb_cluster_name string
cluster_database boolean TRUE
cluster_database_instances integer 2
cluster_interconnects string
SQL> alter system set cluster_database=false scope=spfile;
System altered.
SQL> shu abort
ORACLE instance shut down.
SQL> startup upgrade
ORACLE instance started.
Total System Global Area 8589930600 bytes
Fixed Size 12456040 bytes
Variable Size 1593835520 bytes
Database Buffers 6962544640 bytes
Redo Buffers 21094400 bytes
Database mounted.
Database opened.
接下来就是升级数据字典了！！！----- @?/rdbms/admin/catupgrd.sql
OK 提示我们按照它提供的方式进行升级！！
cd $ORACLE_HOME/rdbms/admin
$ORACLE_HOME/perl/bin/perl catctl.pl catupgrd.sql
----------------------
cd $ORACLE_HOME/rdbms/admin
$ORACLE_HOME/perl/bin/perl catctl.pl -n 24 catupgrd.sql
-n 24 开并发进行升级！！
[oracle@qhdb1 admin]$ $ORACLE_HOME/perl/bin/perl catctl.pl -n 24 catupgrd.sql
升级后补充运行脚本，要是手动升级数据库，但是未运行脚本catuppst.sql脚本，则数据库的性能随着时间下降。
$ORACLE_HOME/perl/bin/perl $ORACLE_HOME/rdbms/admin/catcon.pl -n 24 -e -b catuppst -d "." catuppst.sql
编译无效对象：
$ORACLE_HOME/perl/bin/perl $ORACLE_HOME/rdbms/admin/catcon.pl -n 24 -e -b utlrp -d "." utlrp.sql
----------------------升级完后检查：
1.timezone_file
2.sqlnet.ora
3.ora-20000权限不足检查
4.创建server_name报错………………等等问题
检查脚本：
@?/rdbms/admin/utltz_upg_check.sql
设置timezone_file脚本：
@?/rdbms/admin/utltz_upg_apply.sql
sqlnet.ora
NAMES.DIRECTORY_PATH= (TNSNAMES, ONAMES, HOSTNAME)
SQLNET.ALLOWED_LOGON_VERSION_CLIENT = 8
SQLNET.ALLOWED_LOGON_VERSION_SERVER = 8
SQLNET.ALLOWED_LOGON_VERSION_CLIENT = 10
SQLNET.ALLOWED_LOGON_VERSION_CLIENT = 11
SQLNET.ALLOWED_LOGON_VERSION_CLIENT = 12
--alter system set resource_manager_plan=default_plan;
grant administer database trigger to SYS ;
grant administer database trigger to system ;
--------最后cluster_database=true。