数据库服务器重启之后,检查Oracle集群资源发现四个实例启动了两个,还有两个没有启动(wu,rl)
[grid@db1 ~]$ crsctl stat res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.ARCH.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.CWDATA.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.DADATA.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.DATA.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.LISTENER.lsnr ONLINE ONLINE db1 ONLINE ONLINE db2 ora.OCR.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.KDATA.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.asm ONLINE ONLINE db1 Started ONLINE ONLINE db2 Started ora.gsd OFFLINE OFFLINE db1 OFFLINE OFFLINE db2 ora.net1.network ONLINE ONLINE db1 ONLINE ONLINE db2 ora.ons ONLINE ONLINE db1 ONLINE ONLINE db2 -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE db1 ora.LISTENER_SCAN2.lsnr 1 ONLINE ONLINE db2 ora.LISTENER_SCAN3.lsnr 1 ONLINE ONLINE db2 ora.wu.db 1 ONLINE OFFLINE Instance Shutdown 2 ONLINE ONLINE db2 Open ora.dyl.db 1 ONLINE OFFLINE 2 ONLINE ONLINE db2 Open ora.cvu 1 ONLINE ONLINE db2 ora.da.db 1 ONLINE ONLINE db1 Open 2 ONLINE ONLINE db2 Open ora.db1.vip 1 ONLINE ONLINE db1 ora.db2.vip 1 ONLINE ONLINE db2 ora.oc4j 1 ONLINE ONLINE db2 ora.rl.db 1 ONLINE OFFLINE Instance Shutdown 2 ONLINE ONLINE db2 Open ora.scan1.vip 1 ONLINE ONLINE db1 ora.scan2.vip 1 ONLINE ONLINE db2 ora.scan3.vip 1 ONLINE ONLINE db2
查看rlzy实例的alert.log文件,可以看到以下错误信息“ORA-00600: internal error code, arguments: [2252], [3418], [573259345], [1594], [50675712]”,关于这个ORA-00600 2252在MOS上有相关bug描述,但我这并不是bug引起的,查看信息时我们也是需要关注时间的,这里时间显示为2001年1月1号了,与当前时间相差了16年2个多月。
Picked broadcast on commit scheme to generate SCNs Mon Jan 01 08:23:50 2001 Errors in file /u01/app/oracle/diag/rdbms/rl/RL1/trace/RL1_dbw0_19789.trc (incident=544328): ORA-00600: internal error code, arguments: [2252], [3418], [573259345], [1594], [50675712], [], [], [], [], [], [], [] Incident details in: /u01/app/oracle/diag/rdbms/rl/RL1/incident/incdir_544216/RL1_diag_19753_i544216.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Errors in file /u01/app/oracle/diag/rdbms/rl/RL1/trace/RL1_dbw0_19789.trc: ORA-01186: file 2 failed verification tests ORA-00600: internal error code, arguments: [2252], [3418], [573259345], [1594], [50675712], [], [], [], [], [], [], [] DBW0 (ospid: 19789): terminating the instance due to error 1186 Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Mon Jan 01 08:23:56 2001 ORA-1092 : opitsk aborting process Mon Jan 01 08:23:56 2001
查看系统当前时间,居然变成了2001年1月1号,而且奇怪的是并不是重启时时间就被修改了,因为有两个实例在重启之后正常启动了
[root@db1 ~]# date Mon Jan 1 08:25:34 CST 2001
手工更新为当前时间后并手动启动实例(caiwu,rlzy),就能正常启动
[root@db1 ~]# date Fri Mar 24 11:26:44 CST 2017 [grid@db1 ~]$ srvctl start database -d caiwu [grid@db1 ~]$ srvctl start database -d rlzy [grid@db1 ~]$ crsctl stat res -t -------------------------------------------------------------------------------- NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.ARCH.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.CWDATA.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.DADATA.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.DATA.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.LISTENER.lsnr ONLINE ONLINE db1 ONLINE ONLINE db2 ora.OCR.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.KDATA.dg ONLINE ONLINE db1 ONLINE ONLINE db2 ora.asm ONLINE ONLINE db1 Started ONLINE ONLINE db2 Started ora.gsd OFFLINE OFFLINE db1 OFFLINE OFFLINE db2 ora.net1.network ONLINE ONLINE db1 ONLINE ONLINE db2 ora.ons ONLINE ONLINE db1 ONLINE ONLINE db2 -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE db1 ora.LISTENER_SCAN2.lsnr 1 ONLINE ONLINE db2 ora.LISTENER_SCAN3.lsnr 1 ONLINE ONLINE db2 ora.wu.db 1 ONLINE ONLINE db1 Open 2 ONLINE ONLINE db2 Open ora.dyl.db 1 ONLINE ONLINE db1 Open 2 ONLINE ONLINE db2 Open ora.cvu 1 ONLINE ONLINE db2 ora.da.db 1 ONLINE ONLINE db1 Open 2 ONLINE ONLINE db2 Open ora.db1.vip 1 ONLINE ONLINE db1 ora.db2.vip 1 ONLINE ONLINE db2 ora.oc4j 1 ONLINE ONLINE db2 ora.rl.db 1 ONLINE ONLINE db1 Open 2 ONLINE ONLINE db2 Open ora.scan1.vip 1 ONLINE ONLINE db1 ora.scan2.vip 1 ONLINE ONLINE db2 ora.scan3.vip 1 ONLINE ONLINE db2
问题虽然解决了,但是为什么服务器会在启动两个实例后,系统时间发生改变还是不得而知,需要找浪潮服务器的工程师来检查。
来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/26015009/viewspace-2135996/,如需转载,请注明出处,否则将追究法律责任。
转载于:http://blog.itpub.net/26015009/viewspace-2135996/