asm冗余 oracle_ORACLE 11G RAC ASM磁盘的三种冗余模式

在安装ASM磁盘时采用了high、normal、extenrnal的三种冗余模式。

三种冗余模式

EXTERNAL模式

External redundancy(1OCR and no copies)

An external redundancy disk group requires a minimum of one disk device.

The effective disk space in an external redundancy disk group is the sum of the disk space in all of its devices.

Because Automatic Storage Management does not mirror data in an external

redundancy disk group, Oracle recommends that you use external redundancy with storage devices such as RAID, or other

similar devices that provide their own data protection mechanisms.

NORMAL模式

Normal redundancy(1 OCR and 2 copies )

In a normal redundancy disk group, to increase performance and reliability,

Automatic Storage Management by default uses two-way mirroring. A normal

redundancy disk group requires a minimum of two disk devices (or two

failure groups). The effective disk space in a normal redundancy disk group is

half the sum of the disk space in all of its devices.

For Oracle Clusterware files, Normal redundancy disk groups provide 3

voting disk files, 1 OCR and 2 copies (one primary and one secondary mirror).

With normal redundancy, the cluster can survive the loss of one failure group.

For most installations, Oracle recommends that you select normal redundancy.

HIGH模式

High redundancy(1 OCR and 3 copies )

In a high redundancy disk group, Automatic Storage Management uses

three-way mirroring to increase performance and provide the highest level of

reliability. A high redundancy disk group requires a minimum of three disk

devices (or three failure groups). The effective disk space in a high redundancy

disk group is one-third the sum of the disk space in all of its devices.

For Oracle Clusterware files, High redundancy disk groups provide 5 voting

disk files, 1 OCR and 3 copies (one primary and two secondary mirrors). With

high redundancy, the cluster can survive the loss of two failure groups.

While high redundancy disk groups do provide a high level of data protection,

you should consider the greater cost of additional storage devices before

deciding to select high redundancy disk groups.

METALINK文档描述

[ID 428681.1]

题目：OCR / Vote disk Maintenance Operations: (ADD/REMOVE/REPLACE/MOVE)

11.2+:

From 11.2, votedisk can be stored on either ASM diskgroup or cluster file systems. The following commands can only be

executed when GI is running either in cluster mode or exclusive mode. As grid user:

1. To add a Voting Disk

a. When votedisk is on cluster file system:

$ crsctl add css votedisk

b. When votedisk is on ASM diskgroup, no add option available. The number of votedisk is determined by the diskgroup

redundancy. If more copy of votedisk is desired, one can move votedisk to a diskgroup with higher redundancy.

2. To delete a Voting Disk

a. When votedisk is on cluster file system:

$ crsctl delete css votedisk

b. When votedisk is on ASM, no delete option available, one can only replace the existing votedisk group with another ASM

diskgroup

3. To move a Voting Disk

a. When votedisk is on cluster file system:

$ crsctl add css votedisk

$ crsctl delete css votedisk

b. When votedisk is on ASM or moving votedisk between cluster file system and ASM diskgroup

$ crsctl replace votedisk |

eg:

move to cluster file system:

$ crsctl replace votedisk /shared/vote.dat

Now formatting voting disk: /shared/vote.dat.

CRS-4256: Updating the profile

Successful addition of voting disk 32ff90ab38a04f65bf0c428c8fea9721.

Successful deletion of voting disk 3d34623f09b64f9dbfa44fabf455513e.

Successful deletion of voting disk 7043c38000a24f1abf36473ca7e9cd9e.

Successful deletion of voting disk 18de241007df4f9cbf3fbb4193f0ecb4.

CRS-4256: Updating the profile

CRS-4266: Voting file(s) successfully replaced

more to ASM diskgroup:

$ crsctl replace votedisk +OCRVOTE

CRS-4256: Updating the profile

Successful addition of voting disk 3d34623f09b64f9dbfa44fabf455513e.

Successful addition of voting disk 7043c38000a24f1abf36473ca7e9cd9e.

Successful addition of voting disk 18de241007df4f9cbf3fbb4193f0ecb4.

Successful deletion of voting disk a32c9b158e644fabbfdcc239c76f22a0.

Successfully replaced voting disk group with +CRS.

CRS-4256: Updating the profile

CRS-4266: Voting file(s) successfully replaced

联机文档描述

One reason to drop a disk group is to change the redundancy level (normal, high, or external). Because you cannot change

the redundancy of a disk group, you must drop the disk group and then re-create it with the proper redundancy. In this

case, you must back up or move disk group data before you drop the disk group

小结

如果要想改变asm磁盘组的冗余方式，就必须删除并重建磁盘组。

移走OCR磁盘内容2.1 备份2.1.1 备份OCR

ocrconfig –export /tmp/ocr.bak

或

ocrconfig –manualbackup

$ ocrconfig -showbackup

node2     2012/02/24 12:09:34  /u01/grid/cdata/PMSBIPD/backup00.ocr

node2     2012/02/24 08:03:21     /u01/grid/cdata/PMSBIPD/backup01.ocr

node2     2012/02/24 04:03:21     /u01/grid/cdata/PMSBIPD/backup02.ocr

node2     2012/02/22 19:38:28     /u01/grid/cdata/PMSBIPD/day.ocr

node2     2012/02/10 11:48:21     /u01/grid/cdata/PMSBIPD/week.ocr

PROT-25: Manual backups for the Oracle Cluster Registry are not available

备份VOTEDISK

In Oracle Clusterware 11g release 2 (11.2), you no longer have to back up the voting disk. The voting disk data is

automatically backed up in OCR as part of any configuration change and is automatically restored to any voting disk added.

If all voting disks are corrupted, however, you can restore them as described in "Restoring Voting Disks".---联机文档说不需

要备份votedisk，可以使用dd备份

查看VOTEDISK位置

$ crsctl query css votedisk

##  STATE    File Universal Id                File Name Disk group

--  -----    -----------------                --------- ---------

1. ONLINE   077fcbf9dd704f03bf95d2b1490208f2 (/dev/rdisk/disk444) [OCR_ORCL]

Located 1 voting disk(s).

备份

备份votedisk盘：

[root@node1 bin]# dd if=/dev/raw/raw5 of=/home/oracle/voting_disk.bak

256977+0 records in

256977+0 records out

恢复

恢复votedisk盘(如果真需要恢复的话)：

[root@node1 bin]# dd if=/home/oracle/voting_disk.bak of=/dev/raw/raw5

256977+0 records in

256977+0 records out

备份ASM参数文件

在OCR的磁盘组里，有OCR、VOTEDISK、ASM的SPFILE参数文件，但是这里ASM的参数文件是不需要备份的，因为我们一会在改ASM参数文件位置

的时候，会生产PFILE文件，那时就已经是备份了。

移动OCR

备注：这里将OCR移动到DATA_ORCL这个磁盘组上，在一个节点操作即可。

当OCR只有一份时，如果移动则会报错

NODE1#[/u01/grid/bin]./ocrconfig -replace OCR_ORCL -replacement  DATA_ORCL

PROT-28: Cannot delete or replace the only configured Oracle Cluster Registry location

添加一份OCR

NODE1#[/u01/grid/bin]./ocrconfig -add +DATA_ORCL

检查OCR位置

NODE1#[/u01/grid/bin]./ocrcheck

Status of Oracle Cluster Registry is as follows :

Version                  :          3

Total space (kbytes)     :     262120

Used space (kbytes)      :       2852

Available space (kbytes) :     259268

ID                       :  941087888

Device/File Name         :    +OCR_ORCL

Device/File integrity check succeeded

Device/File Name         :   +DATA_ORCL

Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

删除一份OCR

NODE1#[/u01/grid/bin]./ocrconfig -delete +OCR_ORCL

NODE1#[/u01/grid/bin]./ocrcheck

Status of Oracle Cluster Registry is as follows :

Version                  :          3

Total space (kbytes)     :     262120

Used space (kbytes)      :       2852

Available space (kbytes) :     259268

ID                       :  941087888

Device/File Name         :   +DATA_ORCL

Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

NODE1#[/u01/grid/bin]cat /var/opt/oracle/ocr.loc

#Device/file +OCR_ORCL getting replaced by device +DATA_ORCL

ocrconfig_loc=+DATA_ORCL

移动VOTEDISK

NODE1#[/u01/grid/bin]./crsctl replace votedisk +DATA_ORCL

Successful addition of voting disk 21dffc4c1ef84feebfaf2340c61853ba.

Successful deletion of voting disk 077fcbf9dd704f03bf95d2b1490208f2.

Successfully replaced voting disk group with +DATA_ORCL.

CRS-4266: Voting file(s) successfully replaced

NODE1#[/u01/grid/bin]./crsctl query css votedisk

##  STATE    File Universal Id                File Name Disk group

--  -----    -----------------                --------- ---------

1. ONLINE   21dffc4c1ef84feebfaf2340c61853ba (/dev/rdisk/disk447) [DATA_ORCL]

备注：在一个节点操作即可。

移动ASM参数文件

查看ASM位置

$sqlplus / as sysasm

SQL> show parameter spfile

NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

spfile                               string      +OCR_ORCL/pmsbipd/asmparameterfile/registry.253.760201253

创建PFILE

SQL> create pfile='/home/grid/asminit.ora' from spfile='+OCR_ORCL/pmsbipd/asmparameterfile/registry.253.760201253';

File created.

查看PFILE内容

$ cat asminit.ora

+ASM2.asm_diskgroups='DATA_ORCL','REDO_ORCL'#Manual Mount

+ASM1.asm_diskgroups='DATA_ORCL','REDO_ORCL'#Manual Mount

*.asm_power_limit=1

*.diagnostic_dest='/u01/oracle'

*.instance_type='asm'

*.large_pool_size=12M

*.remote_login_passwordfile='EXCLUSIVE'

关闭ASM实例

NODE2#[/u01/grid/bin]./srvctl stop asm -o abort -f

以PFILE启动ASM实例

$sqlplus / as sysasm

SQL> startup pfile='/home/grid/asminit.ora';

ASM instance started

Total System Global Area  283930624 bytes

Fixed Size                  2169104 bytes

Variable Size             256595696 bytes

ASM Cache                  25165824 bytes

ASM diskgroups mounted

SQL> show parameter spfile

NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

spfile                               string

创建SPFILE

SQL> create spfile='+DATA_ORCL' from pfile='/home/grid/asminit.ora';

File created.

以SPFILE启动ASM实例

NODE2#[/u01/grid/bin]./srvctl stop asm -o abort -f

$sqlplus / as sysasm

SQL> startup

ASM instance started

Total System Global Area  283930624 bytes

Fixed Size                  2169104 bytes

Variable Size             256595696 bytes

ASM Cache                  25165824 bytes

ASM diskgroups mounted

SQL> show parameter spfile

NAME    TYPE    VALUE

--------------     -----------------------------

spfile   string    +DATA_ORCL/pmsbipd/asmparameterfile/registry.253.776257911

节点二重启CLUSTER

将另一个节点的cluster重启以保证asm的spfile是新的参数文件

此时最好是将2个节点的cluster都重启下，让所有的资源都正常。

删除OCR磁盘组

查看OCR磁盘组状态

节点一：

SQL>  select name,state from v$asm_diskgroup;

NAME                           STATE

------------------------------ -----------

DATA_ORCL                        MOUNTED

REDO_ORCL                        MOUNTED

OCR_ORCL                         DISMOUNTED

节点二：

SQL>  select name,state from v$asm_diskgroup;

NAME                           STATE

------------------------------ -----------

DATA_ORCL                        MOUNTED

REDO_ORCL                        MOUNTED

OCR_ORCL                         DISMOUNTED

查看OCR磁盘组包括的磁盘

SQL> select name,GROUP_NUMBER ,DISK_NUMBER,state,REDUNDANCY ,TOTAL_MB,FREE_MB,path from v$asm_disk where name like 'OCR%';

NAME                           GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB PATH

------------------------------ ------------ ----------- -------- ------- ---------- ----------

----------------------------------------

OCR_ORCL_0000                               1           0 NORMAL   UNKNOWN       1031        844 /dev/rdisk/disk444

OCR_ORCL_0001                               1           1 NORMAL   UNKNOWN       1031        908 /dev/rdisk/disk445

OCR_ORCL_0002                               1           2 NORMAL   UNKNOWN       1031        909 /dev/rdisk/disk446

节点2卸载OCR磁盘

备注：节点一不卸载

SQL> alter diskgroup OCR_ORCL dismount ;

Diskgroup altered.

SQL>  select name,state from v$asm_diskgroup;

NAME                           STATE

------------------------------ -----------

DATA_ORCL                        MOUNTED

REDO_ORCL                        MOUNTED

OCR_ORCL                         DISMOUNTED

OCR磁盘组删除

备注：在节点一操作

SQL>  drop diskgroup OCR_ORCL including contents;

Diskgroup dropped.

3.5 两节点查看

SQL> select name,state from v$asm_diskgroup;

NAME                           STATE

------------------------------ -----------

DATA_ORCL                        MOUNTED

REDO_ORCL                        MOUNTED

SQL> select name,state from v$asm_diskgroup;

NAME                           STATE

------------------------------ -----------

DATA_ORCL                        MOUNTED

REDO_ORCL                        MOUNTED

ASMCA重建OCR磁盘组

打开

创建注意选择normal冗余

完成

查看状态

SQL> select name,state from v$asm_diskgroup;

NAME                           STATE

------------------------------ -----------

DATA_ORCL                        MOUNTED

REDO_ORCL                        MOUNTED

OCR_ORCL                         MOUNTED

将信息移回OCR磁盘组

OCR移回

查看OCR位置

NODE1#[/u01/grid/bin]./ocrcheck

Status of Oracle Cluster Registry is as follows :

Version                  :          3

Total space (kbytes)     :     262120

Used space (kbytes)      :       2852

Available space (kbytes) :     259268

ID                       :  941087888

Device/File Name         :   +DATA_ORCL

Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

./       Logical corruption check succeeded

添加一份OCR

NODE1#[/u01/grid/bin]./ocrconfig -add +OCR_ORCL

NODE1#[/u01/grid/bin]./ocrcheck

Status of Oracle Cluster Registry is as follows :

Version                  :          3

Total space (kbytes)     :     262120

Used space (kbytes)      :       2852

Available space (kbytes) :     259268

ID                       :  941087888

Device/File Name         :   +DATA_ORCL

Device/File integrity check succeeded

Device/File Name         :    +OCR_ORCL

Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

删除一份OCR

NODE1#[/u01/grid/bin]./ocrconfig -delete +DATA_ORCL

NODE2#[/u01/grid/bin]./ocrcheck

Status of Oracle Cluster Registry is as follows :

Version                  :          3

Total space (kbytes)     :     262120

Used space (kbytes)      :       2852

Available space (kbytes) :     259268

ID                       :  941087888

Device/File Name         :    +OCR_ORCL

Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

VOTEDISK移回

NODE1#[/u01/grid/bin]./crsctl query css votedisk

##  STATE    File Universal Id                File Name Disk group

--  -----    -----------------                --------- ---------

1. ONLINE   21dffc4c1ef84feebfaf2340c61853ba (/dev/rdisk/disk447) [DATA_ORCL]

Located 1 voting disk(s).

NODE1#[/u01/grid/bin]./crsctl replace votedisk +OCR_ORCL

Successful addition of voting disk 57ebf38a74524fbebf146a38ef3e1088.

Successful addition of voting disk 376ef62cf1c14febbfd6635bd9786366.

Successful addition of voting disk 6658efafcb194f7cbff49fa4bb8b5408.

Successful deletion of voting disk 21dffc4c1ef84feebfaf2340c61853ba.

Successfully replaced voting disk group with +OCR_ORCL.

CRS-4266: Voting file(s) successfully replaced

NODE1#[/u01/grid/bin]./crsctl query css votedisk

##  STATE    File Universal Id                File Name Disk group

--  -----    -----------------                --------- ---------

1. ONLINE   57ebf38a74524fbebf146a38ef3e1088 (/dev/rdisk/disk444) [OCR_ORCL]

2. ONLINE   376ef62cf1c14febbfd6635bd9786366 (/dev/rdisk/disk445) [OCR_ORCL]

3. ONLINE   6658efafcb194f7cbff49fa4bb8b5408 (/dev/rdisk/disk446) [OCR_ORCL]

Located 3 voting disk(s).

ASM参数文件移回

创建PFILE

$ sqlplus / as sysasm

SQL> show parameter spfile

NAME   TYPE        VALUE

-------------  -----------------------

spfile   string  +DATA_ORCL/pmsbipd/asmparameterfile/registry.253.776257911

SQL> create pfile='/home/grid/asmocrinit.ora' from spfile='+DATA_ORCL/pmsbipd/asmparameterfile/registry.253.776257911';

File created.

关闭ASM实例

NODE1#[/u01/grid/bin]./srvctl stop asm -o abort -f

启动ASM实例

$ sqlplus / as sysasm

SQL> startup pfile='/home/grid/asmocrinit.ora';

ASM instance started

Total System Global Area  283930624 bytes

Fixed Size                  2169104 bytes

Variable Size             256595696 bytes

ASM Cache                  25165824 bytes

ASM diskgroups mounted

创建SPFILE

SQL> create spfile='+OCR_ORCL' from pfile='/home/grid/asmocrinit.ora';

File created.

关闭ASM实例

NODE1#[/u01/grid/bin]./srvctl stop asm -o abort -f

节点一启动CLUSTER

NODE1#[/u01/grid/bin]./crsctl start cluster -n node1

节点二启动CLUSTER

NODE2#[/u01/grid/bin]./crsctl start cluster -n node2

查看OCR、VOTEDISK、ASM参数文件位置

查看OCR位置

NODE2#[/u01/grid/bin]./ocrcheck

Status of Oracle Cluster Registry is as follows :

Version                  :          3

Total space (kbytes)     :     262120

Used space (kbytes)      :       2852

Available space (kbytes) :     259268

ID                       :  941087888

Device/File Name         :    +OCR_ORCL

Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

查看VOTEDISK位置

NODE2#[/u01/grid/bin]./crsctl query css votedisk

##  STATE    File Universal Id                File Name Disk group

--  -----    -----------------                --------- ---------

1. ONLINE   57ebf38a74524fbebf146a38ef3e1088 (/dev/rdisk/disk444) [OCR_ORCL]

2. ONLINE   376ef62cf1c14febbfd6635bd9786366 (/dev/rdisk/disk445) [OCR_ORCL]

3. ONLINE   6658efafcb194f7cbff49fa4bb8b5408 (/dev/rdisk/disk446) [OCR_ORCL]

Located 3 voting disk(s).

查看ASM参数文件位置

$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.1.0 Production on Sun Feb 26 14:31:43 2012

Copyright (c) 1982, 2009, Oracle.  All rights reserved.

Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production

With the Real Application Clusters and Automatic Storage Management options

SQL> show parameter spfile

NAME                                 TYPE        VALUE