AIX_RAC_ASM存储迁移
0. 准备磁盘
0.1 在两个节点确认新增5个盘的lunid是否一致
lsattr -El hdisk6 | grep lun_id
lsattr -El hdisk7 | grep lun_id
lsattr -El hdisk8 | grep lun_id
lsattr -El hdisk9 | grep lun_id
lsattr -El hdisk10 | grep lun_id
0.2 修改reserve_policy为no_reserve
chdev -l hdisk6 -a reserve_policy=no_reserve
chdev -l hdisk7 -a reserve_policy=no_reserve
chdev -l hdisk8 -a reserve_policy=no_reserve
chdev -l hdisk9 -a reserve_policy=no_reserve
chdev -l hdisk10 -a reserve_policy=no_reserve
确认reserve_policy为no_reserve
查看共享磁盘属性
lsattr -El hdisk6 | grep reserve_policy
lsattr -El hdisk7 | grep reserve_policy
lsattr -El hdisk8 | grep reserve_policy
lsattr -El hdisk9 | grep reserve_policy
lsattr -El hdisk10 | grep reserve_policy
0.3 配置裸设备属性
chown grid:asmadmin /dev/rhdisk6
chown grid:asmadmin /dev/rhdisk7
chown grid:asmadmin /dev/rhdisk8
chown grid:asmadmin /dev/rhdisk9
chown grid:asmadmin /dev/rhdisk10
chmod 660 /dev/rhdisk6
chmod 660 /dev/rhdisk7
chmod 660 /dev/rhdisk8
chmod 660 /dev/rhdisk9
chmod 660 /dev/rhdisk10
chdev -l hdisk6 -a pv=clear
chdev -l hdisk7 -a pv=clear
chdev -l hdisk8 -a pv=clear
chdev -l hdisk9 -a pv=clear
chdev -l hdisk10 -a pv=clear
1. 备份
1.1 备份数据库
备份前，请将crontab 中的自动删除归档的定时任务注释掉。
XSWAPDB:
run
{
allocate channel c1 type disk;
allocate channel c2 type disk;
allocate channel c3 type disk;
allocate channel c4 type disk;
backup as compressed backupset Database format='/home/oracle/backup/20160512/xswapdb/bak_data_%U_%T' tag='database';
backup current controlfile tag='bak_ctlfile' format='/home/oracle/backup/20160512/xswapdb/ctl_file_%U_%T';
backup spfile tag='spfile' format='/home/oracle/backup/20160512/xswapdb/%d_spfile_%U_%T';
release channel c4;
release channel c3;
release channel c2;
release channel c1;
}
XSWAPDBS:
run
{
allocate channel c1 type disk;
allocate channel c2 type disk;
allocate channel c3 type disk;
allocate channel c4 type disk;
backup as compressed backupset Database format='/home/oracle/backup/20160512/xswapdbs/bak_data_%U_%T' tag='database';
sql 'alter system archive log current';
backup as compressed backupset archivelog all tag='arc_bak' format='/home/oracle/backup/20160512/xswapdbs/arch_%U_%T';
backup current controlfile tag='bak_ctlfile' format='/home/oracle/backup/20160512/xswapdbs/ctl_file_%U_%T';
backup spfile tag='spfile' format='/home/oracle/backup/20160512/xswapdbs/%d_spfile_%U_%T';
release channel c4;
release channel c3;
release channel c2;
release channel c1;
}
注意在199.31.178.24/26上有两个数据库：XSWAPDB,XSWAPDBS 两个数据库
所以要备份两次
1.2 备份GPNP profile和OCR
node1:
su - grid
cd $ORACLE_HOME/gpnp/uirsdbs03/profiles/peer
cp -p profile.xml profile.xml.bak.20160512
node2:
cd $ORACLE_HOME/gpnp/uirsdbs04/profiles/peer
cp -p profile.xml profile.xml.bak.20160512
在任意节点执行：
as root:
/oraapp/grid/gridhome/bin/ocrconfig -manualbackup
/oraapp/grid/gridhome/bin/ocrconfig -showbackup manual
2. 迁移OCR/Voting disk
2.1 新建diskgroup
hdisk6,hdisk7,hdisk8 为3个5G的盘。
asmca 创建新的OCR diskgroup 名称为OCRDG2
或者：
set linesize 200 pagesize 200
col path for a20
select name , path , failgroup,group_number , header_status, state , os_mb from v$asm_disk;
NAME PATH FAILGROUP GROUP_NUMBER HEADER_STATU STATE OS_MB
------------------------------ -------------------- ------------------------------ ------------ ------------ -------- ----------
/dev/rhdisk10 0 CANDIDATE NORMAL 204800
/dev/rhdisk6 0 CANDIDATE NORMAL 5120
/dev/rhdisk7 0 CANDIDATE NORMAL 5120
/dev/rhdisk8 0 CANDIDATE NORMAL 5120
/dev/rhdisk9 0 CANDIDATE NORMAL 153600
OCRDG_0000 /dev/rhdisk1 OCRDG_0000 3 MEMBER NORMAL 5120
OCRDG_0001 /dev/rhdisk2 OCRDG_0001 3 MEMBER NORMAL 5120
OCRDG_0002 /dev/rhdisk3 OCRDG_0002 3 MEMBER NORMAL 5120
FLASHDG_0000 /dev/rhdisk4 FLASHDG_0000 2 MEMBER NORMAL 153600
DATADG_0000 /dev/rhdisk5 DATADG_0000 1 MEMBER NORMAL 204800
create diskgroup OCRDG2 normal redundancy disk '/dev/rhdisk6','/dev/rhdisk7','/dev/rhdisk8' ATTRIBUTE 'compatible.rdbms' = '11.2', 'compatible.asm' = '11.2';
set linesize 200 pagesize 200
col path for a20
select name , path , failgroup,group_number , header_status, state , os_mb from v$asm_disk;
NAME PATH FAILGROUP GROUP_NUMBER HEADER_STATU STATE OS_MB
------------------------------ -------------------- ------------------------------ ------------ ------------ -------- ----------
/dev/rhdisk10 0 CANDIDATE NORMAL 204800
/dev/rhdisk9 0 CANDIDATE NORMAL 153600
OCRDG_0000 /dev/rhdisk1 OCRDG_0000 3 MEMBER NORMAL 5120
OCRDG_0001 /dev/rhdisk2 OCRDG_0001 3 MEMBER NORMAL 5120
OCRDG_0002 /dev/rhdisk3 OCRDG_0002 3 MEMBER NORMAL 5120
FLASHDG_0000 /dev/rhdisk4 FLASHDG_0000 2 MEMBER NORMAL 153600
DATADG_0000 /dev/rhdisk5 DATADG_0000 1 MEMBER NORMAL 204800
OCRDG2_0000 /dev/rhdisk6 OCRDG2_0000 4 MEMBER NORMAL 5120
OCRDG2_0001 /dev/rhdisk7 OCRDG2_0001 4 MEMBER NORMAL 5120
OCRDG2_0002 /dev/rhdisk8 OCRDG2_0002 4 MEMBER NORMAL 5120
2.2 新增OCR diskgroup
as root:
/oraapp/grid/gridhome/bin/ocrconfig -add +OCRDG2
检查OCR
/oraapp/grid/gridhome/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2684
Available space (kbytes) : 259436
ID : 940098899
Device/File Name : +OCRDG
Device/File integrity check succeeded
Device/File Name : +OCRDG2
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
2.3 移除旧的OCR
as root:
/oraapp/grid/gridhome/bin/ocrconfig -delete +OCRDG
/oraapp/grid/gridhome/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2684
Available space (kbytes) : 259436
ID : 940098899
Device/File Name : +OCRDG2
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
2.4 VOTEING DISK 迁移
as root:
/oraapp/grid/gridhome/bin/crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE ac31bee4b4514fa8bfd098fdf74de52a (/dev/rhdisk1) [OCRDG]
2. ONLINE 79f0d18e5e804fcebfdb0cf1b219d24a (/dev/rhdisk2) [OCRDG]
3. ONLINE a6b26c0438ac4fdabf81a1724a4e2e5a (/dev/rhdisk3) [OCRDG]
Located 3 voting disk(s).
替换VOTEDISK
/oraapp/grid/gridhome/bin/crsctl replace votedisk +OCRDG2
Successful addition of voting disk 01679fb067ea4f44bf345e5e1c6667fd.
Successful addition of voting disk 39c78d7a30b34fdabf2132fc06c9a646.
Successful addition of voting disk 05bcb0aea8eb4fc2bfe24514dec3f6fa.
Successful deletion of voting disk ac31bee4b4514fa8bfd098fdf74de52a.
Successful deletion of voting disk 79f0d18e5e804fcebfdb0cf1b219d24a.
Successful deletion of voting disk a6b26c0438ac4fdabf81a1724a4e2e5a.
Successfully replaced voting disk group with +OCRDG2.
CRS-4266: Voting file(s) successfully replaced
检查：
/oraapp/grid/gridhome/bin/crsctl query css votedisk
2.5 从RAC集群管理中移除旧的OCR diskgroup
2.5.1 : 停止OCR diskgroup资源
as grid:
crsctl stop res ora.OCRDG.dg
srvctl status diskgroup -g ocrdg
2.5.2 :移除OCR磁盘组的资源
srvctl disable diskgroup -g OCRDG
srvctl remove diskgroup -g OCRDG
查看OCR磁盘已经从CRS的管理中移除了
crsctl status res -t
在其中一个节点查看ASM磁盘组的信息
SQL> select NAME,STATE,TYPE,COMPATIBILITY from v$asm_diskgroup;
NAME STATE TYPE COMPATIBILITY
------------------------------ ----------- ------ ------------------------------------------------------------
DATADG MOUNTED EXTERN 11.2.0.0.0
FLASHDG MOUNTED EXTERN 11.2.0.0.0
OCRDG QUIESCING NORMAL 11.2.0.0.0
OCRDG2 MOUNTED NORMAL 11.2.0.0.0
强制dismount磁盘组(在两个节点都要运行)
SQL> alter diskgroup OCRDG dismount force;
Diskgroup altered.
检查状态：
SQL> select NAME,STATE,TYPE,COMPATIBILITY from v$asm_diskgroup;
NAME STATE TYPE COMPATIBILITY
------------------------------ ----------- ------ ------------------------------------------------------------
DATADG MOUNTED EXTERN 11.2.0.0.0
FLASHDG MOUNTED EXTERN 11.2.0.0.0
OCRDG DISMOUNTED 0.0.0.0.0
OCRDG2 MOUNTED NORMAL 11.2.0.0.0
--强制删除OCRDG
drop diskgroup OCRDG force including contents;
3. 迁移datadg , flashdg
as grid:
sqlplus / as sysasm
set linesize 200 pagesize 200
col name for a30
col path for a30
SQL> set linesize 200 pagesize 200
SQL> col path for a20
SQL> select name , path , failgroup,group_number , header_status, state , os_mb from v$asm_disk;
NAME PATH FAILGROUP GROUP_NUMBER HEADER_STATU STATE OS_MB
------------------------------ -------------------- ------------------------------ ------------ ------------ -------- ----------
/dev/rhdisk10 0 CANDIDATE NORMAL 204800
/dev/rhdisk1 0 FORMER NORMAL 5120
/dev/rhdisk2 0 FORMER NORMAL 5120
/dev/rhdisk3 0 FORMER NORMAL 5120
/dev/rhdisk9 0 CANDIDATE NORMAL 153600
FLASHDG_0000 /dev/rhdisk4 FLASHDG_0000 2 MEMBER NORMAL 153600
DATADG_0000 /dev/rhdisk5 DATADG_0000 1 MEMBER NORMAL 204800
OCRDG2_0000 /dev/rhdisk6 OCRDG2_0000 4 MEMBER NORMAL 5120
OCRDG2_0001 /dev/rhdisk7 OCRDG2_0001 4 MEMBER NORMAL 5120
OCRDG2_0002 /dev/rhdisk8 OCRDG2_0002 4 MEMBER NORMAL 5120
3.1 增加删除DATADG
alter diskgroup DATADG
add disk '/dev/rhdisk10' drop disk DATADG_0000
rebalance power 4 nowait;
3.2 增加删除FLASHDG
alter diskgroup FLASHDG
add disk '/dev/rhdisk9' drop disk FLASHDG_0000
rebalance power 4 nowait;
3.3 查看rebalance还需要多长时间
select * from v$asm_operation;
select * from gv$asm_operation;
4. 简单验证
SQL> set linesize 200 pagesize 200
SQL> col path for a20
SQL> select name , path , failgroup,group_number , header_status, state , os_mb from v$asm_disk;
SQL> select name , path , failgroup,group_number , header_status, state , os_mb from v$asm_disk;
NAME PATH FAILGROUP GROUP_NUMBER HEADER_STATU STATE OS_MB
------------------------------ -------------------- ------------------------------ ------------ ------------ -------- ----------
/dev/rhdisk4 0 FORMER NORMAL 153600
/dev/rhdisk1 0 FORMER NORMAL 5120
/dev/rhdisk2 0 FORMER NORMAL 5120
/dev/rhdisk3 0 FORMER NORMAL 5120
/dev/rhdisk5 0 FORMER NORMAL 204800
DATADG_0001 /dev/rhdisk10 DATADG_0001 1 MEMBER NORMAL 204800
OCRDG2_0000 /dev/rhdisk6 OCRDG2_0000 4 MEMBER NORMAL 5120
OCRDG2_0001 /dev/rhdisk7 OCRDG2_0001 4 MEMBER NORMAL 5120
OCRDG2_0002 /dev/rhdisk8 OCRDG2_0002 4 MEMBER NORMAL 5120
FLASHDG_0001 /dev/rhdisk9 FLASHDG_0001 2 MEMBER NORMAL 153600
XSWAPDB1
SQL> select sum(BYTES)/1024/1024/1024 from dba_data_files;
SUM(BYTES)/1024/1024/1024
-------------------------
62.4951172
XSWAPDBS1
SQL> select sum(BYTES)/1024/1024/1024 from dba_data_files;
SUM(BYTES)/1024/1024/1024
-------------------------
40.177002
reference:
OCR / Vote disk Maintenance Operations: (ADD/REMOVE/REPLACE/MOVE) (文档 ID 428681.1)
Exact Steps To Migrate ASM Diskgroups To Another SAN/Disk-Array/DAS/etc Without Downtime. (文档 ID 837308.1)
备注
11.2版本中,rebalance操作应该包含如下几个步骤了：
1) planning
2) extent relocation
3) compacting
这里针对这几个步骤简单描述一下：
planning： 也就是说oracle会自己计算，绝对需要将那些extent进行relocation以及需要move到什么地方去，应该也是用的hash算法.
extent relocation:这个其实是根据前面planning的结果，将数据按照extent为单位进行move,移动到其他的disk上，均匀分布。一般来讲，这个操作是非常耗时的，也就是说整个rebalance操作中基本上时间大多的消耗在relocation这一步。在进行extent relocation的阶段，是可以进行并行操作的，该操作是通过我们所熟知的一个参数asm_power_limit来进行控制。
该参数在11.2.0.2以下版本中,其取值范围是0~11. 在11.2.0.2以及以上版本取值范围已经扩展为0~1024了.
该参数控制rbal的slave process个数，换句话将，通常参数越大rebalance操作也就越快，当然这样也要看系统硬件配置.
另外有一点需要注意的是,rbal的slave process的可以动态调整的，例如：
alter diskgroup diskgroup_name rebalance power 5;
需要注意的是，哪怕是你alter diskgroup add disk命令已经发出了,也可以使用上面的方式来动态调整rebalance power值.
当rebalance power值大于1后,oracle 会启动多个rbal salve process,类似rba,rba1这样的命名.
这部分的消耗时间可以通过v$asm_operation.est_minutes来进行估算，但是这个值不一定准确，受cpu,io等因素的影响。
compacting： 这个操作是11gR2引入的一个特性,其目的是在前面extent relocation完成之后,oracle将diskgroup中都的每个disk中的数据进行重组。什么是重组？ 这里的重组其实是disk级别，不再是整个diskgroup级别. 其目的是将数据尽可能的挪到disk的外圈，这样可以加快访问，为什么可以加快访问？ 这样可以降低disk 寻道时间.