配置Centos5上使用ACFS集群文件系统

在centos5.8 64位系统上安装完一套oracle 11.2.0.3 rac测试环境后，发现acfs和advm无法使用，ASMCA图形界面的acfs和volume选项卡为灰色！这是由于acfs不支持centos系统导致的，下面就介绍下如何在配置centos上使用acfs集群文件系统！

一：问题描述

[grid@rac1 ~]$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.3.0 Production on Wed Feb 20 10:34:44 2013
Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> alter diskgroup data set attribute 'COMPATIBLE.ASM'='11.2';
Diskgroup altered.

SQL> alter diskgroup data add volume acfsvol size 1G;
alter diskgroup data add volume acfsvol size 1G
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15477: cannot communicate with the volume driver

[root@rac1 ~]# /u01/app/11.2.0/grid/bin/acfsload start -s
ACFS-9459: ADVM/ACFS is not supported on this OS version: 'centos-release-5-8.el5.centos'

[root@rac1 ~]# su - grid
[grid@rac1 ~]$ export DISPLAY=192.168.1.156:0
[grid@rac1 ~]$ asmca

二：修改osds_acfslib.pm脚本代码,rac2节点做同样的修改

[root@rac1 ~]# cd /u01/app/11.2.0/grid/lib/
[root@rac1 lib]# cp -p osds_acfslib.pm  osds_acfslib.pm.bak
[root@rac1 lib]# vi osds_acfslib.pm
  if ((defined($release)) &&                     # Redhat or OEL if defined
      (($release =~ /^redhat-release/) ||        # straight RH
       ($release =~ /^enterprise-release/) ||    # Oracle Enterprise Linux
       ($release =~ /^oraclelinux-release/)))    # Oracle Linux
  {

将上面的代码片段修改如下:
 if ((defined($release)) &&                     # Redhat or OEL if defined
      (($release =~ /^redhat-release/) ||        # straight RH
       ($release =~ /^enterprise-release/) ||    # Oracle Enterprise Linux
       ($release =~ /^centos-release/) ||        # CentOS hack
       ($release =~ /^oraclelinux-release/)))    # Oracle Linux
  {

三：安装acfs，配置acfs和advm模块启动自动加载，rac2节点做同样的配置

[root@rac1 ~]# /u01/app/11.2.0/grid/bin/acfsload start -s
ACFS-9129: ADVM/ACFS not installed

[root@rac1 ~]# /u01/app/11.2.0/grid/bin/acfsroot install
ACFS-9300: ADVM/ACFS distribution files found.
ACFS-9307: Installing requested ADVM/ACFS software.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9321: Creating udev for ADVM/ACFS.
ACFS-9323: Creating module dependencies - this may take some time.
ACFS-9154: Loading 'oracleoks.ko' driver.
ACFS-9154: Loading 'oracleadvm.ko' driver.
ACFS-9154: Loading 'oracleacfs.ko' driver.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
ACFS-9156: Detecting control device '/dev/ofsctl'.
ACFS-9309: ADVM/ACFS installation correctness verified.

[root@rac1 ~]# /u01/app/11.2.0/grid/bin/acfsload start -s

[root@rac1 ~]# lsmod |grep oracle
oracleacfs           1670360  1
oracleadvm            260320  5
oracleoks             321904  2 oracleacfs,oracleadvm

[root@rac1 ~]# cat > /etc/init.d/acfsload <<EOF
> #!/bin/sh
> # chkconfig: 2345 30 21
> # description: Load Oracle ASM volume driver on system startup
> ORACLE_HOME=/u01/app/11.2.0/grid
> export ORACLE_HOME
> \$ORACLE_HOME/bin/acfsload start -s
> EOF

[root@rac1 ~]# chmod 755 /etc/init.d/acfsload
[root@rac1 ~]# chkconfig --add acfsload
[root@rac1 ~]# chkconfig --list | grep acfsload
acfsload        0:off   1:off   2:on    3:on    4:on    5:on    6:off

四：添加ora.registry.acfs到ocr配置中

[root@rac1 ~]# /u01/app/11.2.0/grid/bin/crsctl add type ora.registry.acfs.type \
> -basetype ora.local_resource.type \
> -file /u01/app/11.2.0/grid/crs/template/registry.acfs.type

[root@rac1 ~]# /u01/app/11.2.0/grid/bin/crsctl add resource ora.registry.acfs \
> -attr ACL=\'owner:root:rwx,pgrp:oinstall:r-x,other::r--\' \
> -type ora.registry.acfs.type -f

[root@rac1 ~]# su - grid -c crs_stat | grep acfs
NAME=ora.registry.acfs
TYPE=ora.registry.acfs.type

[root@rac2 ~]# su - grid -c crs_stat | grep acfs
NAME=ora.registry.acfs
TYPE=ora.registry.acfs.type
[root@rac1 ~]# /sbin/acfsutil registry

五：创建卷并格式化成acfs集群文件系统并挂载

SQL> alter diskgroup data add volume acfsvol size 1G;
Diskgroup altered.

[root@rac1 ~]# mkdir /sharedisk
[root@rac2 ~]# mkdir /sharedisk
[root@rac1 ~]# /sbin/mkfs -t acfs -n ACFSVOL1 /dev/asm/acfsvol-53
mkfs.acfs: version                   = 11.2.0.3.0
mkfs.acfs: on-disk version           = 39.0
mkfs.acfs: volume                    = /dev/asm/acfsvol-53
mkfs.acfs: volume size               = 1073741824
mkfs.acfs: Format complete.

[root@rac1 ~]# /sbin/acfsutil registry -a -f /dev/asm/acfsvol-53 /sharedisk/
acfsutil registry: mount point /sharedisk successfully added to Oracle Registry

[root@rac1 ~]# mount.acfs -o all
[root@rac1 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                       93G   19G   70G  21% /
/dev/sda1             190M   17M  165M   9% /boot
tmpfs                 2.0G  1.2G  807M  60% /dev/shm
/dev/asm/acfsvol-53   1.0G   39M  986M   4% /sharedisk

[root@rac2 ~]# mount.acfs -o all
[root@rac2 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                       93G   12G   77G  14% /
/dev/sda1              99M   17M   78M  18% /boot
tmpfs                 2.0G  1.2G  807M  60% /dev/shm
/dev/asm/acfsvol-53   1.0G   73M  952M   8% /sharedisk

[root@rac1 ~]# /sbin/acfsutil info fs
/sharedisk
    ACFS Version: 11.2.0.3.0
    flags:        MountPoint,Available
    mount time:   Wed Feb 20 12:11:52 2013
    volumes:      1
    total size:   1073741824
    total free:   998002688
    primary volume: /dev/asm/acfsvol-53
        label:                 ACFSVOL1
        flags:                 Primary,Available,ADVM
        on-disk version:       39.0
        allocation unit:       4096
        major, minor:          252, 27137
        size:                  1073741824
        free:                  998002688
        ADVM diskgroup         DATA
        ADVM resize increment: 33554432
        ADVM redundancy:       unprotected
        ADVM stripe columns:   4
        ADVM stripe width:     131072
    number of snapshots:  0
    snapshot space usage: 0
    replication status: DISABLED

参考下列文章，感谢作者分享！

http://www.idevelopment.info/data/Oracle/DBA_tips/Automatic_Storage_Management/ASM_50.shtml