RAC安装之一 安装前准备

  Oracle中的RAC技术是一门很高深的艺术，RAC是real application cluster的英文缩写，中文翻译为真实应用集群。oracle的集群为多个数据库实例对应一个数据库，因而需要使用到共享存储，也就是所谓的盘阵(一盘多机)，在本文中使用ISCSI共享存储实现;RAC的安装是个繁琐的过程，需要一定的linux或者unix系统基础，因而笔者将会整理一个RAC在centos4.8的上安装系列教程，本节为教程的第一讲 ———— 安装前的环境准备。

本节总的内容包括以下几个要点：
1：安装linux操作系统，配置IP和主机名
2：调整系统参数
3：配置共享存储
4：配置raw设备
5：配置ASM

环境介绍：
操作系统：Centos4.8  64位(节点)
节点1主机名：rac1.766.com  
节点1IP地址：192.168.50.191/192.168.122.191 
节点2主机名：rac2.766.com
节点2IP地址：192.168.50.192/192.168.122.192

一：安装操作系统

1：这里采取PXE自动方式进行push安装，安装所有的软件包，可参考：http://ylw6006.blog.51cto.com/470441/587303

2：配置hosts文件，两个节点应当一致，rac1-vip所对应的ip可以不存在
[root@rac1 ~]# cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1               localhost.localdomain localhost

192.168.50.191          rac1.766.com          rac1
192.168.122.191         rac1-priv.766.com     rac1-priv
10.0.0.1                rac1-vip.766.com      rac1-vip


192.168.50.192          rac2.766.com          rac2
192.168.122.192         rac2-priv.766.com     rac2-priv
10.0.0.2                rac2-vip.766.com      rac2-vip

3：关闭selinux
[root@rac1 ~]# getenforce
Disabled

4：创建 Oracle 组和用户帐户，两个节点的oracle用户的uid，gid应当保持一致
[root@rac1 ~]# groupadd  oinstall
[root@rac1 ~]# groupadd  dba
[root@rac1 ~]# useradd -g oinstall -G dba oracle
[root@rac1 ~]# echo "oracle" |passwd --stdin oracle
Changing password for user oracle.
passwd: all authentication tokens updated successfully.

[root@rac1 ~]# id oracle
uid=500(oracle) gid=500(oinstall) groups=500(oinstall),501(dba)

二：调整系统参数

1：修改内核和系统参数
[root@rac2 ~]# id oracle
uid=500(oracle) gid=500(oinstall) groups=500(oinstall),501(dba)

[root@rac1 ~]# cat >> /etc/sysctl.conf <<EOF
> kernel.shmall = 2097152
> kernel.shmmax = 2147483648
> kernel.shmmni = 4096
> kernel.sem = 250 32000 100 128
> fs.file‐max = 65536
> net.ipv4.ip_local_port_range = 1024 65000
> net.core.rmem_default = 262144
> net.core.rmem_max = 262144
> net.core.wmem_default = 262144
> net.core.wmem_max = 262144
> EOF

[root@rac1 ~]# sysctl -p
[root@rac1 ~]# cat >> /etc/security/limits.conf <<EOF
> oracle soft nproc 2047
> oracle hard nproc 16384
> oracle soft nofile 1024
> oracle hard nofile 65536
> EOF

[root@rac1 ~]# cat >> /etc/pam.d/login <<EOF
> session required /lib/security/pam_limits.so
> EOF

[root@rac1 ~]# cat >> /etc/modprobe.conf <<EOF
> options hangcheck-timer hangcheck_tick=30 hangcheck_margin=180
> EOF

[root@rac1 ~]# modprobe -v hangcheck-timer
insmod /lib/modules/2.6.18-164.el5/kernel/drivers/char/hangcheck-timer.ko hangcheck_tick=30 hangcheck_margin=180

2:修改oracle用户环境变量，节点2的ORACLE_SID变量需要改成racdb2
[root@rac1 ~]# su - oracle
[oracle@rac1 ~]$ cat .bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs

PATH=/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin:/root/bin:/root/bin

export EDITOR=vim
export ORACLE_SID=racdb1
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1
export ORA_CRS_HOME=$ORACLE_BASE/product/10.2.0/crs_1
export LD_LIBRARY_PATH=/lib
alias sqlplus='/usr/local/rlwrap/bin/rlwrap sqlplus'
alias rman='/usr/local/rlwrap/bin/rlwrap rman'
export NLS_DATE_FORMAT='YYYY-MM-DD HH24:MI:SS'
export NLS_LANG=american_america.UTF8
export PATH=$ORACLE_HOME/bin:$ORA_CRS_HOME/bin:$PATH
umask 022


3：创建相关目录
[oracle@rac1 ~]$ mkdir -p $ORACLE_BASE/admin
[oracle@rac1 ~]$ mkdir -p $ORACLE_HOME
[oracle@rac1 ~]$ mkdir -p $ORA_CRS_HOME

4：配置oracle用户等效性
[oracle@rac1 ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Created directory '/home/oracle/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
99:4c:75:0b:3a:81:11:6a:e8:71:74:9e:94:89:00:d4 [email protected]
[oracle@rac1 ~]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
df:f6:c0:ab:7f:84:91:af:d8:d8:e0:fd:76:77:74:f2 [email protected]

[oracle@rac1 ~]$ cat .ssh/id_dsa.pub  >> .ssh/authorized_keys
[oracle@rac1 ~]$ cat .ssh/id_rsa.pub  >> .ssh/authorized_keys
[oracle@rac1 ~]$ scp /home/oracle/.ssh/authorized_keys [email protected]:/home/oracle/.ssh/

验证：不需要密码登录
在rac1节点上面分别ssh rac1;ssh rac1.766.com ; ssh rac2; ssh rac2.766.com
在rac2节点上面分别ssh rac1;ssh rac1.766.com ; ssh rac2; ssh rac2.766.com

三：配置共享存储

服务器IP：192.168.50.4
操作系统：RHEL6 64位

1:服务器端，详细配置步骤可参考http://ylw6006.blog.51cto.com/470441/580568
[root@rhel6 ~]# lvcreate -n sharedisk -L 80G VolGroup00
  Logical volume "sharedisk" created
[root@rhel6 ~]# yum -y install scsi-target-utils.x86_64
[root@rhel6 ~]# service tgtd start
[root@rhel6 ~]# service tgtd restart
Stopping SCSI target daemon: [  OK  ]
Starting SCSI target daemon: [  OK  ]
[root@rhel6 ~]# chkconfig --add tgtdrules
[root@rhel6 ~]# service tgtdrules start
Starting Tgtdrules Server:

[root@rhel6 ~]# service tgtdrules status
Target 1: iqn.2011-06-01.com.766.rac:shareddisk
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB
            Online: Yes
            Removable media: No
            Backing store type: rdwr
            Backing store path: None
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 85899 MB
            Online: Yes
            Removable media: No
            Backing store type: rdwr
            Backing store path: /dev/VolGroup00/sharedisk
    Account information:
    ACL information:
        192.168.50.191
        192.168.50.192



2：配置共享存储客户端，centos4和rhel5以及rhel6的配置略微有点不同
[root@rac1 ~]# wget ftp://ftp.is.co.za/mirror/centos/4.8/os/x86_64/CentOS/RPMS/iscsi-initiator-utils-4.0.3.0-8.x86_64.rpm
[root@rac1 ~]# rpm -ivh iscsi-initiator-utils-4.0.3.0-8.x86_64.rpm
warning: iscsi-initiator-utils-4.0.3.0-8.x86_64.rpm: V3 DSA signature: NOKEY, key ID 443e1821
Preparing...                ########################################### [100%]
   1:iscsi-initiator-utils  ########################################### [100%]

[root@rac1 ~]# grep -v '^#' /etc/iscsi.conf  |grep -v '^$'
DiscoveryAddress=192.168.50.4
TargetName=iqn.2011-06-01.com.766.rac:shareddisk
Enabled=yes

[root@rac1 ~]# service iscsi start
Checking iscsi config:  [  OK  ]
Loading iscsi driver:  [  OK  ]
Starting iscsid: [  OK  ]

[root@rac1 ~]# iscsi-ls
*******************************************************************************
SFNet iSCSI Driver Version ...4:0.1.11-7(14-Apr-2008)
*******************************************************************************
TARGET NAME             : iqn.2011-06-01.com.766.rac:shareddisk
TARGET ALIAS            :
HOST ID                 : 0
BUS ID                  : 0
TARGET ID               : 0
TARGET ADDRESS          : 192.168.50.4:3260,1
SESSION STATUS          : ESTABLISHED AT Sun Jun 19 07:23:36 CST 2011
SESSION ID              : ISID 00023d000001 TSIH 100
*******************************************************************************

[root@rac1 ~]# fdisk -l
Disk /dev/sda: 85.8 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sda doesn't contain a valid partition table

四：配置raw设备

1：将共享存储/dev/sda分成8个分区，前5个分区sda5-9分别为2G，sda10-12分别为20G
[root@rac1 ~]# fdisk -l
Disk /dev/sda: 85.8 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1       10443    83883366    5  Extended
/dev/sda5               1         244     1959867   83  Linux
/dev/sda6             245         488     1959898+  83  Linux
/dev/sda7             489         732     1959898+  83  Linux
/dev/sda8             733         976     1959898+  83  Linux
/dev/sda9             977        1220     1959898+  83  Linux
/dev/sda10           1221        3653    19543041   83  Linux
/dev/sda11           3654        6086    19543041   83  Linux
/dev/sda12           6087        8519    19543041   83  Linux
[root@rac1 ~]#

2：将sda5-9绑定为裸设备，节点2上需要做相同的配置
[root@rac1 ~]# cat /etc/sysconfig/rawdevices
# This file and interface are deprecated.
# Applications needing raw device access should open regular
# block devices with O_DIRECT.
# raw device bindings
# format:  <rawdev> <major> <minor>
#          <rawdev> <blockdev>
# example: /dev/raw/raw1 /dev/sda1
#          /dev/raw/raw2 8 5
/dev/raw/raw5    /dev/sda5
/dev/raw/raw6    /dev/sda6
/dev/raw/raw7    /dev/sda7
/dev/raw/raw8    /dev/sda8
/dev/raw/raw9    /dev/sda9

[root@rac1 ~]# service rawdevices restart
[root@rac1 ~]# raw -qa
/dev/raw/raw5:  bound to major 8, minor 5
/dev/raw/raw6:  bound to major 8, minor 6
/dev/raw/raw7:  bound to major 8, minor 7
/dev/raw/raw8:  bound to major 8, minor 8
/dev/raw/raw9:  bound to major 8, minor 9

[root@rac1 ~]# ll /dev/raw/raw*
crw-rw----  1 oracle oinstall 162,  5 Jun 19 08:10 /dev/raw/raw5
crw-rw----  1 oracle oinstall 162,  6 Jun 19 08:10 /dev/raw/raw6
crw-rw----  1 oracle oinstall 162,  7 Jun 19 08:10 /dev/raw/raw7
crw-rw----  1 oracle oinstall 162,  8 Jun 19 08:10 /dev/raw/raw8
crw-rw----  1 oracle oinstall 162,  9 Jun 19 08:10 /dev/raw/raw9
[root@rac1 ~]# chkconfig rawdevices on

五：配置ASM

1：从oracle网站下载并安装相应的包，节点2上需要做相同的配置
[root@rac1 ~]# cat down.txt
http://oss.oracle.com/projects/oracleasm-support/dist/files/RPMS/rhel4/amd64/2.1.3/oracleasm-support-2.1.3-1.el4.x86_64.rpm
http://download.oracle.com/otn_software/asmlib/oracleasmlib-2.0.4-1.el4.x86_64.rpm
http://oss.oracle.com/projects/oracleasm/dist/files/RPMS/rhel4/amd64/2.0.5/2.6.9-89.EL/oracleasm-2.6.9-89.EL-debuginfo-2.0.5-1.el4.x86_64.rpm
http://oss.oracle.com/projects/oracleasm/dist/files/RPMS/rhel4/amd64/2.0.5/2.6.9-89.EL/oracleasm-2.6.9-89.EL-2.0.5-1.el4.x86_64.rpm
[root@rac1 ~]# wget -i down.txt
[root@rac1 ~]# rpm -ivh *.rpm
warning: oracleasm-2.6.9-89.EL-2.0.5-1.el4.x86_64.rpm: V3 DSA signature: NOKEY, key ID b38a8516
Preparing...                ########################################### [100%]
   1:oracleasm-support      ########################################### [ 25%]
   2:oracleasm-2.6.9-89.EL  ########################################### [ 50%]
   3:oracleasm-2.6.9-89.EL-d########################################### [ 75%]
   4:oracleasmlib           ########################################### [100%]

2：配置asm，节点2上需要做相同的配置
[root@rac1 ~]# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: oracle
Default group to own the driver interface []: oinstall
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver: [  OK  ]
Scanning the system for Oracle ASMLib disks: [  OK  ]

3：创建ASM磁盘
[root@rac1 ~]# /etc/init.d/oracleasm createdisk VOL1 /dev/sda10
Marking disk "VOL1" as an ASM disk: [  OK  ]
[root@rac1 ~]# /etc/init.d/oracleasm createdisk VOL2 /dev/sda11
Marking disk "VOL2" as an ASM disk: [  OK  ]
[root@rac1 ~]# /etc/init.d/oracleasm createdisk VOL3 /dev/sda12
Marking disk "VOL3" as an ASM disk: [  OK  ]
[root@rac1 ~]# /etc/init.d/oracleasm listdisks
VOL1
VOL2
VOL3

3：在节点2上应该可以看到同样的结果
[root@rac2 ~]# /etc/init.d/oracleasm  enable
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver: [  OK  ]
Scanning the system for Oracle ASMLib disks: [  OK  ]
[root@rac2 ~]# /etc/init.d/oracleasm listdisks
VOL1
VOL2
VOL3