一，简介
二，配置虚拟机
1，创建虚拟机
(1)添加三块儿网卡：
| 主节点 | 二节点 |
| eth0: 公网 192.168.1.20/24 NAT | eth0： 公网 192.168.1.21/24 NAT |
| eth1：私网 192.168.2.20/24 host-only | eth1: 私网 192.168.2.21/24 host-only |
| eth2: DHCP Bridged | eth2： DHCP Bridged |
| 主机名： testdb11a | 主机名： testdb11b |
(2)添加五块儿磁盘：其中一块儿分配20G的空间供系统使用，其余四块儿为ASM磁盘，作为共享
创建4块儿独立磁盘，大小分配1GB，选择SCSI，独立永久
现在分配
共享磁盘放到单独的一个目录中，在复制二节点的时候方便寻找，取名字分别为：asm0.vmdk asm1.vmdk asm2.vmdk asm3.vmdk（一定要有后缀.vmdk）
Finsh~~
下面的三块执行同样的操作！
接下来，四块共享磁盘分别分配到和系统硬盘完全不同的总线下面
总体如下：
2，安装操作系统
安装的时候在这里需要注意：
只选择sda，其余的盘对勾全部去掉，继续~
全部默认，软件只要有图形界面即可，主机名，IP地址，和需要的软件我们在安装完操作系统后按需安装
安装11g的时候如果不适用VNS或者X-manager等远程图形界面工具，需要安装vm-tools，否则在开启图形界面的时候会特别的闪
3，配置 ----主节点
(1)关闭iptables和selinux
查看状态
[root@localhost ~]# /etc/init.d/iptables status
Table: filter
Chain INPUT (policy ACCEPT)
num target prot opt source destination
1 RH-Firewall-1-INPUT all -- 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT)
num target prot opt source destination
1 RH-Firewall-1-INPUT all -- 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT)
num target prot opt source destination
Chain RH-Firewall-1-INPUT (2 references)
num target prot opt source destination
1 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
2 ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 icmp type 255
3 ACCEPT esp -- 0.0.0.0/0 0.0.0.0/0
4 ACCEPT ah -- 0.0.0.0/0 0.0.0.0/0
5 ACCEPT udp -- 0.0.0.0/0 224.0.0.251 udp dpt:5353
6 ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:631
7 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:631
8 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
9 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:22
10 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
关闭防火墙
[root@localhost ~]# service iptables stop
Flushing firewall rules: [ OK ]
Setting chains to policy ACCEPT: filter [ OK ]
Unloading iptables modules: [ OK ]
[root@localhost ~]# chkconfig iptables off
查看
[root@localhost ~]# chkconfig --list | grep iptables
iptables 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[root@localhost ~]# service iptables status
Firewall is stopped.
禁用seLinux
查看selinux状态
[root@localhost ~]# getenforce
Enforcing
[root@localhost ~]# cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - SELinux is fully disabled.
SELINUX=enforcing
# SELINUXTYPE= type of policy in use. Possible values are:
# targeted - Only targeted network daemons are protected.
# strict - Full SELinux protection.
SELINUXTYPE=targeted
修改SELINUX=enforcing为
SELINUX=disabled
注：这里需要重启服务器，这里我们等所有的配置都完成后再重启
(2)网络配置
a.配置IP地址
修改配置文件
第一块网卡
[root@localhost ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth0
# Intel Corporation 82545EM Gigabit Ethernet Controller (Copper)
DEVICE=eth0
BOOTPROTO=static
IPADDR=192.168.1.20
NETMASK=255.255.255.0
GATEWAY=192.168.1.1
HWADDR=00:0C:29:0B:FD:9E
ONBOOT=yes
第二块儿网卡
[root@localhost ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth1
# Intel Corporation 82545EM Gigabit Ethernet Controller (Copper)
DEVICE=eth1
BOOTPROTO=static
IPADDR=192.168.2.20
NETMASK=255.255.255.0
GATEWAY=192.168.2.1
HWADDR=00:0C:29:0B:FD:A8
ONBOOT=yes
第三块网卡（保持默认）
[root@localhost ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth2
# Intel Corporation 82545EM Gigabit Ethernet Controller (Copper)
DEVICE=eth2
BOOTPROTO=dhcp
HWADDR=00:0C:29:0B:FD:B2
ONBOOT=no
b.修改主机名
[root@localhost ~]# vi /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=yes
HOSTNAME=testdb11a
[root@localhost ~]# hostname testdb11a
c.配置hosts文件
说明：如果不适用DNS，在/etc/hosts文件中必须包含虚假的SCAN IP信息
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
#上面两行不能注释
# Public
192.168.1.20 testdb11a
192.168.1.21 testdb11b
# Private
192.168.2.20 testdb11a-priv
192.168.2.21 testdb11b-priv
# Virtual
192.168.1.22 testdb11a-vip
192.168.1.23 testdb11b-vip
# SCAN
192.168.1.24 testdb11-scan
192.168.1.25 testdb11-scan
192.168.1.26 testdb11-scan
说明：SCAN 地址不应该在hosts文件中定义，而是在DNS中定义和public IP地址在同一个网段内的3个循环地址。在这里安装使用hosts文件避免了SCAN 地址，这种方式只实用于11.2.0.2以前的版本，11。2.0.2以后的版本，必须配置DNS，我们安装的是11.2.0.3.0版本，所以DNS是必须配置的，下面使用dnsmasq配置一个简单的DNS
在上面的/etc/hosts文件中加入3个和public网段相同的没有使用的iP地址，编辑下面的文件(nameserver后面可以是127.0.0.1，也可以是本地具体的IP地址)为了方便我们就选其中的一个节点做DNS服务器 [root@testdb11a ~]# more /etc/resolv.conf search localdomain nameserver 192.168.1.20 启动dnsmasq服务 [root@testdb11a e22489]# /etc/init.d/dnsmasq start Starting dnsmasq: [ OK ] 查看 [root@testdb11a e22489]# nslookup testdb11-scan Server: 192.168.1.20 Address: 192.168.1.20#53 Name: testdb11-scan Address: 192.168.1.26 Name: testdb11-scan Address: 192.168.1.24 Name: testdb11-scan Address: 192.168.1.25
(3)关于时间同步
节点之间的时间同步对于oracle RAC来说是非常重要的，在oracle11g r2之前需要通过操作系统搭建NTP服务完成节点之间时间的同步。在oracle 11g 开始新增加了一个CTSS(Cluster Time Synchronization Services，集群时间同步服务)，来完成节点间时间的同步，在安装Grid Infrastructure的时候，如果没有发现有NTP服务，则会自动安装ctss服务。这只对于RAC节点间时间的同步，对于其他系统来说是不起作用的。要使用oracle自己的CTSS，除了要停止、禁止自动重启NTP服务外，/etc/ntp.conf配置文件也不允许存在.(When the installer finds that the NTP protocol is not active, the Cluster Time Synchronization Service is automatically installed in active mode and synchronizes the time across the nodes. If NTP is found configured, then the Cluster Time Synchronization Service is started in observer mode, and no active time synchronization is performed by Oracle Clusterware within the cluster. )
[root@localhost ~]# service ntpd stop
Shutting down ntpd: [ OK ]
[root@localhost ~]# chkconfig ntpd off
[root@localhost ~]# chkconfig --list | grep ntpd
ntpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[root@localhost ~]# mv /etc/ntp.conf /etc/ntp.conf.
[root@localhost ~]# rm /var/run/ntpd.pid ###这个文件是维护nptd进程pid的
如果想要使用NTP服务进行时间的同步，必须在"/etc/sysconfig/ntpd"中加入“-x” 标记
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"
然后重启ntpd服务
#service ntpd restart
(4)内核参数及安全设置
修改/etc/sysctl.conf 添加下面内容
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 1054504960
kernel.shmmni = 4096
# semaphores: semmsl, semmns, semopm, semmni
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default=262144
net.core.rmem_max=4194304
net.core.wmem_default=262144
net.core.wmem_max=1048586
生效
sysctl -p
修改/etc/security/limits.conf
加入
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
session required pam_limits.so
(5)创建用户和组，以及所需要的目录
创建用户和组
groupadd -g 501 oinstall
groupadd -g 502 dba
groupadd -g 503 oper
groupadd -g 504 asmadmin
groupadd -g 505 asmoper
groupadd -g 506 asmdba
useradd -g oinstall -G dba,asmdba,oper oracle
useradd -g oinstall -G asmadmin,asmdba,asmoper,oper,dba grid
设置密码
passwd oracle
passwd grid
创建所需目录并给于相应的权限
mkdir -p /u01/app/grid mkdir -p /u01/app/11.2.0/grid chown -R grid:oinstall /u01 mkdir -p /u01/app/oracle chown oracle:oinstall /u01/app/oracle chmod -R 775 /u01
(6)安装所需要的rpm包
RHEL 5和Oracle Linux 5所需的包如下
binutils-2.17.50.0.6
compat-libstdc++-33-3.2.3
compat-libstdc++-33-3.2.3 (32 bit)
elfutils-libelf-0.125
elfutils-libelf-devel-0.125
gcc-4.1.2
gcc-c++-4.1.2
glibc-2.5-24
glibc-2.5-24 (32 bit)
glibc-common-2.5
glibc-devel-2.5
glibc-devel-2.5 (32 bit)
glibc-headers-2.5
ksh-20060214
libaio-0.3.106
libaio-0.3.106 (32 bit)
libaio-devel-0.3.106
libaio-devel-0.3.106 (32 bit)
libgcc-4.1.2
libgcc-4.1.2 (32 bit)
libstdc++-4.1.2
libstdc++-4.1.2 (32 bit)
libstdc++-devel 4.1.2
make-3.81
sysstat-7.0.2
-
unixODBC-2.2.11 (32-bit)or later -
unixODBC-devel-2.2.11 (64-bit)or later -
unixODBC-2.2.11 (64-bit)or later
检查
rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' binutils \
compat-libstdc++-33 \
elfutils-libelf \
elfutils-libelf-devel \
gcc \
gcc-c++ \
glibc \
glibc-common \
glibc-devel \
glibc-headers \
ksh \
libaio \
libaio-devel \
libgcc-4.1.2 \
libstdc++ \
libstdc++-devel \
make \
sysstat \
unixODBC \
unixODBC-devel
安装显示 not installed的包
使用yum安装，避免依赖关系的检查
配置yum源
[root@testdb11a ~]# mount /dev/cdrom /mnt
mount: block device /dev/cdrom is write-protected, mounting read-only
[root@testdb11a ~]#
[root@testdb11a ~]#
[root@testdb11a ~]# cat /etc/yum.repos.d/dvd.repo
[dvd]
name=install dvd
baseurl=file:///mnt/Server
enabled=1
gpgcheck=0
[root@testdb11a ~]# yum clean all
Loaded plugins: katello, product-id, security, subscription-manager
Updating certificate-based repositories.
Unable to read consumer identity
Cleaning up Everything
[root@testdb11a ~]#yum list
安装
[root@testdb11a ~]# rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' binutils compat-libstdc++-33 elfutils-libelf elfutils-libelf-devel gcc gcc-c++ glibc glibc-common glibc-devel glibc-headers ksh libaio libaio-devel libgcc-4.1.2 libstdc++ libstdc++-devel make sysstat unixODBC unixODBC-devel binutils-2.17.50.0.6-20.el5 (x86_64) compat-libstdc++-33-3.2.3-61 (x86_64) compat-libstdc++-33-3.2.3-61 (i386) elfutils-libelf-0.137-3.el5 (x86_64) elfutils-libelf-0.137-3.el5 (i386) elfutils-libelf-devel-0.137-3.el5 (i386) elfutils-libelf-devel-0.137-3.el5 (x86_64) package gcc is not installed package gcc-c++ is not installed glibc-2.5-81 (x86_64) glibc-2.5-81 (i686) glibc-common-2.5-81 (x86_64) package glibc-devel is not installed package glibc-headers is not installed ksh-20100621-5.el5 (x86_64) libaio-0.3.106-5 (x86_64) libaio-0.3.106-5 (i386) package libaio-devel is not installed libgcc-4.1.2-52.el5 (x86_64) libgcc-4.1.2-52.el5 (i386) libstdc++-4.1.2-52.el5 (x86_64) libstdc++-4.1.2-52.el5 (i386) package libstdc++-devel is not installed make-3.81-3.el5 (x86_64) package sysstat is not installed package unixODBC is not installed package unixODBC-devel is not installed [root@testdb11a ~]# yum install -y gcc gcc-c++ glibc-develglibc-headerslibaio-devel libstdc++-devel sysstat unixODBC ---把显示not install的包都跟在后面 Loaded plugins: katello, product-id, security, subscription-manager Updating certificate-based repositories. Unable to read consumer identity Setting up Install Process No package glibc-develglibc-headerslibaio-devel available. Resolving Dependencies --> Running transaction check ---> Package gcc.x86_64 0:4.1.2-52.el5 set to be updated --> Processing Dependency: glibc-devel >= 2.2.90-12 for package: gcc ---> Package gcc-c++.x86_64 0:4.1.2-52.el5 set to be updated ---> Package libstdc++-devel.i386 0:4.1.2-52.el5 set to be updated dvd/filelists | 2.8 MB 00:00 ---> Package libstdc++-devel.x86_64 0:4.1.2-52.el5 set to be updated ---> Package sysstat.x86_64 0:7.0.2-11.el5 set to be updated ---> Package unixODBC.i386 0:2.2.11-10.el5 set to be updated --> Processing Dependency: unixODBC-libs = 2.2.11-10.el5 for package: unixODBC --> Processing Dependency: libodbcinst.so.1 for package: unixODBC --> Processing Dependency: libodbc.so.1 for package: unixODBC ---> Package unixODBC.x86_64 0:2.2.11-10.el5 set to be updated --> Running transaction check ---> Package glibc-devel.x86_64 0:2.5-81 set to be updated --> Processing Dependency: glibc-headers = 2.5-81 for package: glibc-devel --> Processing Dependency: glibc-headers for package: glibc-devel ---> Package unixODBC-libs.i386 0:2.2.11-10.el5 set to be updated ---> Package unixODBC-libs.x86_64 0:2.2.11-10.el5 set to be updated --> Running transaction check ---> Package glibc-headers.x86_64 0:2.5-81 set to be updated --> Processing Dependency: kernel-headers >= 2.2.1 for package: glibc-headers --> Processing Dependency: kernel-headers for package: glibc-headers --> Running transaction check ---> Package kernel-headers.x86_64 0:2.6.18-308.el5 set to be updated --> Finished Dependency Resolution Dependencies Resolved ===================================================================================================================================================== Package Arch Version Repository Size ===================================================================================================================================================== Installing: gcc x86_64 4.1.2-52.el5 dvd 5.3 M gcc-c++ x86_64 4.1.2-52.el5 dvd 3.8 M libstdc++-devel i386 4.1.2-52.el5 dvd 2.8 M libstdc++-devel x86_64 4.1.2-52.el5 dvd 2.8 M sysstat x86_64 7.0.2-11.el5 dvd 187 k unixODBC i386 2.2.11-10.el5 dvd 290 k unixODBC x86_64 2.2.11-10.el5 dvd 291 k Installing for dependencies: glibc-devel x86_64 2.5-81 dvd 2.4 M glibc-headers x86_64 2.5-81 dvd 596 k kernel-headers x86_64 2.6.18-308.el5 dvd 1.4 M unixODBC-libs i386 2.2.11-10.el5 dvd 551 k unixODBC-libs x86_64 2.2.11-10.el5 dvd 554 k Transaction Summary ===================================================================================================================================================== Install 12 Package(s) Upgrade 0 Package(s) Total download size: 21 M Downloading Packages: ----------------------------------------------------------------------------------------------------------------------------------------------------- Total 465 MB/s | 21 MB 00:00 Running rpm_check_debug Running Transaction Test Finished Transaction Test Transaction Test Succeeded Running Transaction Installing : unixODBC-libs 1/12 Installing : kernel-headers 2/12 Installing : glibc-headers 3/12 Installing : glibc-devel 4/12 Installing : libstdc++-devel 5/12 Installing : unixODBC 6/12 Installing : sysstat 7/12 Installing : libstdc++-devel 8/12 Installing : unixODBC-libs 9/12 Installing : gcc 10/12 Installing : gcc-c++ 11/12 Installing : unixODBC 12/12 Installed products updated. Installed: gcc.x86_64 0:4.1.2-52.el5 gcc-c++.x86_64 0:4.1.2-52.el5 libstdc++-devel.i386 0:4.1.2-52.el5 libstdc++-devel.x86_64 0:4.1.2-52.el5 sysstat.x86_64 0:7.0.2-11.el5 unixODBC.i386 0:2.2.11-10.el5 unixODBC.x86_64 0:2.2.11-10.el5 Dependency Installed: glibc-devel.x86_64 0:2.5-81 glibc-headers.x86_64 0:2.5-81 kernel-headers.x86_64 0:2.6.18-308.el5 unixODBC-libs.i386 0:2.2.11-10.el5 unixODBC-libs.x86_64 0:2.2.11-10.el5 Complete!
最后检查
[root@testdb11a ~]# rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' binutils compat-libstdc++-33 elfutils-libelf elfutils-libelf-devel gcc gcc-c++ glibc glibc-common glibc-devel glibc-headers ksh libaio libaio-devel libgcc-4.1.2 libstdc++ libstdc++-devel make sysstat unixODBC unixODBC-devel binutils-2.17.50.0.6-20.el5 (x86_64) compat-libstdc++-33-3.2.3-61 (x86_64) compat-libstdc++-33-3.2.3-61 (i386) elfutils-libelf-0.137-3.el5 (x86_64) elfutils-libelf-0.137-3.el5 (i386) elfutils-libelf-devel-0.137-3.el5 (i386) elfutils-libelf-devel-0.137-3.el5 (x86_64) gcc-4.1.2-52.el5 (x86_64) gcc-c++-4.1.2-52.el5 (x86_64) glibc-2.5-81 (x86_64) glibc-2.5-81 (i686) glibc-common-2.5-81 (x86_64) glibc-devel-2.5-81 (x86_64) glibc-headers-2.5-81 (x86_64) ksh-20100621-5.el5 (x86_64) libaio-0.3.106-5 (x86_64) libaio-0.3.106-5 (i386) libaio-devel libgcc-4.1.2-52.el5 (x86_64) libgcc-4.1.2-52.el5 (i386) libstdc++-4.1.2-52.el5 (x86_64) libstdc++-4.1.2-52.el5 (i386) libstdc++-devel-4.1.2-52.el5 (i386) libstdc++-devel-4.1.2-52.el5 (x86_64) make-3.81-3.el5 (x86_64) sysstat-7.0.2-11.el5 (x86_64) unixODBC-2.2.11-10.el5 (x86_64) unixODBC-2.2.11-10.el5 (i386) unixODBC-devel-2.2.11-10.el5 (x86_64) unixODBC-devel-2.2.11-10.el5 (i386)
已经全部安装~~~
(7)设置grid和oracle的环境变量
grid用户：
[root@localhost ~]# su - grid
[grid@testdb11a ~]$ vi .bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin
#export PATH
##################################################
# User specific environment and startup programs #
##################################################
export ORACLE_BASE=/u01/app/grid
####export ORACLE_HOME=$ORACLE_BASE/grid/product/11.2.0
export ORACLE_HOME=/u01/app/11.2.0/grid ####这里一定不能使用上面的安装目录，否则在安装grid Infrastructure的时候包INS-32026
export ORACLE_PATH=$ORACLE_BASE/common/oracle/sql:.:$ORACLE_HOME/rdbms/admin
export ORACLE_SID=+ASM1
export PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin:$ORACLE_HOME/bin
export ORACLE_TERM=xterm
#export TNS_ADMIN=$ORACLE_HOME/network/admin
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export CLASSPATH=$ORACLE_HOME/JRE
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export THREADS_FLAG=native
export TEMP=/tmp
export TMPDIR=/tmp
##################################################
# set NLS_LANG to resolve messy code in SQLPLUS #
##################################################
export NLS_LANG=AMERICAN_AMERICA.WE8ISO8859P1
##################################################
# Shell setting. #
##################################################
umask 022
set -o vi
export PS1="\${ORACLE_SID}@`hostname` \${PWD}$ "
oracle用户：
[root@localhost ~]# su - oracle
dbrac1@testdb11a /home/oracle$ cat .bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin
#export PATH
##################################################
# User specific environment and startup programs#
##################################################
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/11.2.0
export ORACLE_PATH=$ORACLE_BASE/common/oracle/sql:.:$ORACLE_HOME/rdbms/admin
export ORACLE_SID=db111
export PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin:$ORACLE_HOME/bin
export ORACLE_TERM=xterm
#export TNS_ADMIN=$ORACLE_HOME/network/admin
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
export CLASSPATH=$ORACLE_HOME/JRE
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
export CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
export THREADS_FLAG=native
export TEMP=/tmp
export TMPDIR=/tmp
##################################################
# set NLS_LANG to resolve messy code in SQLPLUS
##################################################
export NLS_LANG=AMERICAN_AMERICA.WE8ISO8859P1
##################################################
# Shell setting.
##################################################
umask 022
set -o vi
export PS1="\${ORACLE_SID}@`hostname` \${PWD}$ "
(8)创建共享磁盘
[root@testdb11a dev]# ls sd*
sda sda1 sda2 sdb sdc sdd sde
分区
[root@testdb11a dev]# fdisk /dev/sdb
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-130, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-130, default 130):
Using default value 130
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
后面的sdc，sdd，sde执行相同的操作
[root@testdb11a dev]# ls sd*
sda sda1 sda2 sdb sdb1 sdc sdc1 sdd sdd1 sde sde1
编辑文件/etc/sysconfig/rawdevice
[root@testdb11a ~]# vi /etc/sysconfig/rawdevices
# raw device bindings
# format:
#
# example: /dev/raw/raw1 /dev/sda1
# /dev/raw/raw2 8 5
/dev/raw/raw1 /dev/sdb1
/dev/raw/raw2 /dev/sdc1
/dev/raw/raw3 /dev/sdd1
/dev/raw/raw4 /dev/sde1
启动rawdevice
[root@testdb11a ~]# /etc/init.d/rawdevices start
Assigning devices:
/dev/raw/raw1 --> /dev/sdb1
/dev/raw/raw1: bound to major 8, minor 17
/dev/raw/raw2 --> /dev/sdc1
/dev/raw/raw2: bound to major 8, minor 33
/dev/raw/raw3 --> /dev/sdd1
/dev/raw/raw3: bound to major 8, minor 49
/dev/raw/raw4 --> /dev/sde1
/dev/raw/raw4: bound to major 8, minor 65
done
查看
[root@testdb11a ~]# raw -qa
/dev/raw/raw1: bound to major 8, minor 17
/dev/raw/raw2: bound to major 8, minor 33
/dev/raw/raw3: bound to major 8, minor 49
/dev/raw/raw4: bound to major 8, minor 65
修改设备权限
[root@testdb11a raw]# ll
total 0
crw------- 1 root root 162, 1 Sep 15 23:07 raw1
crw------- 1 root root 162, 2 Sep 15 23:07 raw2
crw------- 1 root root 162, 3 Sep 15 23:07 raw3
crw------- 1 root root 162, 4 Sep 15 23:07 raw4
[root@testdb11a raw]# chown grid:asmdba /dev/raw/raw[1-4]
[root@testdb11a raw]# chmod 660 /dev/raw/raw[1-4]
[root@testdb11a raw]# ll
total 0
crw-rw---- 1 grid asmdba 162, 1 Sep 15 23:07 raw1
crw-rw---- 1 grid asmdba 162, 2 Sep 15 23:07 raw2
crw-rw---- 1 grid asmdba 162, 3 Sep 15 23:07 raw3
crw-rw---- 1 grid asmdba 162, 4 Sep 15 23:07 raw4
设置重启自动修改设备权限
编辑/etc/rc.d/rc.local
加入
chown grid:asmdba /dev/raw/raw[1-4]
chmod 660 /dev/raw/raw[1-4]
PS：需要安装vm-tools
4，配置副节点
(1)关闭系统
[root@testdb11a ~]# shutdown -h 0
Broadcast message from root (pts/2) (Mon Sep 15 23:59:39 2014):
The system is going down for system halt NOW!
把克隆的节点放到对应的目录下
(3)在主节点和二节点的虚拟机.vmx参数文件中加入
disk.locking="false" diskLib.dataCacheMaxSize = "0" diskLib.dataCacheMaxReadAheadSize = "0" diskLib.DataCacheMinReadAheadSize = "0" diskLib.dataCachePageSize = "4096" diskLib.maxUnsyncedWrites = "0" scsi1:0.deviceType = "disk" scsi1:1.deviceType = "disk" scsi1:2.deviceType = "disk" scsi1:3.deviceType = "disk" disk.EnableUUID ="true"
启动rac1和rac2两个虚拟机
(4)修改二节点中的与一节点中不同的配置
root用户登录系统，修改网络参数
- hostname: testdb11b
- IP Address eth0: 192.168.1.21 (public address)
- Default Gateway eth0: 192.168.1.1 (public address)
- IP Address eth1: 192.168.2.21 (private address)
- Default Gateway eth1: none
修改主机名
[root@testdb11a ~]# vi /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=yes
HOSTNAME=testdb11b
修改IP地址（ifcfg-eth0和ifcfg-eth1两个文件）
[root@testdb11a ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0 # Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) DEVICE=eth0 BOOTPROTO=static IPADDR=192.168.1.21 NETMASK=255.255.255.0 GATAWAY=192.168.1.1 ONBOOT=yes HWADDR=00:0c:29:b6:22:54 [root@testdb11a ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1 # Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) DEVICE=eth1 BOOTPROTO=static IPADDR=192.168.2.21 NETMASK=255.255.255.0 ONBOOT=yes HWADDR=00:0c:29:b6:22:5e
修改环境变量
grid用户
.bash_profile 中的 export ORACLE_SID=+ASM2
oracle用户
.bash_profile文件的export ORACLE_SID=db112
在rac2上查看共享磁盘
[root@testdb11a raw]# raw -qa
/dev/raw/raw1: bound to major 8, minor 17
/dev/raw/raw2: bound to major 8, minor 33
/dev/raw/raw3: bound to major 8, minor 49
/dev/raw/raw4: bound to major 8, minor 65
[root@testdb11a raw]# cd /dev/raw
[root@testdb11a raw]# ll
total 0
crw-rw---- 1 grid asmdba 162, 1 Sep 16 00:11 raw1
crw-rw---- 1 grid asmdba 162, 2 Sep 16 00:11 raw2
crw-rw---- 1 grid asmdba 162, 3 Sep 16 00:11 raw3
crw-rw---- 1 grid asmdba 162, 4 Sep 16 00:11 raw4
重启二节点虚拟机
执行
ping -c 3 testdb11a
ping -c 3 testdb11b
ping -c 3 testdb11a-priv
ping -c 3 testdb11b-priv
保证两台虚拟机之间网络互通
在安装之前，软件解压包的目录下有一个节点间校验系统要求的工具，可以使用下面的命令进行校验预安装要求，但是使用这个工具必须配置ssh互信，所有这里就不做校验
./runcluvfy.sh stage -pre crsinst -n testdb11a,testdb11b -verbose
校验结果样本
三，安装grid Infrastructure
(1)通过ftp或者其他方式上传软件，并解压
创建一个放置oracle软件的目录
mkdir -p /oracle/media chown -R oracle:oinstall /oracle/media chmod -R 775 /oracle/media
[root@testdb11a media]# ll total 3401628 -rw-r--r-- 1 oracle oinstall 1358454646 Nov 25 2011 p10404530_112030_Linux-x86-64_1of7.zip -rw-r--r-- 1 oracle oinstall 1142195302 Nov 27 2011 p10404530_112030_Linux-x86-64_2of7.zip ----上面两个为数据库软件 -rw-r--r-- 1 grid oinstall 979195792 Sep 15 18:11 p10404530_112030_Linux-x86-64_3of7.zip ----grid 软件 [root@testdb11a media]#
解压
[root@testdb11a app]# cd /oracle/media/ [root@testdb11a media]# ll total 8 drwxr-xr-x 8 oracle oinstall 4096 Sep 22 2011 database drwxr-xr-x 8 grid oinstall 4096 Sep 22 2011 grid [root@testdb11a media]#
在/oracle/media/grid/rpm这个目录下有个cvuqdisk-1.0.9-1.rpm包，在两个节点上安装
scp cvuqdisk-1.0.9-1.rpm testdb11b:/home/grid [root@testdb11b grid]# rpm -ivh cvuqdisk-1.0.9-1.rpm Preparing... ########################################### [100%] Using default group oinstall to install package 1:cvuqdisk ########################################### [100%]
(2)启动图形界面
执行./runInstaller
跳过软件更新
选择“install and Configure Grid Infrastructure for a Cluster ”
选择“Typical Installation”
输入正确的SCAN名，点击Add把另一个节点加入
填入二节点的正确的Hostname和vip name
点击“SSH Connectivity”，输入grid的密码，点击“Setup”配置互信（备注：下面的这幅截图SCAN Name写错了，应该是testdb11-scan，和hosts文件中的相同）
出现上面的提示，则互信配置成功，下面点击“Test”进行测试
点击“Identify network interfaces..”检查网络配置是否正确，确认无误后，点击“OK”
输入grid的安装目录，选择“Oracle Automatic Storage Management ”,输入ASM的密码，下一步
使用简单密码，继续
选择“External”，选择所有的磁盘，如果磁盘不显示，点击“Change Discovery Path”，输入“/dev/raw/raw*”
默认
下面的这几个警告可以忽略，勾选"Ignore All"，继续（说明，下图中第三个警告不能跳过）
安装
提示需要执行两个脚本
执行结果如下
节点一，节点二
在这里二节点执行root.sh的时候报错
CRS-5005: IP Address: 192.168.1.24 is already in use in the network . For details refer to "(:CLSN00107:)" in "/u01/app/11.2.0/grid/log/testdb11b/agent/crsd/orarootagent_root/orarootagent_root.log".
出现上面的错误的原因是：我没有配置dns服务，只在hosts文件中加入了相应的scan-ip，希望能骗过oracle，但是这种做法在11.2.0.2开始这种做法已经不可取，所以只能配置dns去解决。下面的安装软件过程中检测出来的错误，在11.2.0.2之前是不检查/etc/resolv.conf这个文件的(两个节点都需要配置)
重新执行脚本，
显示成功后，点击OK继续安装
查看日志
INFO: Checking Single Client Access Name (SCAN)... INFO: ERROR: INFO: PRVF-5056 : Scan Listener "LISTENER_SCAN1" not running INFO: Checking name resolution setup for "testdb11-scan"... INFO: ERROR: INFO: PRVF-4657 : Name resolution setup check for "testdb11-scan" (IP address: 192.168.1.25) failed INFO: ERROR: INFO: PRVF-4657 : Name resolution setup check for "testdb11-scan" (IP address: 192.168.1.26) failed INFO: ERROR: INFO: PRVF-4664 : Found inconsistent name resolution entries for SCAN name "testdb11-scan" INFO: Verification of SCAN VIP and Listener setup failed
可以放心跳过
四，安装数据库软件
进入oracle用户，database目录，执行
./runInstaller
点击下一步
点击“SSH Connectivity”配置oracle用户的节点间互信。然后点击“Setup”，并“Test”
选择语言，然后下一步
选择企业版
选择安装路径
选择管理组
先决条件检查，选择忽略
我分配的内存为1GB，内存绝对不能小于1GB推荐最小1.5g。卡在56%半天才过去
执行脚本
执行完成后，点击OK，database soft安装完成
五，使用DBCA进行建库
出现上面的结果，表明顺利安装完成！
验证：
查看两个节点的服务状态：
[root@testdb11a db11]# /u01/app/11.2.0/grid/bin/crs_stat -t -v Name Type R/RA F/FT Target State Host ---------------------------------------------------------------------- ora.DATA.dg ora....up.type 0/5 0/ ONLINE ONLINE testdb11a ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE testdb11a ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE testdb11a ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE testdb11a ora.cvu ora.cvu.type 0/5 0/0 ONLINE ONLINE testdb11a ora.db11.db ora....se.type 0/2 0/1 ONLINE ONLINE testdb11a ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE ora....network ora....rk.type 0/5 0/ ONLINE ONLINE testdb11a ora.oc4j ora.oc4j.type 0/1 0/2 ONLINE ONLINE testdb11a ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE testdb11a ora....ry.acfs ora....fs.type 0/5 0/ ONLINE ONLINE testdb11a ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE testdb11a ora....SM1.asm application 0/5 0/0 ONLINE ONLINE testdb11a ora....1A.lsnr application 0/5 0/0 ONLINE ONLINE testdb11a ora....11a.gsd application 0/5 0/0 OFFLINE OFFLINE ora....11a.ons application 0/3 0/0 ONLINE ONLINE testdb11a ora....11a.vip ora....t1.type 0/0 0/0 ONLINE ONLINE testdb11a ora....SM2.asm application 0/5 0/0 ONLINE ONLINE testdb11b ora....1B.lsnr application 0/5 0/0 ONLINE ONLINE testdb11b ora....11b.gsd application 0/5 0/0 OFFLINE OFFLINE ora....11b.ons application 0/3 0/0 ONLINE ONLINE testdb11b ora....11b.vip ora....t1.type 0/0 0/0 ONLINE ONLINE testdb11b
说明：gsd是在9i rac中用于管理客户端的服务，11g默认的关闭的。
查看数据库状态
db112@testdb11b /home/oracle$ sqlplus / as sysdba SQL*Plus: Release 11.2.0.3.0 Production on Sun Sep 21 00:07:14 2014 Copyright (c) 1982, 2011, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP, Data Mining and Real Application Testing options SQL> select instance_name,status from gv$instance; INSTANCE_NAME STATUS ---------------- ------------ db111 OPEN db112 OPEN
SQL> select * from v$active_instances;
INST_NUMBER INST_NAME
----------- ------------------------------------------------------------
1 testdb11a:db111
2 testdb11b:db112
db111@testdb11a /home/oracle$ srvctl config database -d db11 Database unique name: db11 Database name: db11 Oracle home: /u01/app/oracle/product/11.2.0 Oracle user: oracle Spfile: +DATA/db11/spfiledb11.ora Domain: Start options: open Stop options: immediate Database role: PRIMARY Management policy: AUTOMATIC Server pools: db11 Database instances: db111,db112 Disk Groups: DATA Mount point paths: Services: Type: RAC Database is administrator managed db111@testdb11a /home/oracle$ srvctl status database -d db11 Instance db111 is running on node testdb11a Instance db112 is running on node testdb11b
六，遇到的问题
问题1：二节点执行root.sh脚本报错
问题2：只能看到一个节点
解决：oracle 11g RAC安装节点二执行结果错误CRS-5005: IP Address: 192.168.1.24 is already in use in the network