【Ora12c】基于RHEL6.6部署RAC

0. 规划:

OS: RedHat Linux 6.6 64bit
Grid: 12.1.0.1
Oracle: 12.1.0.1

节点定义:

名称 hostname 公有IP VIP 私网IP scan-ip
节点1 ora-node1 10.1.7.81 10.1.7.86 172.116.0.81 10.1.7.239
节点2 ora-node2 10.1.7.82 10.1.7.87 172.116.0.82

磁盘:

名称 配置方式 容量 数量 用途
/dev/vdb LVM 50G 1 grid、oracle软件、BASE目录、HOME目录
DATA udev 50G 3 数据磁盘组
FRA udev 15G 2 闪回区
OCR_VOTING udev 8G 3 OCR,voting disk

1. OS准备过程

1.1 安装系统包

yum install gcc gcc-c++ binutils compat-libcap1 compat-libstdc++-33 \
glibc glibc-devel ksh libaio libaio-devel libgcc libstdc++ libstdc++-devel \
libxcb libX11 libXau libXi libXtst make net-tools nfs-utils smartmontools \
sysstat xterm unixODBC unixODBC-devel -y

1.2 禁用SELINUX

所有节点

sed 's,'SELINUX=enforcing','SELINUX=disabled',g' -i /etc/sysconfig/selinux 
sed 's,'SELINUX=enforcing','SELINUX=disabled',g' -i /etc/selinux/config
setenforce 0

测试环境可以停用防火墙,但生产环境不要停用防火墙。

service iptables stop
chkconfig iptables off

1.3 配置时间同步

所有节点
使用CTSS,所以要停用 NTP 服务。

service ntpd stop
chkconfig ntpd off
mv /etc/ntp.conf /etc/ntp.conf.old
rm -rf /var/run/ntpd.pid

1.4 设置系统用户资源限制

所有节点

1.4.1 修改/etc/security/limits.conf
cat >> /etc/security/limits.conf <> /etc/pam.d/login <> /etc/profile <> /etc/sysctl.conf <> /etc/sysconfig/network
```
注:如果不加入该配置,在安装grid之前的检查会出现如下信息:

![Paste_Image.png](http://upload-images.jianshu.io/upload_images/5260316-1cfa71b24e49bcd3.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![Paste_Image.png](http://upload-images.jianshu.io/upload_images/5260316-d45c1808c70863bd.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)
可以参考http://blog.csdn.net/msdnchina/article/details/45462319。


####1.5 编辑hosts
```
vim /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

10.1.7.81 ora-node1
172.116.0.81 ora-node1-priv
10.1.7.86 ora-node1-vip

10.1.7.82 ora-node2
172.116.0.82 ora-node2-priv
10.1.7.87 ora-node2-vip
```
注意:不要在此文件中配置scan地址和名字。

####1.6 创建用户和组
两个节点都执行
```
groupadd -g 1000 oinstall 
groupadd -g 1200 asmadmin 
groupadd -g 1201 asmdba 
groupadd -g 1202 asmoper 
groupadd -g 1300 dba 
groupadd -g 1301 oper 
useradd -m -u 1100 -g oinstall -Gasmadmin,asmdba,asmoper,dba -d \
      /home/grid -s /bin/bash grid 
useradd -m -u 1101 -g oinstall -Gdba,oper,asmdba -d \
     /home/oracle -s /bin/bash oracle
gpasswd -a grid dba                                      #将grid用户加入到dba组
echo "oracle" | passwd --stdin grid
echo "oracle" | passwd --stdin oracle
```
节点用户(grid,oracle)间互信。以下命令分别在各个节点上的oracle用户和grid用户下执行。
```
ssh-keygen -t rsa
ssh-keygen -t dsa
```
下面的命令可以在其中一个节点执行,然后将生成的authorized_keys文件复制到其它节点。
```
cat .ssh/id_rsa.pub >> .ssh/authorized_keys
cat .ssh/id_dsa.pub >> .ssh/authorized_keys
ssh ora-node2 cat ~/.ssh/id_rsa.pub >> .ssh/authorized_keys
ssh ora-node2 cat ~/.ssh/id_dsa.pub >> .ssh/authorized_keys
scp .ssh/authorized_keys ora-node2:~/.ssh/.
```
验证。在各个节点执行如下格式命令。
```
ssh {hostname} date
```
####1.7 创建目录
所有节点
```
mkdir -p /u01/app/soft                  #安装的clusterware和DB软件包都放在该目录下.
mkdir -p /u01/app/12.1.0/grid           #grid的ORACLE_HOME
chown -R grid:oinstall /u01/app/
mkdir -p /u01/app/oracle                #oracle的ORACLE_BASE
chown -R oracle:oinstall /u01/app/oracle   
chmod -R 775 /u01/
```
####1.8 编辑grid,oracle用户的环境变量
grid用户的。
```
# su - grid
$ vim .bash_profile 

export ORACLE_SID=+ASM1
#export ORACLE_SID=+ASM2
export ORACLE_BASE=/u01/app/grid
export ORACLE_HOME=/u01/app/12.1.0/grid_1
export TEMP=/tmp
export TMP=/tmp
export TMPDIR=/tmp
export PATH=$ORACLE_HOME/bin:$PATH:$HOME/bin
umask 022

$ . .bash_profile
```
oracle用户的。
```
# su - oracle
$ vim .bash_profile
export ORACLE_SID=ORCL1
#export ORACLE_SID=ORCL2
export ORACLE_UNQNAME=ORCL
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/12.1.0/db_1
LD_LIBRARY_PATH=/lib:/usr/lib:/usr/local/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$ORACLE_HOME/oracm/lib:$LD_LIBRARY_PATH

export CLASSPATH=$ORACLE_HOME/jre:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/network/jlib
export THREADS_FLAG=native
export TEMP=/tmp
export TMPDIR=/tmp
export PATH=$ORACLE_HOME/bin:$PATH:$HOME/bin
umask 022
#以下可以不加
#export JAVA_HOME=/u01/app/12.1.0/grid_1/jdk
#export ORACLE_TERM=xterm 
#export NLS_DATE_FORMAT="YYYY:MM:DDHH24:MI:SS" 
#export NLS_LANG=american_america.AL32UTF8 
#export TNS_ADMIN=$ORACLE_HOME/network/admin 
#export ORA_NLS11=$ORACLE_HOME/nls/data

$ . .bash_profile
```
####1.9 使用udev配置asm使用的磁盘
**仅仅在一个节点处理即可**

######1.9.1 使用fdisk处理磁盘
```
# fdisk -l | grep sd
Disk /dev/sda: 53.7 GB, 53687091200 bytes
Disk /dev/sdb: 8589 MB, 8589934592 bytes
Disk /dev/sdc: 8589 MB, 8589934592 bytes
Disk /dev/sdd: 8589 MB, 8589934592 bytes
Disk /dev/sde: 16.1 GB, 16106127360 bytes
Disk /dev/sdf: 16.1 GB, 16106127360 bytes
Disk /dev/sdg: 53.7 GB, 53687091200 bytes
```
将上面磁盘进行分区,如下:
```
# fdisk /dev/sda
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xb85ca157.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-8192, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-8192, default 8192): 
Using default value 8192

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
```
######1.9.2 编写shell脚本,生成udev配置规则文件
```
# vim genasmdisk.sh
#!/bin/bash
cat /dev/null > /etc/udev/rules.d/99-oracle-asmdisk.rules
for i in a b c d e f g;
do 
    echo "KERNEL==\"sd?1\", BUS==\"scsi\",PROGRAM==\"/sbin/scsi_id -g -u -d /dev/\$parent\", RESULT==\"`/sbin/scsi_id -g -u -d /dev/sd$i`\", NAME=\"asm-disk$i\", OWNER=\"grid\",GROUP=\"asmadmin\", MODE=\"0660\"" >> /etc/udev/rules.d/99-oracle-asmdisk.rules
done
```
执行脚本。
```
# chmod +x genasmdisk.sh
# ./genasmdisk.sh
```
######1.9.3 检查udev规则文件内容
```
# cat /etc/udev/rules.d/99-oracle-asmdisk.rules
KERNEL=="sd?1", BUS=="scsi",PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="0QEMU_QEMU_HARDDISK_244c9f1f-f15d-49b3-a", NAME="asm-diska", OWNER="grid",GROUP="asmadmin", MODE="0660"
KERNEL=="sd?1", BUS=="scsi",PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="0QEMU_QEMU_HARDDISK_080377de-ef70-4de5-8", NAME="asm-diskb", OWNER="grid",GROUP="asmadmin", MODE="0660"
KERNEL=="sd?1", BUS=="scsi",PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="0QEMU_QEMU_HARDDISK_7b9b90c2-7330-4fe9-9", NAME="asm-diskc", OWNER="grid",GROUP="asmadmin", MODE="0660"
KERNEL=="sd?1", BUS=="scsi",PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="0QEMU_QEMU_HARDDISK_45f70178-1dfe-454a-9", NAME="asm-diskd", OWNER="grid",GROUP="asmadmin", MODE="0660"
KERNEL=="sd?1", BUS=="scsi",PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="0QEMU_QEMU_HARDDISK_ca7283c4-c521-459f-a", NAME="asm-diske", OWNER="grid",GROUP="asmadmin", MODE="0660"
KERNEL=="sd?1", BUS=="scsi",PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="0QEMU_QEMU_HARDDISK_6fdb0e82-8c1d-4694-a", NAME="asm-diskf", OWNER="grid",GROUP="asmadmin", MODE="0660"
KERNEL=="sd?1", BUS=="scsi",PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="0QEMU_QEMU_HARDDISK_65a9f8aa-e13b-4719-a", NAME="asm-diskg", OWNER="grid",GROUP="asmadmin", MODE="0660"
```
注意:将该rules文件复制到其它节点。

######1.9.4 重启udev,使rules生效(所有节点都执行)
```
# partprobe /dev/sda
# partprobe /dev/sdb
......
```
或
```
# start_udev
```
**建议:这时所有节点重启机器。**

######1.9.5 检查asm disk
```
# ls -al /dev/asm*
brw-rw----. 1 grid asmadmin 8,  1 May 17 15:31 /dev/asm-diska
brw-rw----. 1 grid asmadmin 8, 17 May 17 15:31 /dev/asm-diskb
brw-rw----. 1 grid asmadmin 8, 33 May 17 15:31 /dev/asm-diskc
brw-rw----. 1 grid asmadmin 8, 49 May 17 15:31 /dev/asm-diskd
brw-rw----. 1 grid asmadmin 8, 65 May 17 15:31 /dev/asm-diske
brw-rw----. 1 grid asmadmin 8, 81 May 17 15:31 /dev/asm-diskf
brw-rw----. 1 grid asmadmin 8, 97 May 17 15:31 /dev/asm-diskg
```
到此系统层面的准备工作基本完成(注意swap的要求,这里忽略)。开始安装Oracle集群套件。

####2. 安装Grid Infrastructure
######2.1 下载软件到/u01/app/soft/目录下
从[下载地址](http://www.oracle.com/technetwork/database/enterprise-edition/downloads/database12c-linux-download-2240591.html)下载软件包。解压软件包,然后运行`runInstall`。
- linuxamd64_12c_grid_1of2.zip
- linuxamd64_12c_grid_2of2.zip

######2.2 开始OUI界面安装

![安装开始](http://upload-images.jianshu.io/upload_images/5260316-e533d8184c3e0062.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![在12C中,标准的集群,基本上可以等同11g的RAC。Flex集群是12c新特性。](http://upload-images.jianshu.io/upload_images/5260316-9839f4394552abdb.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![高级安装](http://upload-images.jianshu.io/upload_images/5260316-54723a4d36fc528c.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![语言选择,默认](http://upload-images.jianshu.io/upload_images/5260316-2adaa1590bc35e50.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![定义进群名称,和SCAN名称](http://upload-images.jianshu.io/upload_images/5260316-9dfb45af90074f29.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![增加集群节点](http://upload-images.jianshu.io/upload_images/5260316-0c41ce24c4159f67.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![配置、测试节点的连通性](http://upload-images.jianshu.io/upload_images/5260316-f8d158ac2ec730d3.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![配置公有、私有、ASM网络](http://upload-images.jianshu.io/upload_images/5260316-d5b206fdcd6c5d03.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![配置ASM,直接配置的Flex ASM](http://upload-images.jianshu.io/upload_images/5260316-02c51be8976a5105.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![创建ASM磁盘组](http://upload-images.jianshu.io/upload_images/5260316-871e107f309ed158.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![配置ASM管理员密码,Oracle_123](http://upload-images.jianshu.io/upload_images/5260316-b6a58f259ff30c82.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![配置Failure,默认](http://upload-images.jianshu.io/upload_images/5260316-7f1d79cbdd79be5d.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![配置EM,默认](http://upload-images.jianshu.io/upload_images/5260316-07cbfa88e29e805a.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![配置ASM的OS组](http://upload-images.jianshu.io/upload_images/5260316-a5628412ec878982.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![配置ORACLE_BASE和安装目录及ORACLE_HOME](http://upload-images.jianshu.io/upload_images/5260316-c76aa92e8e6f41b6.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![创建Invertory](http://upload-images.jianshu.io/upload_images/5260316-3ef495519be1c7ca.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![是否自动执行root脚本,默认](http://upload-images.jianshu.io/upload_images/5260316-44e2e68e4e5f78cb.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![检查系统是否还存在问题](http://upload-images.jianshu.io/upload_images/5260316-94b70a3b58108576.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

######解决问题:
- **cvuqdisk**包处理
`rpm -ivh /u01/app/soft/grid/stage/cvu/cv/remenv/cvuqdisk-1.0.9-1.rpm`,所有节点都执行。
- **panic_on_oops**系统内核参数处理,所有节点都执行。
```
echo 'kernel.panic_on_oops = 1' >> /etc/sysctl.conf
sysctl -p
```
- **zeroconf check**处理,所有节点都执行。
```
# cat >> /etc/sysconfig/network << EOF
NOZEROCONF=yes
EOF
```
- **Device Checks for ASM**
网上说可能是udev生成磁盘属组和第13步中的系统组不一致造成的,但经过查对没有不一致,该问题忽略,后面的安装没有出现相应的问题?!
- **其它问题,忽略即可。**


![重新检查,并忽略这些问题,生产环境要一一查明解决掉才可](http://upload-images.jianshu.io/upload_images/5260316-a5255e793d91c764.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![总结](http://upload-images.jianshu.io/upload_images/5260316-5e8bc966b16caf60.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![安装进程,在所有节点上执行root脚本](http://upload-images.jianshu.io/upload_images/5260316-5e19256472ee010e.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)
```
# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
```
```
# /u01/app/12.1.0/grid_1/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/12.1.0/grid_1

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/12.1.0/grid_1/crs/install/crsconfig_params
2017/05/18 13:39:15 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.

2017/05/18 13:39:44 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.

2017/05/18 13:39:45 CLSRSC-363: User ignored prerequisites during installation

OLR initialization - successful
  root wallet
  root wallet cert
  root cert export
  peer wallet
  profile reader wallet
  pa wallet
  peer wallet keys
  pa wallet keys
  peer cert request
  pa cert request
  peer cert
  pa cert
  peer root cert TP
  profile reader root cert TP
  pa root cert TP
  peer pa cert TP
  pa peer cert TP
  profile reader pa cert TP
  profile reader peer cert TP
  peer user cert
  pa user cert
2017/05/18 13:40:33 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.conf'

CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.evmd' on 'ora-node1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'ora-node1'
CRS-2676: Start of 'ora.mdnsd' on 'ora-node1' succeeded
CRS-2676: Start of 'ora.evmd' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'ora-node1'
CRS-2676: Start of 'ora.gpnpd' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ora-node1'
CRS-2672: Attempting to start 'ora.gipcd' on 'ora-node1'
CRS-2676: Start of 'ora.cssdmonitor' on 'ora-node1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'ora-node1'
CRS-2672: Attempting to start 'ora.diskmon' on 'ora-node1'
CRS-2676: Start of 'ora.diskmon' on 'ora-node1' succeeded
CRS-2676: Start of 'ora.cssd' on 'ora-node1' succeeded

ASM created and started successfully.

Disk Group OCR_VOTING created successfully.

CRS-2672: Attempting to start 'ora.crf' on 'ora-node1'
CRS-2672: Attempting to start 'ora.storage' on 'ora-node1'
CRS-2676: Start of 'ora.storage' on 'ora-node1' succeeded
CRS-2676: Start of 'ora.crf' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'ora-node1'
CRS-2676: Start of 'ora.crsd' on 'ora-node1' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk d280951c2a7d4f8cbff978c13aa956b4.
Successful addition of voting disk 1f43b0992d254fb4bf564cd383254107.
Successful addition of voting disk ff17d288f6ae4f25bf134fcc3a7542ab.
Successfully replaced voting disk group with +OCR_VOTING.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   d280951c2a7d4f8cbff978c13aa956b4 (/dev/asm-diskb) [OCR_VOTING]
 2. ONLINE   1f43b0992d254fb4bf564cd383254107 (/dev/asm-diskc) [OCR_VOTING]
 3. ONLINE   ff17d288f6ae4f25bf134fcc3a7542ab (/dev/asm-diskd) [OCR_VOTING]
Located 3 voting disk(s).
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ora-node1'
CRS-2673: Attempting to stop 'ora.crsd' on 'ora-node1'
CRS-2677: Stop of 'ora.crsd' on 'ora-node1' succeeded
CRS-2673: Attempting to stop 'ora.storage' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'ora-node1'
CRS-2677: Stop of 'ora.storage' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'ora-node1' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.evmd' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.asm' on 'ora-node1'
CRS-2677: Stop of 'ora.mdnsd' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.crf' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.asm' on 'ora-node1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'ora-node1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'ora-node1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'ora-node1'
CRS-2677: Stop of 'ora.cssd' on 'ora-node1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'ora-node1'
CRS-2677: Stop of 'ora.gipcd' on 'ora-node1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ora-node1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'ora-node1'
CRS-2672: Attempting to start 'ora.evmd' on 'ora-node1'
CRS-2676: Start of 'ora.mdnsd' on 'ora-node1' succeeded
CRS-2676: Start of 'ora.evmd' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'ora-node1'
CRS-2676: Start of 'ora.gpnpd' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'ora-node1'
CRS-2676: Start of 'ora.gipcd' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ora-node1'
CRS-2676: Start of 'ora.cssdmonitor' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'ora-node1'
CRS-2672: Attempting to start 'ora.diskmon' on 'ora-node1'
CRS-2676: Start of 'ora.diskmon' on 'ora-node1' succeeded
CRS-2676: Start of 'ora.cssd' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'ora-node1'
CRS-2672: Attempting to start 'ora.ctssd' on 'ora-node1'
CRS-2676: Start of 'ora.ctssd' on 'ora-node1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'ora-node1'
CRS-2676: Start of 'ora.asm' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'ora-node1'
CRS-2676: Start of 'ora.storage' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'ora-node1'
CRS-2676: Start of 'ora.crf' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'ora-node1'
CRS-2676: Start of 'ora.crsd' on 'ora-node1' succeeded
CRS-6023: Starting Oracle Cluster Ready Services-managed resources
CRS-6017: Processing resource auto-start for servers: ora-node1
CRS-6016: Resource auto-start has completed for server ora-node1
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2017/05/18 13:46:58 CLSRSC-343: Successfully started Oracle Clusterware stack

CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'ora-node1'
CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'ora-node1'
CRS-2676: Start of 'ora.asm' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.OCR_VOTING.dg' on 'ora-node1'
CRS-2676: Start of 'ora.OCR_VOTING.dg' on 'ora-node1' succeeded
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ora-node1'
CRS-2673: Attempting to stop 'ora.crsd' on 'ora-node1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'ora-node1'
CRS-2673: Attempting to stop 'ora.OCR_VOTING.dg' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.oc4j' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.cvu' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.ora-node1.vip' on 'ora-node1'
CRS-2677: Stop of 'ora.cvu' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.OCR_VOTING.dg' on 'ora-node1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'ora-node1'
CRS-2677: Stop of 'ora.asm' on 'ora-node1' succeeded
CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'ora-node1'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'ora-node1' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'ora-node1'
CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.ora-node1.vip' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.scan1.vip' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.oc4j' on 'ora-node1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'ora-node1'
CRS-2677: Stop of 'ora.ons' on 'ora-node1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'ora-node1'
CRS-2677: Stop of 'ora.net1.network' on 'ora-node1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'ora-node1' has completed
CRS-2677: Stop of 'ora.crsd' on 'ora-node1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.crf' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.evmd' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.storage' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'ora-node1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'ora-node1'
CRS-2677: Stop of 'ora.storage' on 'ora-node1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'ora-node1'
CRS-2677: Stop of 'ora.drivers.acfs' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.crf' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'ora-node1' succeeded
CRS-2677: Stop of 'ora.asm' on 'ora-node1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'ora-node1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'ora-node1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'ora-node1'
CRS-2677: Stop of 'ora.cssd' on 'ora-node1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'ora-node1'
CRS-2677: Stop of 'ora.gipcd' on 'ora-node1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ora-node1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'ora-node1'
CRS-2672: Attempting to start 'ora.evmd' on 'ora-node1'
CRS-2676: Start of 'ora.mdnsd' on 'ora-node1' succeeded
CRS-2676: Start of 'ora.evmd' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'ora-node1'
CRS-2676: Start of 'ora.gpnpd' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'ora-node1'
CRS-2676: Start of 'ora.gipcd' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ora-node1'
CRS-2676: Start of 'ora.cssdmonitor' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'ora-node1'
CRS-2672: Attempting to start 'ora.diskmon' on 'ora-node1'
CRS-2676: Start of 'ora.diskmon' on 'ora-node1' succeeded
CRS-2676: Start of 'ora.cssd' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'ora-node1'
CRS-2672: Attempting to start 'ora.ctssd' on 'ora-node1'
CRS-2676: Start of 'ora.ctssd' on 'ora-node1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'ora-node1'
CRS-2676: Start of 'ora.asm' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'ora-node1'
CRS-2676: Start of 'ora.storage' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'ora-node1'
CRS-2676: Start of 'ora.crf' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'ora-node1'
CRS-2676: Start of 'ora.crsd' on 'ora-node1' succeeded
CRS-6023: Starting Oracle Cluster Ready Services-managed resources
CRS-2664: Resource 'ora.OCR_VOTING.dg' is already running on 'ora-node1'
CRS-6017: Processing resource auto-start for servers: ora-node1
CRS-2672: Attempting to start 'ora.net1.network' on 'ora-node1'
CRS-2672: Attempting to start 'ora.oc4j' on 'ora-node1'
CRS-2676: Start of 'ora.net1.network' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.cvu' on 'ora-node1'
CRS-2672: Attempting to start 'ora.ons' on 'ora-node1'
CRS-2672: Attempting to start 'ora.ora-node1.vip' on 'ora-node1'
CRS-2672: Attempting to start 'ora.scan1.vip' on 'ora-node1'
CRS-2676: Start of 'ora.cvu' on 'ora-node1' succeeded
CRS-2676: Start of 'ora.ora-node1.vip' on 'ora-node1' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'ora-node1' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'ora-node1'
CRS-2676: Start of 'ora.ons' on 'ora-node1' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'ora-node1' succeeded
CRS-2676: Start of 'ora.oc4j' on 'ora-node1' succeeded
CRS-6016: Resource auto-start has completed for server ora-node1
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2017/05/18 13:50:33 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... Succeeded
```
![执行结果](http://upload-images.jianshu.io/upload_images/5260316-727795dda446cf9c.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)


![继续安装](http://upload-images.jianshu.io/upload_images/5260316-d1ddf35fa62ea323.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![完成](http://upload-images.jianshu.io/upload_images/5260316-d7bec019a471b9f4.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

######2.3 检查安装情况
```
# su - grid

$ crs_stat -t
Name           Type           Target    State     Host        
------------------------------------------------------------
ora....SM.lsnr ora....er.type ONLINE    ONLINE    ora-node1   
ora....ER.lsnr ora....er.type ONLINE    ONLINE    ora-node1   
ora....N1.lsnr ora....er.type ONLINE    ONLINE    ora-node1   
ora.MGMTLSNR   ora....nr.type ONLINE    ONLINE    ora-node1   
ora....TING.dg ora....up.type ONLINE    ONLINE    ora-node1   
ora.asm        ora.asm.type   ONLINE    ONLINE    ora-node1   
ora.cvu        ora.cvu.type   ONLINE    ONLINE    ora-node1   
ora.mgmtdb     ora....db.type ONLINE    ONLINE    ora-node1   
ora....network ora....rk.type ONLINE    ONLINE    ora-node1   
ora.oc4j       ora.oc4j.type  ONLINE    ONLINE    ora-node1   
ora.ons        ora.ons.type   ONLINE    ONLINE    ora-node1   
ora....E1.lsnr application    ONLINE    ONLINE    ora-node1   
ora....de1.ons application    ONLINE    ONLINE    ora-node1   
ora....de1.vip ora....t1.type ONLINE    ONLINE    ora-node1   
ora....E2.lsnr application    ONLINE    ONLINE    ora-node2   
ora....de2.ons application    ONLINE    ONLINE    ora-node2   
ora....de2.vip ora....t1.type ONLINE    ONLINE    ora-node2   
ora.scan1.vip  ora....ip.type ONLINE    ONLINE    ora-node1   
```
```
$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       ora-node1                STABLE
               ONLINE  ONLINE       ora-node2                STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       ora-node1                STABLE
               ONLINE  ONLINE       ora-node2                STABLE
ora.OCR_VOTING.dg
               ONLINE  ONLINE       ora-node1                STABLE
               ONLINE  ONLINE       ora-node2                STABLE
ora.net1.network
               ONLINE  ONLINE       ora-node1                STABLE
               ONLINE  ONLINE       ora-node2                STABLE
ora.ons
               ONLINE  ONLINE       ora-node1                STABLE
               ONLINE  ONLINE       ora-node2                STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       ora-node1                STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       ora-node1                169.254.48.110 172.1
                                                             16.0.81,STABLE
ora.asm
      1        ONLINE  ONLINE       ora-node1                Started,STABLE
      2        ONLINE  ONLINE       ora-node2                Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       ora-node1                STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       ora-node1                Open,STABLE
ora.oc4j
      1        ONLINE  ONLINE       ora-node1                STABLE
ora.ora-node1.vip
      1        ONLINE  ONLINE       ora-node1                STABLE
ora.ora-node2.vip
      1        ONLINE  ONLINE       ora-node2                STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       ora-node1                STABLE
--------------------------------------------------------------------------------
```

####3. 配置ASM磁盘组
```
$ asmca
```

![开始创建](http://upload-images.jianshu.io/upload_images/5260316-1d23f0ee66347f35.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![创建DATA磁盘组](http://upload-images.jianshu.io/upload_images/5260316-5b2eb5829722953f.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![创建成功](http://upload-images.jianshu.io/upload_images/5260316-daa48c1bd927f3d3.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![创建闪回FRA磁盘组](http://upload-images.jianshu.io/upload_images/5260316-304df1101116cca2.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

####4. 安装数据库软件
这里只是记录部分关键步骤。下载软件包,解压,运行`runInstall`。

![开始](http://upload-images.jianshu.io/upload_images/5260316-aef08740d3e4df8a.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![仅安装软件](http://upload-images.jianshu.io/upload_images/5260316-d324e811e3af4a99.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![安装RAC](http://upload-images.jianshu.io/upload_images/5260316-c9295da03607117e.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![选择节点](http://upload-images.jianshu.io/upload_images/5260316-50708edf9318e9be.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![配置BASE和HOME目录](http://upload-images.jianshu.io/upload_images/5260316-6bbfc818f30aff00.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![root脚本](http://upload-images.jianshu.io/upload_images/5260316-142e02f50796fbe7.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)
完成数据库软件的安装,下面安装数据库。

####5. 安装数据库
```
$ dbca
```

![数据库模板,Admin-Managed。还有一种是Policy-Managed的,安装后是在任意一个节点安装一个实例,这个还没太搞清楚,后期补充//TODO](http://upload-images.jianshu.io/upload_images/5260316-8104b5324386995d.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![配置实例名](http://upload-images.jianshu.io/upload_images/5260316-761f8d29d0ed0a4e.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![选择部署节点](http://upload-images.jianshu.io/upload_images/5260316-881a36aa41329329.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![确认](http://upload-images.jianshu.io/upload_images/5260316-26aa4e3f1ef81775.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![配置ServerPools](http://upload-images.jianshu.io/upload_images/5260316-9331dc8747d6ae67.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)


![配置CVU和EM](http://upload-images.jianshu.io/upload_images/5260316-79e74427c9028dac.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![管理员密码,Oracle_123](http://upload-images.jianshu.io/upload_images/5260316-2bedf709c7fa9a17.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![配置存储](http://upload-images.jianshu.io/upload_images/5260316-1cae933a9ca33387.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![初始化参数,注意字符集](http://upload-images.jianshu.io/upload_images/5260316-570b9262173c214c.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![检查,开始安装](http://upload-images.jianshu.io/upload_images/5260316-bd1938fa2bd7d026.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![安装完成](http://upload-images.jianshu.io/upload_images/5260316-d59a9d9cf6192b29.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

```
$ crsctl status res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       ora-node1                STABLE
               ONLINE  ONLINE       ora-node2                STABLE
ora.DATA.dg
               ONLINE  ONLINE       ora-node1                STABLE
               ONLINE  ONLINE       ora-node2                STABLE
ora.FRA.dg
               ONLINE  ONLINE       ora-node1                STABLE
               ONLINE  ONLINE       ora-node2                STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       ora-node1                STABLE
               ONLINE  ONLINE       ora-node2                STABLE
ora.OCR_VOTING.dg
               ONLINE  ONLINE       ora-node1                STABLE
               ONLINE  ONLINE       ora-node2                STABLE
ora.net1.network
               ONLINE  ONLINE       ora-node1                STABLE
               ONLINE  ONLINE       ora-node2                STABLE
ora.ons
               ONLINE  ONLINE       ora-node1                STABLE
               ONLINE  ONLINE       ora-node2                STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       ora-node1                STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       ora-node1                169.254.48.110 172.1
                                                             16.0.81,STABLE
ora.asm
      1        ONLINE  ONLINE       ora-node1                Started,STABLE
      2        ONLINE  ONLINE       ora-node2                Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       ora-node1                STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       ora-node1                Open,STABLE
ora.mkdb.db
      1        ONLINE  ONLINE       ora-node1                Open,STABLE
      2        ONLINE  ONLINE       ora-node2                Open,STABLE
ora.oc4j
      1        ONLINE  ONLINE       ora-node1                STABLE
ora.ora-node1.vip
      1        ONLINE  ONLINE       ora-node1                STABLE
ora.ora-node2.vip
      1        ONLINE  ONLINE       ora-node2                STABLE
ora.orcl.db
      1        OFFLINE OFFLINE                               Instance Shutdown,ST
                                                             ABLE
ora.scan1.vip
      1        ONLINE  ONLINE       ora-node1                STABLE
--------------------------------------------------------------------------------
```

![连接数据库](http://upload-images.jianshu.io/upload_images/5260316-30c23097f4e63eed.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

![查询数据库状态](http://upload-images.jianshu.io/upload_images/5260316-f405d626f27d6917.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)

你可能感兴趣的:(【Ora12c】基于RHEL6.6部署RAC)