使用UDEV在Oracle Linux 6上安装Oracle 11g RAC(11.2.0.3) (四)
使用UDEV在Oracle Linux 6上安装Oracle 11g RAC(11.2.0.3) (一)
使用UDEV在Oracle Linux 6上安装Oracle 11g RAC(11.2.0.3) (二)
将grid用户加入到dba组中
再次使用Cluster Verification Utility检查CRS安装环境
Skip software updates
node2
安装程序将运行 Oracle 网络服务 (NETCA)、自动存储管理 (ASMCA) 和 Oracle 专用互连 (VIPCA) 的配置助手。OUI 执行的最后一步是运行集群验证实用程序 (CVU)。如果配置助手和 CVU 运行成功，可单击 [Next] 然后单击 [Close] 退出 OUI。如果 CVU 只报告了此错误，可放心忽略此检查并继续。
node2
检查 CRS 状态
检查集群节点
检测CRS版本
检查 Oracle 集群注册表 (OCR)
检查votedisk
检查asm
至此GI安装完毕
使用UDEV在Oracle Linux 6上安装Oracle 11g RAC(11.2.0.3) (二)
使用UDEV在Oracle Linux 6上安装Oracle 11g RAC(11.2.0.3) (三)
将安装介质上传到node1,解压准备安装grid infrastructure
使用grid用户解压
p10404530_112030_Linux-x86-64_3of7.zip
使用Cluster Verification Utility检查CRS安装环境
[grid@node1 grid]$ ./runcluvfy.sh stage -pre crsinst -n node1,node2 -fixup -verbose Performing pre-checks for cluster services setup Checking node reachability... Check: Node reachability from node "node1" Destination Node Reachable? ------------------------------------ ------------------------ node1 yes node2 yes Result: Node reachability check passed from node "node1" Checking user equivalence... Check: User equivalence for user "grid" Node Name Status ------------------------------------ ------------------------ node2 passed node1 passed Result: User equivalence check passed for user "grid" Checking node connectivity... Checking hosts config file... Node Name Status ------------------------------------ ------------------------ node2 passed node1 passed Verification of the hosts config file successful Interface information for node "node2" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.52 192.168.0.0 0.0.0.0 192.168.1.254 00:0C:29:5C:FC:76 1500 eth1 172.168.1.52 172.168.0.0 0.0.0.0 192.168.1.254 00:0C:29:5C:FC:80 1500 Interface information for node "node1" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.51 192.168.0.0 0.0.0.0 192.168.1.254 00:0C:29:79:33:95 1500 eth1 172.168.1.51 172.168.0.0 0.0.0.0 192.168.1.254 00:0C:29:79:33:9F 1500 Check: Node connectivity of subnet "192.168.0.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- node2[192.168.1.52] node1[192.168.1.51] yes Result: Node connectivity passed for subnet "192.168.0.0" with node(s) node2,node1 Check: TCP connectivity of subnet "192.168.0.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- node1:192.168.1.51 node2:192.168.1.52 passed Result: TCP connectivity check passed for subnet "192.168.0.0" Check: Node connectivity of subnet "172.168.0.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- node2[172.168.1.52] node1[172.168.1.51] yes Result: Node connectivity passed for subnet "172.168.0.0" with node(s) node2,node1 Check: TCP connectivity of subnet "172.168.0.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- node1:172.168.1.51 node2:172.168.1.52 passed Result: TCP connectivity check passed for subnet "172.168.0.0" Interfaces found on subnet "192.168.0.0" that are likely candidates for VIP are: node2 eth0:192.168.1.52 node1 eth0:192.168.1.51 Interfaces found on subnet "172.168.0.0" that are likely candidates for VIP are: node2 eth1:172.168.1.52 node1 eth1:172.168.1.51 WARNING: Could not find a suitable set of interfaces for the private interconnect Checking subnet mask consistency... Subnet mask consistency check passed for subnet "192.168.0.0". Subnet mask consistency check passed for subnet "172.168.0.0". Subnet mask consistency check passed. Result: Node connectivity check passed Checking multicast communication... Checking subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0"... Check of subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0" passed. Checking subnet "172.168.0.0" for multicast communication with multicast group "230.0.1.0"... Check of subnet "172.168.0.0" for multicast communication with multicast group "230.0.1.0" passed. Check of multicast communication passed. Checking ASMLib configuration. Node Name Status ------------------------------------ ------------------------ node2 passed node1 passed Result: Check for ASMLib configuration passed. Check: Total memory Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 2.9462GB (3089356.0KB) 1.5GB (1572864.0KB) passed node1 2.9462GB (3089356.0KB) 1.5GB (1572864.0KB) passed Result: Total memory check passed Check: Available memory Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 2.7915GB (2927064.0KB) 50MB (51200.0KB) passed node1 2.6722GB (2801984.0KB) 50MB (51200.0KB) passed Result: Available memory check passed Check: Swap space Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 6GB (6291452.0KB) 2.9462GB (3089356.0KB) passed node1 6GB (6291452.0KB) 2.9462GB (3089356.0KB) passed Result: Swap space check passed Check: Free disk space for "node2:/tmp" Path Node Name Mount point Available Required Status ---------------- ------------ ------------ ------------ ------------ ------------ /tmp node2 / 88.7852GB 1GB passed Result: Free disk space check passed for "node2:/tmp" Check: Free disk space for "node1:/tmp" Path Node Name Mount point Available Required Status ---------------- ------------ ------------ ------------ ------------ ------------ /tmp node1 / 81.6788GB 1GB passed Result: Free disk space check passed for "node1:/tmp" Check: User existence for "grid" Node Name Status Comment ------------ ------------------------ ------------------------ node2 passed exists(1100) node1 passed exists(1100) Checking for multiple users with UID value 1100 Result: Check for multiple users with UID value 1100 passed Result: User existence check passed for "grid" Check: Group existence for "oinstall" Node Name Status Comment ------------ ------------------------ ------------------------ node2 passed exists node1 passed exists Result: Group existence check passed for "oinstall" Check: Group existence for "dba" Node Name Status Comment ------------ ------------------------ ------------------------ node2 passed exists node1 passed exists Result: Group existence check passed for "dba" Check: Membership of user "grid" in group "oinstall" [as Primary] Node Name User Exists Group Exists User in Group Primary Status ---------------- ------------ ------------ ------------ ------------ ------------ node2 yes yes yes yes passed node1 yes yes yes yes passed Result: Membership check for user "grid" in group "oinstall" [as Primary] passed Check: Membership of user "grid" in group "dba" Node Name User Exists Group Exists User in Group Status ---------------- ------------ ------------ ------------ ---------------- node2 yes yes no failed node1 yes yes no failed Result: Membership check for user "grid" in group "dba" failed Check: Run level Node Name run level Required Status ------------ ------------------------ ------------------------ ---------- node2 3 3,5 passed node1 5 3,5 passed Result: Run level check passed Check: Hard limits for "maximum open file descriptors" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- node2 hard 65536 65536 passed node1 hard 65536 65536 passed Result: Hard limits check passed for "maximum open file descriptors" Check: Soft limits for "maximum open file descriptors" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- node2 soft 1024 1024 passed node1 soft 1024 1024 passed Result: Soft limits check passed for "maximum open file descriptors" Check: Hard limits for "maximum user processes" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- node2 hard 16384 16384 passed node1 hard 16384 16384 passed Result: Hard limits check passed for "maximum user processes" Check: Soft limits for "maximum user processes" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- node2 soft 2047 2047 passed node1 soft 2047 2047 passed Result: Soft limits check passed for "maximum user processes" Check: System architecture Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 x86_64 x86_64 passed node1 x86_64 x86_64 passed Result: System architecture check passed Check: Kernel version Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 2.6.39-200.24.1.el6uek.x86_64 2.6.32 passed node1 2.6.39-200.24.1.el6uek.x86_64 2.6.32 passed Result: Kernel version check passed Check: Kernel parameter for "semmsl" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 250 250 250 passed node1 250 250 250 passed Result: Kernel parameter check passed for "semmsl" Check: Kernel parameter for "semmns" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 32000 32000 32000 passed node1 32000 32000 32000 passed Result: Kernel parameter check passed for "semmns" Check: Kernel parameter for "semopm" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 100 100 100 passed node1 100 100 100 passed Result: Kernel parameter check passed for "semopm" Check: Kernel parameter for "semmni" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 128 128 128 passed node1 128 128 128 passed Result: Kernel parameter check passed for "semmni" Check: Kernel parameter for "shmmax" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 4398046511104 4398046511104 1581750272 passed node1 4398046511104 4398046511104 1581750272 passed Result: Kernel parameter check passed for "shmmax" Check: Kernel parameter for "shmmni" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 4096 4096 4096 passed node1 4096 4096 4096 passed Result: Kernel parameter check passed for "shmmni" Check: Kernel parameter for "shmall" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 1073741824 1073741824 2097152 passed node1 1073741824 1073741824 2097152 passed Result: Kernel parameter check passed for "shmall" Check: Kernel parameter for "file-max" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 6815744 6815744 6815744 passed node1 6815744 6815744 6815744 passed Result: Kernel parameter check passed for "file-max" Check: Kernel parameter for "ip_local_port_range" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 between 9000.0 & 65500.0 between 9000.0 & 65500.0 between 9000.0 & 65500.0 passed node1 between 9000.0 & 65500.0 between 9000.0 & 65500.0 between 9000.0 & 65500.0 passed Result: Kernel parameter check passed for "ip_local_port_range" Check: Kernel parameter for "rmem_default" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 262144 262144 262144 passed node1 262144 262144 262144 passed Result: Kernel parameter check passed for "rmem_default" Check: Kernel parameter for "rmem_max" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 4194304 4194304 4194304 passed node1 4194304 4194304 4194304 passed Result: Kernel parameter check passed for "rmem_max" Check: Kernel parameter for "wmem_default" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 262144 262144 262144 passed node1 262144 262144 262144 passed Result: Kernel parameter check passed for "wmem_default" Check: Kernel parameter for "wmem_max" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 1048576 1048576 1048576 passed node1 1048576 1048576 1048576 passed Result: Kernel parameter check passed for "wmem_max" Check: Kernel parameter for "aio-max-nr" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 1048576 1048576 1048576 passed node1 1048576 1048576 1048576 passed Result: Kernel parameter check passed for "aio-max-nr" Check: Package existence for "binutils" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 binutils-2.20.51.0.2-5.34.el6 binutils-2.20.51.0.2 passed node1 binutils-2.20.51.0.2-5.34.el6 binutils-2.20.51.0.2 passed Result: Package existence check passed for "binutils" Check: Package existence for "compat-libcap1" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 compat-libcap1-1.10-1 compat-libcap1-1.10 passed node1 compat-libcap1-1.10-1 compat-libcap1-1.10 passed Result: Package existence check passed for "compat-libcap1" Check: Package existence for "compat-libstdc++-33(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 compat-libstdc++-33(x86_64)-3.2.3-69.el6 compat-libstdc++-33(x86_64)-3.2.3 passed node1 compat-libstdc++-33(x86_64)-3.2.3-69.el6 compat-libstdc++-33(x86_64)-3.2.3 passed Result: Package existence check passed for "compat-libstdc++-33(x86_64)" Check: Package existence for "libgcc(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 libgcc(x86_64)-4.4.6-4.el6 libgcc(x86_64)-4.4.4 passed node1 libgcc(x86_64)-4.4.6-4.el6 libgcc(x86_64)-4.4.4 passed Result: Package existence check passed for "libgcc(x86_64)" Check: Package existence for "libstdc++(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 libstdc++(x86_64)-4.4.6-4.el6 libstdc++(x86_64)-4.4.4 passed node1 libstdc++(x86_64)-4.4.6-4.el6 libstdc++(x86_64)-4.4.4 passed Result: Package existence check passed for "libstdc++(x86_64)" Check: Package existence for "libstdc++-devel(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 libstdc++-devel(x86_64)-4.4.6-4.el6 libstdc++-devel(x86_64)-4.4.4 passed node1 libstdc++-devel(x86_64)-4.4.6-4.el6 libstdc++-devel(x86_64)-4.4.4 passed Result: Package existence check passed for "libstdc++-devel(x86_64)" Check: Package existence for "sysstat" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 sysstat-9.0.4-20.el6 sysstat-9.0.4 passed node1 sysstat-9.0.4-20.el6 sysstat-9.0.4 passed Result: Package existence check passed for "sysstat" Check: Package existence for "gcc" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 gcc-4.4.6-4.el6 gcc-4.4.4 passed node1 gcc-4.4.6-4.el6 gcc-4.4.4 passed Result: Package existence check passed for "gcc" Check: Package existence for "gcc-c++" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 gcc-c++-4.4.6-4.el6 gcc-c++-4.4.4 passed node1 gcc-c++-4.4.6-4.el6 gcc-c++-4.4.4 passed Result: Package existence check passed for "gcc-c++" Check: Package existence for "ksh" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 ksh-20100621-16.el6 ksh-20100621 passed node1 ksh-20100621-16.el6 ksh-20100621 passed Result: Package existence check passed for "ksh" Check: Package existence for "make" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 make-3.81-20.el6 make-3.81 passed node1 make-3.81-20.el6 make-3.81 passed Result: Package existence check passed for "make" Check: Package existence for "glibc(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 glibc(x86_64)-2.12-1.80.el6_3.6 glibc(x86_64)-2.12 passed node1 glibc(x86_64)-2.12-1.80.el6_3.6 glibc(x86_64)-2.12 passed Result: Package existence check passed for "glibc(x86_64)" Check: Package existence for "glibc-devel(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 glibc-devel(x86_64)-2.12-1.80.el6_3.6 glibc-devel(x86_64)-2.12 passed node1 glibc-devel(x86_64)-2.12-1.80.el6_3.6 glibc-devel(x86_64)-2.12 passed Result: Package existence check passed for "glibc-devel(x86_64)" Check: Package existence for "libaio(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 libaio(x86_64)-0.3.107-10.el6 libaio(x86_64)-0.3.107 passed node1 libaio(x86_64)-0.3.107-10.el6 libaio(x86_64)-0.3.107 passed Result: Package existence check passed for "libaio(x86_64)" Check: Package existence for "libaio-devel(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 libaio-devel(x86_64)-0.3.107-10.el6 libaio-devel(x86_64)-0.3.107 passed node1 libaio-devel(x86_64)-0.3.107-10.el6 libaio-devel(x86_64)-0.3.107 passed Result: Package existence check passed for "libaio-devel(x86_64)" Checking for multiple users with UID value 0 Result: Check for multiple users with UID value 0 passed Check: Current group ID Result: Current group ID check passed Starting check for consistency of primary group of root user Node Name Status ------------------------------------ ------------------------ node2 passed node1 passed Check for consistency of root user's primary group passed Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... The NTP configuration file "/etc/ntp.conf" is available on all nodes NTP Configuration file check passed No NTP Daemons or Services were found to be running PRVF-5507 : NTP daemon or service is not running on any node but NTP configuration file exists on the following node(s): node2,node1 Result: Clock synchronization check using Network Time Protocol(NTP) failed Checking Core file name pattern consistency... Core file name pattern consistency check passed. Checking to make sure user "grid" is not in "root" group Node Name Status Comment ------------ ------------------------ ------------------------ node2 passed does not exist node1 passed does not exist Result: User "grid" is not part of "root" group. Check passed Check default user file creation mask Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- node2 0022 0022 passed node1 0022 0022 passed Result: Default user file creation mask check passed Checking consistency of file "/etc/resolv.conf" across nodes Checking the file "/etc/resolv.conf" to make sure only one of domain and search entries is defined File "/etc/resolv.conf" does not have both domain and search entries defined Checking if domain entry in file "/etc/resolv.conf" is consistent across the nodes... domain entry in file "/etc/resolv.conf" is consistent across nodes Checking if search entry in file "/etc/resolv.conf" is consistent across the nodes... search entry in file "/etc/resolv.conf" is consistent across nodes Checking file "/etc/resolv.conf" to make sure that only one search entry is defined All nodes have one search entry defined in file "/etc/resolv.conf" Checking all nodes to make sure that search entry is "localdomain" as found on node "node2" All nodes of the cluster have same value for 'search' Checking DNS response time for an unreachable node Node Name Status ------------------------------------ ------------------------ node2 passed node1 passed The DNS response time for an unreachable node is within acceptable limit on all nodes File "/etc/resolv.conf" is consistent across nodes Check: Time zone consistency Result: Time zone consistency check passed Fixup information has been generated for following node(s): node2,node1 Please run the following script on each node as "root" user to execute the fixups: '/tmp/CVU_11.2.0.3.0_grid/runfixup.sh' Pre-check for cluster services setup was unsuccessful on all the nodes.
发现有两个failed,一个是ntp,一个是grid用户不在dba组中
oracle11gR2 RAC中使用Cluster Time Synchronization Service(CTSS)同步各节点的时间,当安装程序发现 NTP 协议处于非活动状态时，安装集群时间同步服务将以活动模式自动进行安装并通过所有节点的时间。如果发现配置了 NTP，则以观察者模式 启动集群时间同步服务，Oracle Clusterware 不会在集群中进行活动的时间同步。
使用root用户在node1,node2中执行以下操作,关闭ntp服务,mv配置文件
[root@node2 ~]# service ntpd stop Shutting down ntpd: [FAILED] [root@node2 ~]# chkconfig ntpd off [root@node2 ~]# mv /etc/ntp.conf /etc/ntp.conf.original [root@node2 ~]# rm -rf /var/run/ntpd.pid [root@node2 ~]#
将grid用户加入到dba组中
[root@node1 ~]# id grid uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall),1200(asmadmin),1201(asmdba),1202(asmoper) [root@node1 ~]# /tmp/CVU_11.2.0.3.0_grid/runfixup.sh Response file being used is :/tmp/CVU_11.2.0.3.0_grid/fixup.response Enable file being used is :/tmp/CVU_11.2.0.3.0_grid/fixup.enable Log file location: /tmp/CVU_11.2.0.3.0_grid/orarun.log uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall),1200(asmadmin),1201(asmdba),1202(asmoper) [root@node1 ~]# id grid uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall),1200(asmadmin),1201(asmdba),1202(asmoper),1300(dba) [root@node1 ~]#
[root@node1 ~]# rpm -ivh /home/grid/grid/rpm/cvuqdisk-1.0.9-1.rpm Preparing... ########################################### [100%] Using default group oinstall to install package 1:cvuqdisk ########################################### [100%]
再次使用Cluster Verification Utility检查CRS安装环境
[grid@node1 grid]$ ./runcluvfy.sh stage -pre crsinst -n node1,node2 -fixup -verbose Performing pre-checks for cluster services setup Checking node reachability... Check: Node reachability from node "node1" Destination Node Reachable? ------------------------------------ ------------------------ node1 yes node2 yes Result: Node reachability check passed from node "node1" Checking user equivalence... Check: User equivalence for user "grid" Node Name Status ------------------------------------ ------------------------ node2 passed node1 passed Result: User equivalence check passed for user "grid" Checking node connectivity... Checking hosts config file... Node Name Status ------------------------------------ ------------------------ node2 passed node1 passed Verification of the hosts config file successful Interface information for node "node2" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.52 192.168.0.0 0.0.0.0 192.168.1.254 00:0C:29:5C:FC:76 1500 eth1 172.168.1.52 172.168.0.0 0.0.0.0 192.168.1.254 00:0C:29:5C:FC:80 1500 Interface information for node "node1" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 192.168.1.51 192.168.0.0 0.0.0.0 192.168.1.254 00:0C:29:79:33:95 1500 eth1 172.168.1.51 172.168.0.0 0.0.0.0 192.168.1.254 00:0C:29:79:33:9F 1500 Check: Node connectivity of subnet "192.168.0.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- node2[192.168.1.52] node1[192.168.1.51] yes Result: Node connectivity passed for subnet "192.168.0.0" with node(s) node2,node1 Check: TCP connectivity of subnet "192.168.0.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- node1:192.168.1.51 node2:192.168.1.52 passed Result: TCP connectivity check passed for subnet "192.168.0.0" Check: Node connectivity of subnet "172.168.0.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- node2[172.168.1.52] node1[172.168.1.51] yes Result: Node connectivity passed for subnet "172.168.0.0" with node(s) node2,node1 Check: TCP connectivity of subnet "172.168.0.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- node1:172.168.1.51 node2:172.168.1.52 passed Result: TCP connectivity check passed for subnet "172.168.0.0" Interfaces found on subnet "192.168.0.0" that are likely candidates for VIP are: node2 eth0:192.168.1.52 node1 eth0:192.168.1.51 Interfaces found on subnet "172.168.0.0" that are likely candidates for VIP are: node2 eth1:172.168.1.52 node1 eth1:172.168.1.51 WARNING: Could not find a suitable set of interfaces for the private interconnect Checking subnet mask consistency... Subnet mask consistency check passed for subnet "192.168.0.0". Subnet mask consistency check passed for subnet "172.168.0.0". Subnet mask consistency check passed. Result: Node connectivity check passed Checking multicast communication... Checking subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0"... Check of subnet "192.168.0.0" for multicast communication with multicast group "230.0.1.0" passed. Checking subnet "172.168.0.0" for multicast communication with multicast group "230.0.1.0"... Check of subnet "172.168.0.0" for multicast communication with multicast group "230.0.1.0" passed. Check of multicast communication passed. Checking ASMLib configuration. Node Name Status ------------------------------------ ------------------------ node2 passed node1 passed Result: Check for ASMLib configuration passed. Check: Total memory Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 2.9462GB (3089356.0KB) 1.5GB (1572864.0KB) passed node1 2.9462GB (3089356.0KB) 1.5GB (1572864.0KB) passed Result: Total memory check passed Check: Available memory Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 2.7901GB (2925668.0KB) 50MB (51200.0KB) passed node1 2.649GB (2777688.0KB) 50MB (51200.0KB) passed Result: Available memory check passed Check: Swap space Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 6GB (6291452.0KB) 2.9462GB (3089356.0KB) passed node1 6GB (6291452.0KB) 2.9462GB (3089356.0KB) passed Result: Swap space check passed Check: Free disk space for "node2:/tmp" Path Node Name Mount point Available Required Status ---------------- ------------ ------------ ------------ ------------ ------------ /tmp node2 / 88.7832GB 1GB passed Result: Free disk space check passed for "node2:/tmp" Check: Free disk space for "node1:/tmp" Path Node Name Mount point Available Required Status ---------------- ------------ ------------ ------------ ------------ ------------ /tmp node1 / 81.6679GB 1GB passed Result: Free disk space check passed for "node1:/tmp" Check: User existence for "grid" Node Name Status Comment ------------ ------------------------ ------------------------ node2 passed exists(1100) node1 passed exists(1100) Checking for multiple users with UID value 1100 Result: Check for multiple users with UID value 1100 passed Result: User existence check passed for "grid" Check: Group existence for "oinstall" Node Name Status Comment ------------ ------------------------ ------------------------ node2 passed exists node1 passed exists Result: Group existence check passed for "oinstall" Check: Group existence for "dba" Node Name Status Comment ------------ ------------------------ ------------------------ node2 passed exists node1 passed exists Result: Group existence check passed for "dba" Check: Membership of user "grid" in group "oinstall" [as Primary] Node Name User Exists Group Exists User in Group Primary Status ---------------- ------------ ------------ ------------ ------------ ------------ node2 yes yes yes yes passed node1 yes yes yes yes passed Result: Membership check for user "grid" in group "oinstall" [as Primary] passed Check: Membership of user "grid" in group "dba" Node Name User Exists Group Exists User in Group Status ---------------- ------------ ------------ ------------ ---------------- node2 yes yes yes passed node1 yes yes yes passed Result: Membership check for user "grid" in group "dba" passed Check: Run level Node Name run level Required Status ------------ ------------------------ ------------------------ ---------- node2 3 3,5 passed node1 5 3,5 passed Result: Run level check passed Check: Hard limits for "maximum open file descriptors" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- node2 hard 65536 65536 passed node1 hard 65536 65536 passed Result: Hard limits check passed for "maximum open file descriptors" Check: Soft limits for "maximum open file descriptors" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- node2 soft 1024 1024 passed node1 soft 1024 1024 passed Result: Soft limits check passed for "maximum open file descriptors" Check: Hard limits for "maximum user processes" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- node2 hard 16384 16384 passed node1 hard 16384 16384 passed Result: Hard limits check passed for "maximum user processes" Check: Soft limits for "maximum user processes" Node Name Type Available Required Status ---------------- ------------ ------------ ------------ ---------------- node2 soft 2047 2047 passed node1 soft 2047 2047 passed Result: Soft limits check passed for "maximum user processes" Check: System architecture Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 x86_64 x86_64 passed node1 x86_64 x86_64 passed Result: System architecture check passed Check: Kernel version Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 2.6.39-200.24.1.el6uek.x86_64 2.6.32 passed node1 2.6.39-200.24.1.el6uek.x86_64 2.6.32 passed Result: Kernel version check passed Check: Kernel parameter for "semmsl" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 250 250 250 passed node1 250 250 250 passed Result: Kernel parameter check passed for "semmsl" Check: Kernel parameter for "semmns" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 32000 32000 32000 passed node1 32000 32000 32000 passed Result: Kernel parameter check passed for "semmns" Check: Kernel parameter for "semopm" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 100 100 100 passed node1 100 100 100 passed Result: Kernel parameter check passed for "semopm" Check: Kernel parameter for "semmni" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 128 128 128 passed node1 128 128 128 passed Result: Kernel parameter check passed for "semmni" Check: Kernel parameter for "shmmax" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 4398046511104 4398046511104 1581750272 passed node1 4398046511104 4398046511104 1581750272 passed Result: Kernel parameter check passed for "shmmax" Check: Kernel parameter for "shmmni" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 4096 4096 4096 passed node1 4096 4096 4096 passed Result: Kernel parameter check passed for "shmmni" Check: Kernel parameter for "shmall" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 1073741824 1073741824 2097152 passed node1 1073741824 1073741824 2097152 passed Result: Kernel parameter check passed for "shmall" Check: Kernel parameter for "file-max" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 6815744 6815744 6815744 passed node1 6815744 6815744 6815744 passed Result: Kernel parameter check passed for "file-max" Check: Kernel parameter for "ip_local_port_range" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 between 9000.0 & 65500.0 between 9000.0 & 65500.0 between 9000.0 & 65500.0 passed node1 between 9000.0 & 65500.0 between 9000.0 & 65500.0 between 9000.0 & 65500.0 passed Result: Kernel parameter check passed for "ip_local_port_range" Check: Kernel parameter for "rmem_default" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 262144 262144 262144 passed node1 262144 262144 262144 passed Result: Kernel parameter check passed for "rmem_default" Check: Kernel parameter for "rmem_max" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 4194304 4194304 4194304 passed node1 4194304 4194304 4194304 passed Result: Kernel parameter check passed for "rmem_max" Check: Kernel parameter for "wmem_default" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 262144 262144 262144 passed node1 262144 262144 262144 passed Result: Kernel parameter check passed for "wmem_default" Check: Kernel parameter for "wmem_max" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 1048576 1048576 1048576 passed node1 1048576 1048576 1048576 passed Result: Kernel parameter check passed for "wmem_max" Check: Kernel parameter for "aio-max-nr" Node Name Current Configured Required Status Comment ---------------- ------------ ------------ ------------ ------------ ------------ node2 1048576 1048576 1048576 passed node1 1048576 1048576 1048576 passed Result: Kernel parameter check passed for "aio-max-nr" Check: Package existence for "binutils" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 binutils-2.20.51.0.2-5.34.el6 binutils-2.20.51.0.2 passed node1 binutils-2.20.51.0.2-5.34.el6 binutils-2.20.51.0.2 passed Result: Package existence check passed for "binutils" Check: Package existence for "compat-libcap1" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 compat-libcap1-1.10-1 compat-libcap1-1.10 passed node1 compat-libcap1-1.10-1 compat-libcap1-1.10 passed Result: Package existence check passed for "compat-libcap1" Check: Package existence for "compat-libstdc++-33(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 compat-libstdc++-33(x86_64)-3.2.3-69.el6 compat-libstdc++-33(x86_64)-3.2.3 passed node1 compat-libstdc++-33(x86_64)-3.2.3-69.el6 compat-libstdc++-33(x86_64)-3.2.3 passed Result: Package existence check passed for "compat-libstdc++-33(x86_64)" Check: Package existence for "libgcc(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 libgcc(x86_64)-4.4.6-4.el6 libgcc(x86_64)-4.4.4 passed node1 libgcc(x86_64)-4.4.6-4.el6 libgcc(x86_64)-4.4.4 passed Result: Package existence check passed for "libgcc(x86_64)" Check: Package existence for "libstdc++(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 libstdc++(x86_64)-4.4.6-4.el6 libstdc++(x86_64)-4.4.4 passed node1 libstdc++(x86_64)-4.4.6-4.el6 libstdc++(x86_64)-4.4.4 passed Result: Package existence check passed for "libstdc++(x86_64)" Check: Package existence for "libstdc++-devel(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 libstdc++-devel(x86_64)-4.4.6-4.el6 libstdc++-devel(x86_64)-4.4.4 passed node1 libstdc++-devel(x86_64)-4.4.6-4.el6 libstdc++-devel(x86_64)-4.4.4 passed Result: Package existence check passed for "libstdc++-devel(x86_64)" Check: Package existence for "sysstat" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 sysstat-9.0.4-20.el6 sysstat-9.0.4 passed node1 sysstat-9.0.4-20.el6 sysstat-9.0.4 passed Result: Package existence check passed for "sysstat" Check: Package existence for "gcc" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 gcc-4.4.6-4.el6 gcc-4.4.4 passed node1 gcc-4.4.6-4.el6 gcc-4.4.4 passed Result: Package existence check passed for "gcc" Check: Package existence for "gcc-c++" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 gcc-c++-4.4.6-4.el6 gcc-c++-4.4.4 passed node1 gcc-c++-4.4.6-4.el6 gcc-c++-4.4.4 passed Result: Package existence check passed for "gcc-c++" Check: Package existence for "ksh" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 ksh-20100621-16.el6 ksh-20100621 passed node1 ksh-20100621-16.el6 ksh-20100621 passed Result: Package existence check passed for "ksh" Check: Package existence for "make" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 make-3.81-20.el6 make-3.81 passed node1 make-3.81-20.el6 make-3.81 passed Result: Package existence check passed for "make" Check: Package existence for "glibc(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 glibc(x86_64)-2.12-1.80.el6_3.6 glibc(x86_64)-2.12 passed node1 glibc(x86_64)-2.12-1.80.el6_3.6 glibc(x86_64)-2.12 passed Result: Package existence check passed for "glibc(x86_64)" Check: Package existence for "glibc-devel(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 glibc-devel(x86_64)-2.12-1.80.el6_3.6 glibc-devel(x86_64)-2.12 passed node1 glibc-devel(x86_64)-2.12-1.80.el6_3.6 glibc-devel(x86_64)-2.12 passed Result: Package existence check passed for "glibc-devel(x86_64)" Check: Package existence for "libaio(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 libaio(x86_64)-0.3.107-10.el6 libaio(x86_64)-0.3.107 passed node1 libaio(x86_64)-0.3.107-10.el6 libaio(x86_64)-0.3.107 passed Result: Package existence check passed for "libaio(x86_64)" Check: Package existence for "libaio-devel(x86_64)" Node Name Available Required Status ------------ ------------------------ ------------------------ ---------- node2 libaio-devel(x86_64)-0.3.107-10.el6 libaio-devel(x86_64)-0.3.107 passed node1 libaio-devel(x86_64)-0.3.107-10.el6 libaio-devel(x86_64)-0.3.107 passed Result: Package existence check passed for "libaio-devel(x86_64)" Checking for multiple users with UID value 0 Result: Check for multiple users with UID value 0 passed Check: Current group ID Result: Current group ID check passed Starting check for consistency of primary group of root user Node Name Status ------------------------------------ ------------------------ node2 passed node1 passed Check for consistency of root user's primary group passed Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle Cluster Time Synchronization Service(CTSS) can be used instead of NTP for time synchronization on the cluster nodes No NTP Daemons or Services were found to be running Result: Clock synchronization check using Network Time Protocol(NTP) passed Checking Core file name pattern consistency... Core file name pattern consistency check passed. Checking to make sure user "grid" is not in "root" group Node Name Status Comment ------------ ------------------------ ------------------------ node2 passed does not exist node1 passed does not exist Result: User "grid" is not part of "root" group. Check passed Check default user file creation mask Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- node2 0022 0022 passed node1 0022 0022 passed Result: Default user file creation mask check passed Checking consistency of file "/etc/resolv.conf" across nodes Checking the file "/etc/resolv.conf" to make sure only one of domain and search entries is defined File "/etc/resolv.conf" does not have both domain and search entries defined Checking if domain entry in file "/etc/resolv.conf" is consistent across the nodes... domain entry in file "/etc/resolv.conf" is consistent across nodes Checking if search entry in file "/etc/resolv.conf" is consistent across the nodes... search entry in file "/etc/resolv.conf" is consistent across nodes Checking file "/etc/resolv.conf" to make sure that only one search entry is defined All nodes have one search entry defined in file "/etc/resolv.conf" Checking all nodes to make sure that search entry is "localdomain" as found on node "node2" All nodes of the cluster have same value for 'search' Checking DNS response time for an unreachable node Node Name Status ------------------------------------ ------------------------ node2 passed node1 passed The DNS response time for an unreachable node is within acceptable limit on all nodes File "/etc/resolv.conf" is consistent across nodes Check: Time zone consistency Result: Time zone consistency check passed Pre-check for cluster services setup was successful.
在node1上使用grid用户安装grid infrastructure
[grid@node1 grid]$ ./runInstaller Starting Oracle Universal Installer... Checking Temp space: must be greater than 120 MB. Actual 79780 MB Passed Checking swap space: must be greater than 150 MB. Actual 6143 MB Passed Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed Preparing to launch Oracle Universal Installer from /tmp/OraInstall2012-12-29_09-28-08AM. Please wait ...[grid@node1 grid]$
Skip software updates
Install any Configure Oracle Grid Infrastructure for a Cluster
Advanced Installation
add Simplified Chinese
不配置GNS,填写scan名称
add node2
如果之前没有配置ssh用户等效,也可以在这里配置
Validating remote binaries..
Remote binaries check succeeded
指定用于Public网络和Private网络的网络接口。进行所需更改以与下表中各值保持一致
配置ASM存储
+CRS
Use same passwords for these accounts
这里警告密码过于简单
Do not use Intelligent Platform Management Interface (IPMI)
ASM使用的是UDEV绑定设备,没有使用asmlib,忽略这个警告
Install
使用root用户在node1,node2上执行以下脚本,这里使用 one by one方式
node1
[root@node1 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[root@node1 ~]# /u01/app/11.2.0/grid/root.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
OLR initialization - successful
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Adding Clusterware entries to upstart
CRS-2672: Attempting to start 'ora.mdnsd' on 'node1'
CRS-2676: Start of 'ora.mdnsd' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'node1'
CRS-2676: Start of 'ora.gpnpd' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'node1'
CRS-2672: Attempting to start 'ora.gipcd' on 'node1'
CRS-2676: Start of 'ora.cssdmonitor' on 'node1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'node1'
CRS-2672: Attempting to start 'ora.diskmon' on 'node1'
CRS-2676: Start of 'ora.diskmon' on 'node1' succeeded
CRS-2676: Start of 'ora.cssd' on 'node1' succeeded
ASM created and started successfully.
Disk Group CRS created successfully.
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk 4b4ef03676d84facbf55c02b8c058a07.
Successfully replaced voting disk group with +CRS.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 4b4ef03676d84facbf55c02b8c058a07 (/dev/asm-diskb) [CRS]
Located 1 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on 'node1'
CRS-2676: Start of 'ora.asm' on 'node1' succeeded
CRS-2672: Attempting to start 'ora.CRS.dg' on 'node1'
CRS-2676: Start of 'ora.CRS.dg' on 'node1' succeeded
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@node1 ~]#
node2
[root@node2 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[root@node2 ~]# /u01/app/11.2.0/grid/root.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
OLR initialization - successful
Adding Clusterware entries to upstart
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node node1, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@node2 ~]#
安装程序将运行 Oracle 网络服务 (NETCA)、自动存储管理 (ASMCA) 和 Oracle 专用互连 (VIPCA) 的配置助手。OUI 执行的最后一步是运行集群验证实用程序 (CVU)。如果配置助手和 CVU 运行成功，可单击 [Next] 然后单击 [Close] 退出 OUI。如果 CVU 只报告了此错误，可放心忽略此检查并继续。
close
测试GI的安装
node1
[root@node1 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:79:33:95
inet addr:192.168.1.51 Bcast:192.168.255.255 Mask:255.255.0.0
inet6 addr: fe80::20c:29ff:fe79:3395/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:977978 errors:0 dropped:1345 overruns:0 frame:0
TX packets:2525875 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:106995897 (102.0 MiB) TX bytes:3573509233 (3.3 GiB)
eth0:1 Link encap:Ethernet HWaddr 00:0C:29:79:33:95
inet addr:192.168.1.151 Bcast:192.168.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth0:3 Link encap:Ethernet HWaddr 00:0C:29:79:33:95
inet addr:192.168.1.58 Bcast:192.168.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth0:4 Link encap:Ethernet HWaddr 00:0C:29:79:33:95
inet addr:192.168.1.59 Bcast:192.168.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth1 Link encap:Ethernet HWaddr 00:0C:29:79:33:9F
inet addr:172.168.1.51 Bcast:172.168.255.255 Mask:255.255.0.0
inet6 addr: fe80::20c:29ff:fe79:339f/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:728960 errors:0 dropped:1345 overruns:0 frame:0
TX packets:13833 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:54104908 (51.5 MiB) TX bytes:7561084 (7.2 MiB)
eth1:1 Link encap:Ethernet HWaddr 00:0C:29:79:33:9F
inet addr:169.254.201.146 Bcast:169.254.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:13162 errors:0 dropped:0 overruns:0 frame:0
TX packets:13162 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:7783412 (7.4 MiB) TX bytes:7783412 (7.4 MiB)
[root@node1 ~]# ps -ef|egrep -i "asm|listener"
grid 24390 1 0 10:03 ? 00:00:00 asm_pmon_+ASM1
grid 24392 1 0 10:03 ? 00:00:00 asm_psp0_+ASM1
grid 24394 1 1 10:03 ? 00:00:18 asm_vktm_+ASM1
grid 24398 1 0 10:03 ? 00:00:00 asm_gen0_+ASM1
grid 24400 1 0 10:03 ? 00:00:00 asm_diag_+ASM1
grid 24402 1 0 10:03 ? 00:00:00 asm_ping_+ASM1
grid 24404 1 0 10:03 ? 00:00:02 asm_dia0_+ASM1
grid 24406 1 0 10:03 ? 00:00:02 asm_lmon_+ASM1
grid 24408 1 0 10:03 ? 00:00:01 asm_lmd0_+ASM1
grid 24410 1 0 10:03 ? 00:00:02 asm_lms0_+ASM1
grid 24414 1 0 10:03 ? 00:00:00 asm_lmhb_+ASM1
grid 24416 1 0 10:03 ? 00:00:00 asm_mman_+ASM1
grid 24418 1 0 10:03 ? 00:00:00 asm_dbw0_+ASM1
grid 24420 1 0 10:03 ? 00:00:00 asm_lgwr_+ASM1
grid 24422 1 0 10:03 ? 00:00:00 asm_ckpt_+ASM1
grid 24424 1 0 10:03 ? 00:00:00 asm_smon_+ASM1
grid 24426 1 0 10:03 ? 00:00:00 asm_rbal_+ASM1
grid 24428 1 0 10:03 ? 00:00:00 asm_gmon_+ASM1
grid 24430 1 0 10:03 ? 00:00:00 asm_mmon_+ASM1
grid 24432 1 0 10:03 ? 00:00:00 asm_mmnl_+ASM1
grid 24434 1 0 10:03 ? 00:00:00 asm_lck0_+ASM1
grid 24436 1 0 10:03 ? 00:00:00 oracle+ASM1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
grid 24466 1 0 10:03 ? 00:00:01 oracle+ASM1_ocr (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
grid 24471 1 0 10:03 ? 00:00:00 asm_asmb_+ASM1
grid 24473 1 0 10:03 ? 00:00:00 oracle+ASM1_asmb_+asm1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
grid 24876 1 0 10:04 ? 00:00:00 oracle+ASM1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
grid 25269 1 0 10:05 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN2 -inherit
grid 25283 1 0 10:05 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN3 -inherit
grid 26105 1 0 10:15 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
grid 28183 28182 0 10:21 ? 00:00:00 oracle+ASM1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
root 28263 2146 0 10:26 pts/2 00:00:00 egrep -i asm|listener
node2
[root@node2 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:5C:FC:76
inet addr:192.168.1.52 Bcast:192.168.255.255 Mask:255.255.0.0
inet6 addr: fe80::20c:29ff:fe5c:fc76/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3068626 errors:0 dropped:1348 overruns:0 frame:0
TX packets:185731 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3505670277 (3.2 GiB) TX bytes:39520990 (37.6 MiB)
eth0:1 Link encap:Ethernet HWaddr 00:0C:29:5C:FC:76
inet addr:192.168.1.57 Bcast:192.168.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth0:2 Link encap:Ethernet HWaddr 00:0C:29:5C:FC:76
inet addr:192.168.1.152 Bcast:192.168.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth1 Link encap:Ethernet HWaddr 00:0C:29:5C:FC:80
inet addr:172.168.1.52 Bcast:172.168.255.255 Mask:255.255.0.0
inet6 addr: fe80::20c:29ff:fe5c:fc80/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:729233 errors:0 dropped:1348 overruns:0 frame:0
TX packets:15630 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:53620798 (51.1 MiB) TX bytes:8883597 (8.4 MiB)
eth1:1 Link encap:Ethernet HWaddr 00:0C:29:5C:FC:80
inet addr:169.254.30.23 Bcast:169.254.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:6049 errors:0 dropped:0 overruns:0 frame:0
TX packets:6049 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2377782 (2.2 MiB) TX bytes:2377782 (2.2 MiB)
[root@node2 ~]# ps -ef|egrep -i "asm|listener"
grid 21049 1 0 10:09 ? 00:00:00 asm_pmon_+ASM2
grid 21051 1 0 10:09 ? 00:00:00 asm_psp0_+ASM2
grid 21053 1 1 10:09 ? 00:00:14 asm_vktm_+ASM2
grid 21057 1 0 10:09 ? 00:00:00 asm_gen0_+ASM2
grid 21059 1 0 10:09 ? 00:00:00 asm_diag_+ASM2
grid 21061 1 0 10:09 ? 00:00:00 asm_ping_+ASM2
grid 21063 1 0 10:09 ? 00:00:01 asm_dia0_+ASM2
grid 21065 1 0 10:09 ? 00:00:01 asm_lmon_+ASM2
grid 21067 1 0 10:09 ? 00:00:00 asm_lmd0_+ASM2
grid 21069 1 0 10:09 ? 00:00:02 asm_lms0_+ASM2
grid 21073 1 0 10:09 ? 00:00:00 asm_lmhb_+ASM2
grid 21075 1 0 10:09 ? 00:00:00 asm_mman_+ASM2
grid 21077 1 0 10:09 ? 00:00:00 asm_dbw0_+ASM2
grid 21079 1 0 10:09 ? 00:00:00 asm_lgwr_+ASM2
grid 21081 1 0 10:09 ? 00:00:00 asm_ckpt_+ASM2
grid 21083 1 0 10:09 ? 00:00:00 asm_smon_+ASM2
grid 21085 1 0 10:09 ? 00:00:00 asm_rbal_+ASM2
grid 21087 1 0 10:09 ? 00:00:00 asm_gmon_+ASM2
grid 21089 1 0 10:09 ? 00:00:00 asm_mmon_+ASM2
grid 21091 1 0 10:09 ? 00:00:00 asm_mmnl_+ASM2
grid 21093 1 0 10:09 ? 00:00:00 asm_lck0_+ASM2
grid 21095 1 0 10:09 ? 00:00:00 oracle+ASM2 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
grid 21128 1 0 10:09 ? 00:00:00 oracle+ASM2_ocr (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
grid 21130 1 0 10:09 ? 00:00:00 asm_asmb_+ASM2
grid 21132 1 0 10:09 ? 00:00:00 oracle+ASM2_asmb_+asm2 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
grid 21271 1 0 10:09 ? 00:00:00 oracle+ASM2 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
grid 21326 1 0 10:09 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN1 -inherit
grid 22068 1 0 10:15 ? 00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
root 23551 1979 0 10:26 pts/2 00:00:00 egrep -i asm|listener
检查 CRS 状态
[grid@node2 ~]$ crsctl check crs CRS-4638: Oracle High Availability Services is online CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online
[grid@node2 ~]$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.CRS.dg ora....up.type ONLINE ONLINE node1 ora....ER.lsnr ora....er.type ONLINE ONLINE node1 ora....N1.lsnr ora....er.type ONLINE ONLINE node2 ora....N2.lsnr ora....er.type ONLINE ONLINE node1 ora....N3.lsnr ora....er.type ONLINE ONLINE node1 ora.asm ora.asm.type ONLINE ONLINE node1 ora.cvu ora.cvu.type ONLINE ONLINE node1 ora.gsd ora.gsd.type OFFLINE OFFLINE ora....network ora....rk.type ONLINE ONLINE node1 ora....SM1.asm application ONLINE ONLINE node1 ora....E1.lsnr application ONLINE ONLINE node1 ora.node1.gsd application OFFLINE OFFLINE ora.node1.ons application ONLINE ONLINE node1 ora.node1.vip ora....t1.type ONLINE ONLINE node1 ora....SM2.asm application ONLINE ONLINE node2 ora....E2.lsnr application ONLINE ONLINE node2 ora.node2.gsd application OFFLINE OFFLINE ora.node2.ons application ONLINE ONLINE node2 ora.node2.vip ora....t1.type ONLINE ONLINE node2 ora.oc4j ora.oc4j.type ONLINE ONLINE node1 ora.ons ora.ons.type ONLINE ONLINE node1 ora.scan1.vip ora....ip.type ONLINE ONLINE node2 ora.scan2.vip ora....ip.type ONLINE ONLINE node1 ora.scan3.vip ora....ip.type ONLINE ONLINE node1
[grid@node2 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.CRS.dg
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.LISTENER.lsnr
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.asm
ONLINE ONLINE node1 Started
ONLINE ONLINE node2 Started
ora.gsd
OFFLINE OFFLINE node1
OFFLINE OFFLINE node2
ora.net1.network
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.ons
ONLINE ONLINE node1
ONLINE ONLINE node2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE node2
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE node1
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE node1
ora.cvu
1 ONLINE ONLINE node1
ora.node1.vip
1 ONLINE ONLINE node1
ora.node2.vip
1 ONLINE ONLINE node2
ora.oc4j
1 ONLINE ONLINE node1
ora.scan1.vip
1 ONLINE ONLINE node2
ora.scan2.vip
1 ONLINE ONLINE node1
ora.scan3.vip
1 ONLINE ONLINE node1
检查集群节点
[grid@node2 ~]$ olsnodes -n node1 1 node2 2
检测CRS版本
[grid@node2 ~]$ crsctl query crs activeversion Oracle Clusterware active version on the cluster is [11.2.0.3.0]
检查 Oracle 集群注册表 (OCR)
[grid@node2 ~]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2588
Available space (kbytes) : 259532
ID : 1606856820
Device/File Name : +CRS
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check bypassed due to non-privileged user
检查votedisk
[grid@node2 ~]$ crsctl query css votedisk ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 4b4ef03676d84facbf55c02b8c058a07 (/dev/asm-diskc) [CRS] Located 1 voting disk(s).
检查asm
[grid@node2 ~]$ srvctl config asm -a ASM home: /u01/app/11.2.0/grid ASM listener: LISTENER ASM is enabled. [grid@node2 ~]$ srvctl status asm ASM is running on node2,node1
[grid@node2 ~]$ uname -p x86_64 [grid@node2 ~]$ sqlplus / as sysdba SQL*Plus: Release 11.2.0.3.0 Production on Sat Dec 29 10:45:13 2012 Copyright (c) 1982, 2011, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production With the Real Application Clusters and Automatic Storage Management options SQL> set linesize 100 SQL> show parameter spfile NAME TYPE VALUE ------------------------------------ ---------------------- ------------------------------ spfile string +CRS/cluster-scan/asmparameter file/registry.253.803296901 SQL> select path from v$asm_disk; PATH ---------------------------------------------------------------------------------------------------- /dev/asm-diskg /dev/asm-diskf /dev/asm-diske /dev/asm-diskb /dev/asm-diskc /dev/asm-diskd 6 rows selected.
至此GI安装完毕