【Oracle】11gR2 RAC增加interconnect网卡冗余
在之前的Oracle版本中如果要对interconnect网卡进行冗余，只能使用多网卡绑定的手法，而在Oracle 11.2.0.2版本的RAC中推出了Redundant Interconnect with Highly Available IP(HAIP)，无需通过网卡绑定即可实现对于interconnect网卡的冗余。
对于HAIP的多网卡设置可以有两种方式：
1.在安装GRID时通过图形界面进行设置，添加多块网卡为private：
2.集群安装完毕后手工添加。
接下来进行对于手工添加进行测试：
目前网络环境信息：
[grid@rac1 ~]$ oifcfg getif
eth0 192.168.90.0 global public
eth1 192.168.92.0 global cluster_interconnect
对于HAIP服务，默认该服务为开启状态，使用169.254.*.*网段：
[grid@rac1 ~]$ oifcfg iflist
eth0 192.168.90.0
eth1 192.168.92.0
eth1 169.254.0.0
登陆到ASM可以通过视图gv$cluster_interconnects进行查看：
SQL> select * from gv$cluster_interconnects;
INST_ID NAME IP_ADDRESS IS_ SOURCE
---------- --------------- ---------------- --- -------------------------------
1 eth1:1 169.254.100.222 NO
2 eth1:1 169.254.144.235 NO
集群资源中也可以看到HAIP为ONLINE状态：
[grid@rac1 ~]$ crsctl stat res -t -init
......
ora.cluster_interconnect.haip 1 ONLINE ONLINE rac1
......
现在对两个节点都添加网卡eth2，网段为192.168.94.*
使用oifcfg命令增加eth2为cluster_interconnct：
[grid@rac1 ~]$ oifcfg setif -global eth2/192.168.94.0:cluster_interconnect
[grid@rac1 ~]$ oifcfg getif
eth0 192.168.90.0 global public
eth1 192.168.92.0 global cluster_interconnect
eth2 192.168.94.0 global cluster_interconnect
之后节点1、节点2重启CRS（注意要一起进行重启，不能重启完一个再重启另一个），然后再进行查看：
SQL> select * from gv$cluster_interconnects;
INST_ID NAME IP_ADDRESS IS_ SOURCE
---------- --------------- ---------------- --- -------------------------------
1 eth1:1 169.254.97.55 NO
1 eth2:1 169.254.211.162 NO
2 eth1:1 169.254.1.211 NO
2 eth2:1 169.254.246.51 NO
SQL>
使用oifcfg查看：
[grid@rac1 ~]$ oifcfg iflist
eth0 192.168.90.0
eth1 192.168.92.0
eth1 169.254.0.0
eth2 192.168.94.0
eth2 169.254.128.0
可以看到现在inserconnect已经开始使用eth2了。
下面对单interconnect网卡故障进行模拟：
[root@rac1 bin]# ifdown eth1
可以看到集群并没有因为eth1故障而导致节点重启，接下来用ifconfig查看一下：
[root@rac1 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:B4:0F:83
inet addr:192.168.90.101 Bcast:192.168.90.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:feb4:f83/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:214028 errors:0 dropped:0 overruns:0 frame:0
TX packets:203412 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:245928964 (234.5 MiB) TX bytes:84690019 (80.7 MiB)
eth0:1 Link encap:Ethernet HWaddr 00:0C:29:B4:0F:83
inet addr:192.168.90.105 Bcast:192.168.90.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth0:2 Link encap:Ethernet HWaddr 00:0C:29:B4:0F:83
inet addr:192.168.90.103 Bcast:192.168.90.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth2 Link encap:Ethernet HWaddr 00:0C:29:B4:0F:97
inet addr:192.168.94.101 Bcast:192.168.94.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:feb4:f97/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:11658 errors:0 dropped:0 overruns:0 frame:0
TX packets:10887 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:5133560 (4.8 MiB) TX bytes:4642838 (4.4 MiB)
eth2:1 Link encap:Ethernet HWaddr 00:0C:29:B4:0F:97
inet addr:169.254.97.55 Bcast:169.254.127.255 Mask:255.255.128.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth2:2 Link encap:Ethernet HWaddr 00:0C:29:B4:0F:97
inet addr:169.254.211.162 Bcast:169.254.255.255 Mask:255.255.128.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
可以看到eth1的虚拟地址169.254.97.55已经漂移到eth2网卡上。
之后我们重启eth1网卡，模拟故障恢复：
[root@rac1 ~]# ifup eth1
[root@rac1 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:B4:0F:83
inet addr:192.168.90.101 Bcast:192.168.90.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:feb4:f83/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:216798 errors:0 dropped:0 overruns:0 frame:0
TX packets:207529 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:247023259 (235.5 MiB) TX bytes:86140902 (82.1 MiB)
eth0:1 Link encap:Ethernet HWaddr 00:0C:29:B4:0F:83
inet addr:192.168.90.105 Bcast:192.168.90.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth0:2 Link encap:Ethernet HWaddr 00:0C:29:B4:0F:83
inet addr:192.168.90.103 Bcast:192.168.90.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth1 Link encap:Ethernet HWaddr 00:0C:29:B4:0F:8D
inet addr:192.168.92.101 Bcast:192.168.92.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:feb4:f8d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:92270 errors:0 dropped:0 overruns:0 frame:0
TX packets:107004 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:54674383 (52.1 MiB) TX bytes:74489358 (71.0 MiB)
eth1:1 Link encap:Ethernet HWaddr 00:0C:29:B4:0F:8D
inet addr:169.254.97.55 Bcast:169.254.127.255 Mask:255.255.128.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
eth2 Link encap:Ethernet HWaddr 00:0C:29:B4:0F:97
inet addr:192.168.94.101 Bcast:192.168.94.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:feb4:f97/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:14772 errors:0 dropped:0 overruns:0 frame:0
TX packets:14036 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6763629 (6.4 MiB) TX bytes:6462040 (6.1 MiB)
eth2:2 Link encap:Ethernet HWaddr 00:0C:29:B4:0F:97
inet addr:169.254.211.162 Bcast:169.254.255.255 Mask:255.255.128.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:39833 errors:0 dropped:0 overruns:0 frame:0
TX packets:39833 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:28356186 (27.0 MiB) TX bytes:28356186 (27.0 MiB)
可以看到ip 169.254.97.55已经回到eth1网卡上，整个过程中RAC正常运行。