How to Change Interconnect/Public Interface IP or Subnet in Oracle Clusterware [

Applies to:

Oracle Server - Enterprise Edition - Version: 10.1.0.2 to 11.1.0.8
Information in this document applies to any platform.

Goal

Purpose:

The purpose of this note is to describe how to change or update the cluster_interconnect and/or public interface IP information that is stored in the OCR.  The interface defined as 'public'  will be the interface used to bind the VIP to, whereas the interface defined as 'cluster_interconnect'  will be the default interface used by any RDBMS or ASM instances for cache fusion traffic.  

This note is not intended as a means to change the Public or Private Hostnames themselves.  Public or Private hostnames can only be changed by removing/adding nodes, or reinstalling Oracle Clusterware.   VIP Hostnames can be changed, however.  To change a VIP Hostname, please refer to Note 276434.1 .

 

It may be necessary to change or update interface names, or the IP information associated with an interface if there is a network change affecting the servers, or if the original information that was input during the installation was incorrect.   It may also be the case that for some reason, the Oracle Interface Configuration Assistant  ('oifcfg')  did not succeeed during the installation.

In any of these cases, the 'oifcfg' utility can be used to remove incorrect entries that are stored in the OCR, and to add the correct information back into the OCR.

 

Note: The value stored for the cluster_interconnect via oifcfg  is meant to be used by RDBMS and ASM instances only.  Oracle  Clusterware does not use these values for the interconnect.  Oracle Clusterware uses the private nodenames that were specified at install time for communications between the clusterware daemons. 

If you wish to change the network that Oracle Clusterware is using for communications on the interconnect, this can be done by  modifying the hosts file of each node, such that the private node  name that was specified during the installation would now resolve  to a different IP address. This modification of the hosts file should be done while the Oracle Clusterware Stack is down on all  nodes. After the Oracle Clusterware stack has restarted, you can then run oifcfg as described in this note, to change the cluster_interconnect used by RDBMS and ASM instances.

 

Solution

As noted above, if the IP subnet for either the 'public' or 'cluster_interconnect' interface is incorrect and needs to be changed, you should make the changes using the Oracle Interface Configuration Tool (oifcfg).

Instructions for Changing Interfaces on Linux/Unix systems

You can change the interface information for a cluster using the following as an example:

1.  First - Use the getif option to show the currently configured interfaces.  This will display the interfaces that are currently stored in the OCR, what their subnets are, and what their role is  (i.e. public or cluster_interconnect).  This example is from a Linux system, but the syntax is the same for any platform.  On Unix/Linux systems, the interface names are generally assigned by the OS, and standard names vary by platform.  (For Windows systems, see additional notes below) :

% $ORA_CRS_HOME/bin/oifcfg getif
eth0 10.2.156.0 global public
eth1 192.168.0.0 global cluster_interconnect

2.   If the public interface IP needs to be changed, there is not a 'modify' option - you will need to delete the interface and add it back with the correct subnet IP.  So, in the example here, the subnet is being changed from 10.2.156.0  to 10.2.166.0 via two separate commands - first a 'delif'  followed by a 'setif':

% $ORA_CRS_HOME/bin/oifcfg delif -global eth0
% $ORA_CRS_HOME/bin/oifcfg setif –global eth0/10.2.166.0:public

syntax: oifcfg setif <interface-name>/<subnet>:<cluster_interconnect|public>

Note:  'oifcfg setif' should be run as the same user who did the original CRS installation, because the 'setif' option updates the OCR and sets permissions in the OCR for the interface based on who is running the command.  This is true on all platforms, including Windows.

3.  The same step could be use for the private interface:

% $ORA_CRS_HOME/bin/oifcfg delif –global eth1
% $ORA_CRS_HOME/bin/oifcfg setif –global eth1/192.168.1.0:cluster_interconnect

4. Verify the correct interface subnet is in use by re-running oifcfg with the 'getif' option:

 

% $ORA_CRS_HOME/bin/oifcfg getif
eth0 10.2.166.0 global public
eth1 192.168.1.0 global cluster_interconnect

Notes for Windows Systems

The syntax for changing the interfaces on Windows/RAC clusters is the same as on Unix or Linux, but the interface names will be slightly different.   On Windows systems, the default names assigned to the interfaces are generally names such as:

Local Area Connection
Local Area Connection 1
Local Area Connection 2

If you are using an interface name that has spaces in it, such as the examples above, you must enclose the name in quotes.   Also, be aware that it is case sensitive.  So for example, on Windows, you would use syntax such as the following to add the public subnet in:

C:\oracle\product\10.2.0\crs\bin\oifcfg setif –global 'Local Area Connection 1'/10.2.166.0:public

 However, it is common practice on Windows to rename the interfaces to use names that are more meaningful, such as renaming them to 'Public' and 'Private'.   If interface names are renamed after CRS is installed, then you will need to run 'oifcfg'  to delete the old interface and add the new one back in, as described above.

You can view the available interface names on each node by running the command:

oifcfg iflist

 This command must be run on each node, as the 'iflist' option is showing the interface names that exist at the operating system level on that particular node.   On the other hand, commands such as 'oifcfg getif', 'oifcfg delif'  and 'oifcfg setif'  are actually reading/updating the OCR, which is a shared resource.  So these commands need only be run one time, and should be runnable from any node in the cluster.

Ramifications of Changing Interface Names Using oifcfg

The ramifications of changing the interface names depends on which name you are changing, and whether or not you are also changing the IP address.  In cases where only the interface names are being changed, the ramifications are minor.  If you change the name for the Public Interface that is stored in the OCR, you will also need to modify the nodeapps for each node.   Therefore, the nodeapps will have to be brought down in order for this change to take effect.   See Note 276434.1 for more details on changing the nodeapps to use a new public interface name.

For the Private interface, the database will use the interface stored in the OCR and defined as a 'cluster_interconnect' for cache fusion traffic.  You will be able to see this at startup in the alert log, after the parameter listing - for example:

Cluster communication is configured to use the following interface(s) for this instance
192.168.1.1

If this is incorrect, then once you have corrected the entries stored in the OCR  (by using 'oifcfg' as described in this note) you will have to restart the instance in order for it to re-read the OCR and pick up the correct network. This applies to ASM instances and Database instances alike.

Note 1: If you intend to change the actual IP addresses for the interconnect, this should be done while the CRS stack is down, in order to prevent node evictions. However,  the CRS stack must be running in order to run the 'oifcfg' utility. Once the IP addresses are changed, and the CRS stack has been brought back up, you can then use 'oifcfg' to change the information stored in the OCR. At that point, the instances can then be started.

 

Note 2: On Windows systems, after shutting down the instance, you will also need to stop/restart the OracleService<SID> (or OracleASMService<ASMSID> before the OCR will be re-read.

References

For additional information on these commands for 10gR1, refer to:

Oracle® Real Application Clusters Administrator's Guide 10g Release 1 (10.1) Part No. B10765-01. Chapter 8, "Administrative Options", Administering System and Network Interfaces with the
OIFCFG (Oracle Interface Configuration) Tool

For 10gR2, refer to:

Oracle® Database Oracle Clusterware and Oracle Real Application Clusters Administration and Deployment Guide , 10g Release 2 (10.2) , Part Number B14197-03.  Chapter 9.

For 11gR1, refer to:

Oracle® Clusterware Administration and Deployment Guide
11g Release 1 (11.1) , Part Number B28255-03, Appendix C


Oifcfg Usage:
oifcfg iflist
oifcfg setif {-node <nodename> | -global} {<if_name>/<subnet>:<if_type>}...
oifcfg getif [-node <nodename> | -global] [ -if <if_name>[/<subnet>] [-type <if_type>] ]
oifcfg delif [-node <nodename> | -global] [<if_name>[/<subnet>]]
oifcfg [-help]

<nodename> - name of the host, as known to a communications network
<if_name> - name by which the interface is configured in the system
<subnet> - subnet address of the interface
<if_type> - type of the interface { cluster_interconnect | public | storage }