VIPs Often Go Offline Unexpectedly and Relocate to Another Node(文档 ID 1297867.1)
In this Document
| Symptoms |
| Cause |
| Solution |
APPLIES TO:
Oracle Database - Enterprise Edition - Version 10.2.0.5 and laterInformation in this document applies to any platform.
SYMPTOMS
VIPs often go offline unexpectedly, with the following message in crsd.log:
2011-02-17 15:11:16.441: [ CRSRES][11321]32In stateChanged, ora.node02.vip target is ONLINE
2011-02-17 15:11:16.441: [ CRSRES][11321]32 ora.node02.vip on node02 went OFFLINE unexpectedly
VIP tracing is set by using the following commands:
#crsctl debug log res "ora.node02.vip:5"
Following error messages (highlighted in bold letters) can be seen in the generated VIP trace "CRS_HOME/log/node02:
Fri Feb 18 15:32:39 GMT+08:00 2011 [ 8257768 ] IsIfAlive: RX packets checked if=en1 failed
2011-02-18 15:32:39.481: [ RACG][1] [4587556][1][ora.node02.vip]: Fri Feb 18 15:32:39 GMT+08:00 2011 [ 8257768 ] Interface en1 checked failed (host=node02)
Fri Feb 18 15:32:39 GMT+08:00 2011 [ 8257768 ] IsIfAlive: end for if=en1
Fri Feb 18 15:32:39 GMT+08:00 2011 [ 8257768 ] checkIf: end for if=en1
You can reset the VIP tracing to the default level by using the following commands:
#crsctl debug log res "ora.node01.vip:0"
#crsctl debug log res "ora.node02.vip:0"
CAUSE
The issue can be due to network performance when pinging the gateway using the public IP.
See "man ping" on AIX:
Uses the IP address as the source address in outgoing ping packets.
-c Count
Specifies the number of echo requests, as indicated by the Count
variable, to be sent (and received).
-w timeout
This option works only with the -c option. It causes ping to wait
for a maximum of 'timeout' seconds for a reply (after sending the
last packet).
So the following command will check, if 1 packet sent from 192.168.220.36 to 192.168.220.33 will receive a reply within 1s.
==>192.168.220.36 is the public IP, 192.168.220.33 is the gateway.
If the problem is with the network, the above "ping" command would take longer than 1s, and this leads to VIPs going offline unexpectedly and relocating to another node.
SOLUTION
To resolve the issue, please contact your network administrator to tune your network and ensure that the reply of the ping command is within 1s.
If you can't improve the network performance, please use the following temporary workaround (which is not recommended):
% srvctl stop nodeapps -n
2. Backup then Modify the racgvip script .
Change:
# timeout of ping in number of loops (1 sec)
PING_TIMEOUT=" -c 1 -w 1"
To:
# timeout of ping in number of loops (3 sec)
PING_TIMEOUT=" -c 1 -w 3"
3. Start the node applications and other necessary resources.
% srvctl start nodeapps -n