SCAN,VIP,HAIP,etc.

©OraInternals  Riyaj Shamsudeen
SCAN,VIP,HAIP,etc.
By
Riyaj Shamsudeen
Blog:  orainternals.wordpress.com 
©OraInternals  Riyaj Shamsudeen  2
Who am I?
  19 years using Oracle products/DBA
  OakTable  member
  Oracle ACE Director
  Certified DBA versions 7.0,7.3,8,8i,9i
&10g
  Specializes in RAC, performance tuning,
Internals and E-business suite
  Chief DBA with OraInternals
  Co-author of few books.
  Email: [email protected]
  Blog : orainternals.wordpress.com 
  Web: www.orainternals.com
©OraInternals  Riyaj Shamsudeen  3
Outline
  VIP
  SCAN
  HAIP
©OraInternals  Riyaj Shamsudeen  4
VIP
  Virtual IP address
  An IP address not statically linked to just one node or interface.
  Provides a mechanism for faster failovers.
  Each node must have one Virtual IP address. 
  All primary VIPs of  a cluster must be in the same subnet.
However, second (or more) network is possible.
©OraInternals  Riyaj Shamsudeen  5
ifconfig
$ / sbin/ifconfig  -a
…
e1000g0:1:
flags=1040843
D,IPv4>  mtu  1500 index 2 inet 172.16.140.151 netmask
ffff0000 broadcast 172.16.255.255
...
Notice :1 in the interface name.
©OraInternals  Riyaj Shamsudeen  6
Listener
$ lsnrctl status listener
LSNRCTL for Solaris: Version 11.2.0.2.0 - Production
on 18-FEB-2012 15:31:27
…
Listening Endpoints Summary… 
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY =LISTENER))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)
(HOST=172.16.140.151)(PORT=1521)))
©OraInternals  Riyaj Shamsudeen  7
Connect string
just_vips=
  (description=
    (load_balance=off)
    (address=(protocol=tcp)(host=172.16.140.151)(port=1521))
    (address=(protocol=tcp)(host=172.16.140.152)(port=1521))
  ( connect_data=
        ( service_name=racdb)
    )
  )
©OraInternals  Riyaj Shamsudeen  8
Connect timeout
truss -d -E -vall sqlplus rs@just_vips
...
3.0730  0.0002 connect(9, 0x00539F10, 16, SOV_DEFAULT)  Err#150
EINPROGRESS
              AF_INET  name = 172.16.140.151  port = 1521
3.0731  0.0000 brk(0x00540D90)                                                                = 0
3.0732  0.0000 brk(0x00544D90)                                                                = 0
pollsys(0xFFFFFD7FFFDEFC60, 1, 0xFFFFFD7FFFDEFB40, 0x00000000)
(sleeping...)
                fd=9  ev=POLLOUT rev=0xFFFFFFFF
                timeout: 60.000000000 sec
63.0735  0.0000 pollsys(0xFFFFFD7FFFDEFC60,1,0xFFFFFD7FFFDEFB40,
0x00000000)= 0
                fd=9  ev=POLLOUT rev=0
              timeout: 60.000000000 sec
63.0740  0.0001 close(9)                                         
©OraInternals  Riyaj Shamsudeen  9
VIP failed over
0.0799    so_socket(PF_INET, SOCK_STREAM, IPPROTO_IP, "", SOV_DEFAULT) = 9
0.0800  ioctl(9, FIONBIO, 0xFFFFFD7FFFDEF9D8)                    = 0
                              write 4 bytes
0.0801  connect(9, 0x0053BDF0, 16, SOV_DEFAULT) Err#146  ECONNREFUSED
              AF_INET  name = 172.16.140.151    port = 1521
0.0802  close(9)                                                                              = 0
0.0802  getsockopt(9, SOL_SOCKET, SO_SNDBUF, 0xFFFFFD7FFFDF0214, 
©OraInternals  Riyaj Shamsudeen  10
VIP as a resource
$ crsctl  status resource ora.solrac1.vip
NAME=ora.solrac1.vip
TYPE=ora.cluster_vip_net1.type
TARGET=ONLINE
STATE=ONLINE on solrac1
$ crsctl  status resource ora.solrac1.vip -p |grep
USR_ORA_VIP
GEN_USR_ORA_VIP=
USR_ORA_VIP=solaris1_vip
$ grep solaris1_vip /etc/hosts
172.16.140.151    solaris1_vip.solrac.net solaris1_vip
©OraInternals  Riyaj Shamsudeen  11
Network as a resource
$ crsctl  status resource ora.net1.network -p |more                   
NAME=ora.net1.network
TYPE=ora.network.type 
…
USR_ORA_IF=e1000g0
USR_ORA_NETMASK=255.255.0.0
USR_ORA_SUBNET=172.16.0.0
©OraInternals  Riyaj Shamsudeen  12
SCAN
©OraInternals  Riyaj Shamsudeen  13
Connection: Step 1
solrac_po=
  (description=
    (address=(protocol=tcp)(host=solscan)(port =1521))
  ( connect_data=
        ( service_name=po)
    )
  )
$ ping - s solscan     
PING solscan: 56 data bytes
64 bytes from solscan.solrac.net  (172.16.140.150):
icmp_seq =0. time=0.0870 ms
©OraInternals  Riyaj Shamsudeen  14
Connection: Step 2
$ lsnrctl services listener_scan1
Service "po" has 1  instance(s).
  Instance "solrac1", status READY, has 1 handler(s) for
this service...
      Handler(s):
          "DEDICATED" established:4 refused:0 state:ready 
                REMOTE SERVER
                (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)
(HOST= 172.16.140.151)(PORT=1521)))
©OraInternals  Riyaj Shamsudeen  15
Connection: Step 3
$ lsnrctl services listener
Service "po" has 1  instance(s).
  Instance "solrac1", status READY, has 1 handler(s) for
this service...
      Handler(s):
          "DEDICATED" established:4 refused:0 state:ready 
                LOCAL SERVER
©OraInternals  Riyaj Shamsudeen  16
SCAN: DNS setup                             
nslookup        solscan 
  Name :  solscan 
        Address: 172.16.140.150
  Name : Scan- ip 
        Address: 172.16.140.149
  Name : Scan- ip 
        Address: 172.16.140.148
  DNS setup should return the IP addresses in a cyclic fashion
providing load balance at SCAN IP level.
  First connection will go to first SCAN listener, second
connection to the second SCAN listener etc.
©OraInternals  Riyaj Shamsudeen  17
Parameters
NAME                                TYPE              VALUE
-------------------- ----------- ------------------------------
remote_listener          string          solscan.solrac.net:1521
NAME                                TYPE              VALUE
-------------------- ----------- ------------------------------
local_listener            string          (DESCRIPTION=(ADDRESS_LIST=
                                                                  ADDRESS=(PROTOCOL=TCP)
                                                                  (HOST=172.16.140.151)(PORT=1521))))
©OraInternals  Riyaj Shamsudeen  18
SCAN as a resource
$ crsctl  stat resource ora.scan1.vip     
NAME=ora.scan1.vip
TYPE=ora.scan_vip.type
TARGET=ONLINE
STATE=ONLINE on solrac1
$ crsctl  stat resource ora.scan1.vip –p |grep '^USR_ORA_VIP'
USR_ORA_VIP=172.16.140.150
©OraInternals  Riyaj Shamsudeen  19
SCAN listener as a resource
$crsctl  stat res ora.LISTENER_SCAN1.lsnr -p|
egrep'ENDPOINTS|START_DEP'
ENDPOINTS=TCP:1521
START_DEPENDENCIES=hard(ora.scan1.vip )
dispersion:active(type:ora.scan_listener.type)
pullup(ora.scan1.vip)
©OraInternals  Riyaj Shamsudeen  20
Salient points.
  Three SCAN IP addresses and three SCAN listeners, at most.
  SCAN listeners and IP addresses will be mounted in three
nodes, if  you have more than three nodes.
  SCAN is an abstraction layer. Topology changes do not affect
client connect string.
  SCAN and VIP addresses should be in the same subnet. 
©OraInternals  Riyaj Shamsudeen  21
HAIP
  High Availability IP address for private interconnect traffic.
  OS based solution: Bonding, Trunking  etc
  HAIP Oracle based solution.
  Both clusterware and database uses HAIP plumbed IP address
for private interconnect traffic.
©OraInternals  Riyaj Shamsudeen  22
Oifcfg getif 
Clusterware :
$ olsnodes  –p
DB:
$ oifcfg  getif             
e1000g0  172.16.0.0  global  public
e1000g1  1.3.1.0  global    cluster_interconnect
©OraInternals  Riyaj Shamsudeen  23
Ifconfig output
$ ifconfig  -a
e1000g1:
flags=1000843
mtu  1500 index 3
                inet 1.3.1.170 netmask ffffff00 broadcast
1.3.1.255
e1000g1:1:
flags=1000843
mtu  1500 index 3
                inet 1.3.1.70 netmask ffffff00 broadcast
1.3.1.255
©OraInternals  Riyaj Shamsudeen  24
Ifconfig output
$ifconfig  –a
...
e1000g1:2:
flags=1000843
mtu  1500 index 3
                inet 169.254.201.54 netmask ffff8000
broadcast 169.254.255.255
e1000g1:3:
flags=1000843
mtu  1500 index 3
                inet 169.254.106.96 netmask ffff8000
broadcast 169.254.127.255
…
©OraInternals  Riyaj Shamsudeen  25
Gv$cluster_interconnects
  1* select * from gv$cluster_interconnects
SQL> /
    INST_ID NAME                      IP_ADDRESS            IS_ SOURCE
---------- --------------- ---------------- ---
                1 e1000g1:3            169.254.106.96    NO
                1 e1000g1:2            169.254.201.54    NO
©OraInternals  Riyaj Shamsudeen  26
Gv$cluster_interconnects
$ crsctl  stat res ora.cluster_interconnect.haip -init  |
more
NAME=ora.cluster_interconnect.haip
TYPE=ora.haip.type 
TARGET=ONLINE
STATE=ONLINE on solrac1
©OraInternals  Riyaj Shamsudeen  27
Root agent monitors
2012-04-19 15:10:08.298: [ USRTHRD][59] {0:5:6}  Thread:[NetHAWork]start  {
2012-04-19 15:10:08.342: [ USRTHRD][59] {0:5:6}  Thread:[NetHAWork]start  }
2012-04-19 15:10:08.342: [ USRTHRD][59] {0:5:6} HAIP:  Moving ip
'169.254.201.54' from inf  'e1000g1' to inf  'e1000g1'
2012-04-19 15:10:08.368: [ USRTHRD][59] {0:5:6} pausing thread
2012-04-19 15:10:08.474: [ USRTHRD][59] {0:5:6} posting thread
2012-04-19 15:10:08.490: [ USRTHRD][59] {0:5:6}  Thread:[NetHAWork]start  {
2012-04-19 15:10:08.528: [ USRTHRD][70] {0:5:6} [NetHAWork] thread started
2012-04-19 15:10:08.536: [ USRTHRD][70] {0:5:6}    Arp::sCreateSocket  {
2012-04-19 15:10:08.547: [ USRTHRD][71] {0:5:6} [NetHAWork] thread started
2012-04-19 15:10:08.592: [ USRTHRD][71] {0:5:6}    Arp::sCreateSocket  {
2012-04-19 15:10:08.628: [ USRTHRD][59] {0:5:6}  Thread:[NetHAWork]start  }
2012-04-19 15:10:08.654: [ USRTHRD][70] {0:5:6}    Arp::sCreateSocket  }
2012-04-19 15:10:08.685: [ USRTHRD][70] {0:5:6}  Found existing ip
169.254.106.96 for interface e1000g1, transitioning to DEFEND s
tate
2012-04-19 15:10:08.687: [ USRTHRD][71] {0:5:6}    Arp::sCreateSocket  }
2012-04-19 15:10:08.688: [ USRTHRD][70] {0:5:6} Reattached IP:
  169.254.106.96 on interface e1000g1
2012-04-19 15:10:08.690: [ USRTHRD][71] {0:5:6} Found existing  ip
169.254.201.54 for interface e1000g1, transitioning to DEFEND s
tate
2012-04-19 15:10:08.721: [ USRTHRD][71] {0:5:6} Reattached IP:
  169.254.201.54 on interface e1000g1
©OraInternals  Riyaj Shamsudeen  28
Thank you for attending!
If  you like this presentation, you will love my 
2-part intensive, online RAC webinar.
http://www.orainternals.com/training/
advanced-rac-training
Week  Dates  Time
Week1  TBA  8AM – 2PM PDT
Week 2  TBA  8AM - 2PM PDT