Oracle 11gR2 RAC 进程说明

一.先看Oracle官方文档

参考：

http://download.oracle.com/docs/cd/E11882_01/rac.112/e16794/intro.htm#CWADD91998

Oracle Clusterware Software Concepts and Requirements

Oracle Clusterware uses voting disk files to provide fencing and cluster node membership determination. OCR provides cluster configuration information. You can place the Oracle Clusterware files on either Oracle ASM or on shared common disk storage. If you configure Oracle Clusterware on storage that does not provide file redundancy, then Oracle recommends that you configure multiple locations for OCR and voting disks. The voting disks and OCR are described as follows:

·Voting Disks

Oracle Clusterware uses voting disk files to determine which nodes are members of a cluster. You can configure voting disks on Oracle ASM, or you can configure voting disks on shared storage.

If you configure voting disks on Oracle ASM, then you do not need to manually configure the voting disks. Depending on the redundancy of your disk group, an appropriate number of voting disks are created.

If you do not configure voting disks on Oracle ASM, then for high availability, Oracle recommends that you have a minimum of three voting disks on physically separate storage. This avoids having a single point of failure. If you configure a single voting disk, then you must use external mirroring to provide redundancy.

You should have at least three voting disks, unless you have a storage device, such as a disk array that provides external redundancy. Oracle recommends that you do not use more than five voting disks. The maximum number of voting disks that is supported is 15.

·Oracle Cluster Registry

Oracle Clusterware uses the Oracle Cluster Registry (OCR) to store and manage information about the components that Oracle Clusterware controls, such as Oracle RAC databases,listeners, virtual IP addresses (VIPs), and services and any applications. OCR stores configuration information in a series of key-value pairs in a tree structure. To ensure cluster high availability, Oracle recommends that you define multiple OCR locations. In addition:

oYou can have up to five OCR locations

oEach OCR location must reside on shared storage that is accessible by all of the nodes in the cluster

oYou can replace a failed OCR location online if it is not the only OCR location

oYou must update OCR through supported utilities such as Oracle Enterprise Manager, the Server Control Utility (SRVCTL), the OCR configuration utility (OCRCONFIG), or the Database Configuration Assistant (DBCA)

See Also:

Chapter 2, "Administering Oracle Clusterware"for more information about voting disks and OCR

Oracle Clusterware Network Configuration Concepts

Oracle Clusterware enables a dynamic Grid Infrastructure through the self-management of the network requirements for the cluster. Oracle Clusterware 11grelease 2 (11.2) supports the use of dynamic host configuration protocol (DHCP) for all private interconnect addresses, as well as for most of the VIP addresses. DHCP provides dynamic configuration of the host's IP address, but it does not provide an optimal method of producing names that are useful to external clients.

When you are using Oracle RAC, all of the clients must be able to reach the database. This means that the VIP addresses must be resolved by the clients. This problem is solved by the addition of the OracleGrid Naming Service (GNS) to the cluster. GNS is linked to the corporate domain name service (DNS) so that clients can easily connect to the cluster and the databases running there. Activating GNS in a cluster requires a DHCP service on the public network.

Implementing GNS

To implement GNS, you must collaborate with your network administrator to obtain an IP address on the public network for the GNS VIP. DNS uses the GNS VIP to forward requests for access to the cluster to GNS. The network administrator must delegate a subdomain in the network to the cluster. The subdomain forwards all requests for addresses in the subdomain to the GNS VIP.

GNS and the GNS VIP run on one node in the cluster. The GNS daemon listens on the GNS VIP using port 53 for DNS requests.Oracle Clusterware manages the GNS and the GNS VIP to ensure that they are always available. If the server on which GNS is running fails, then Oracle Clusterware fails GNS over, along with the GNS VIP, to another node in the cluster.

With DHCP on the network, Oracle Clusterware obtains an IP address from the server along with other network information, such as what gateway to use, what DNS servers to use, what domain to use, and what NTP server to use. Oracle Clusterware initially obtains the necessary IP addresses during cluster configuration and it updates the Oracle Clusterware resources with the correct information obtained from the DHCP server.

Single Client Access Name (SCAN)

Oracle RAC 11grelease 2 (11.2) introduces theSingle Client Access Name (SCAN). The SCAN is a single name that resolves to three IP addresses in the public network. When using GNS and DHCP, Oracle Clusterware configures the VIP addresses for the SCAN name that is provided during cluster configuration.

The node VIP and the three SCAN VIPs are obtained from the DHCP server when using GNS. If a new server joins the cluster, then Oracle Clusterware dynamically obtains the required VIP address from the DHCP server, updates the cluster resource, and makes the server accessible through GNS.

Example 1-1shows the DNS entries that delegate a domain to the cluster.

Example 1-1 DNS Entries

# Delegate to gns on mycluster

mycluster.example.com NS myclustergns.example.com

#Let the world know to go to the GNS vip

myclustergns.example.com. 10.9.8.7

See Also:

Oracle Grid Infrastructure Installation Guidefor details about establishing resolution through DNS

Configuring Addresses Manually

Alternatively, you can choose manual address configuration, in which you configure the following:

·One public host name for each node.

·One VIP address for each node.

You must assign a VIP address to each node in the cluster. Each VIP address must be on the same subnet as the public IP address for the node and should be an address that is assigned a name in the DNS. Each VIP address must also be unused and unpingable from within the network before you install Oracle Clusterware.

·Up to three SCAN addresses for the entire cluster.

Note:

The SCAN must resolve to at least one address on the public network. For high availability and scalability, Oracle recommends that you configure the SCAN to resolve to three addresses.

See Also:

Your platform-specificOracle Grid Infrastructure Installation Guideinstallation documentation for information about system requirements and configuring network addresses

Overview of Oracle Clusterware Platform-Specific Software Components

When Oracle Clusterware is operational, several platform-specific processes or services run on each node in the cluster. This section describes these various processes and services.

The Oracle Clusterware Stack

Oracle Clusterware consists of two separate stacks: an upper stack anchored by the Cluster Ready Services (CRS) daemon (crsd) and a lower stack anchored by the Oracle High Availability Services daemon (ohasd). These two stacks have several processesthat facilitate cluster operations. The following sections describe these stacks in more detail:

·The Cluster Ready Services Stack

·The Oracle High Availability Services Stack

The Cluster Ready Services Stack

The list in this section describes the processes that comprise CRS. The list includes components that are processes on Linux and UNIX operating systems, or services on Windows.

·Cluster Ready Services(CRS):The primary program for managing high availability operations in a cluster.

The CRS daemon(crsd) manages cluster resources based on the configuration information that is stored in OCR for each resource. This includes start, stop, monitor, and failover operations. Thecrsdprocess generates events when the status of a resource changes. When you have Oracle RAC installed, thecrsdprocess monitors the Oracle database instance, listener, and so on, and automatically restarts these components when a failure occurs.

·Cluster Synchronization Services (CSS):Manages the cluster configuration by controlling which nodes are members of the cluster and by notifying members when a node joins or leaves the cluster. If you are using certified third-party clusterware, then CSS processes interface with your clusterware to manage node membership information.

Thecssdagentprocess monitors the cluster and provides I/O fencing. This service formerly was provided by Oracle Process Monitor Daemon (oprocd), also known asOraFenceServiceon Windows. Acssdagentfailure may result in Oracle Clusterware restarting the node.

·Oracle ASM: Provides disk management for Oracle Clusterware and Oracle Database.

·Cluster Time Synchronization Service (CTSS): Provides time management in a cluster for Oracle Clusterware.

·Event Management (EVM): A background process that publishes events that Oracle Clusterware creates.

·Oracle Notification Service (ONS): A publish and subscribe service for communicating Fast Application Notification(FAN) events.

·Oracle Agent (oraagent): Extends clusterware to support Oracle-specific requirements and complex resources. This process runs server callout scripts when FAN events occur. This process was known as RACG in Oracle Clusterware 11grelease 1 (11.1).

·Oracle Root Agent (orarootagent): A specializedoraagentprocess that helpscrsdmanage resources owned byroot, such as the network, and the Grid virtual IP address.

TheCluster Synchronization Service (CSS),Event Management (EVM), and Oracle Notification Services (ONS) components communicate with other cluster component layers on other nodes in the same cluster database environment. These components are also the main communication links between Oracle Database, applications, and the Oracle Clusterwarehigh availability components. In addition, these background processes monitor and manage database operations.

The Oracle High Availability Services Stack

This section describes the processes that comprise the Oracle High Availability Services stack. The list includes components that are processes on Linux and UNIX operating systems, or services on Windows.

·Cluster Logger Service (ologgerd): Receives information from all the nodes in the cluster and persists in a CHM Repository-based database. This service runs on only two nodes in a cluster.

·System Monitor Service (osysmond): The monitoring and operating system metric collection service that sends the data to the cluster logger service. This service runs on every node in a cluster.

·Grid Plug and Play (GPNPD):Provides access to the Grid Plug and Play profile, and coordinates updates to the profile among the nodes of the cluster to ensure that all of the nodes have the most recent profile.

·Grid Interprocess Communication (GIPC):A support daemon that enables Redundant Interconnect Usage.

·Multicast Domain Name Service (mDNS): Used by Grid Plug and Play to locate profiles in the cluster, as well as by GNS to perform name resolution. The mDNS process is a background process on Linux and UNIX, and a service on Windows.

·Oracle Grid Naming Service (GNS): Handles requests sent by external DNS servers, performing name resolution for names defined by the cluster.

二.查看OHASD资源

Oracle High Availability Services Daemon (OHASD)：This process anchors the lower part of the Oracle Clusterware stack, which consists of processes that facilitate cluster operations.

在11gR2里面启动CRS的时候，会提示ohasd已经启动。那么这个OHASD到底包含哪些资源。我们可以通过如下命令来查看：

[grid@racnode1 ~]$ crsctl stat res -init -t

---------------------------------------------------------

NAMETARGETSTATESERVERSTATE_DETAILS

---------------------------------------------------------

Cluster Resources

---------------------------------------------------------

ora.asm

1ONLINEONLINEracnode1Started

ora.crsd

1ONLINEONLINEracnode1

ora.cssd

1ONLINEONLINEracnode1

ora.cssdmonitor

1ONLINEONLINEracnode1

ora.ctssd

1ONLINEONLINEracnode1OBSERVER

ora.diskmon

1ONLINEONLINEracnode1

ora.drivers.acfs

1ONLINEUNKNOWNracnode1

ora.evmd

1ONLINEONLINEracnode1

ora.gipcd

1ONLINEONLINEracnode1

ora.gpnpd

1ONLINEONLINEracnode1

ora.mdnsd

1ONLINEONLINEracnode1

在10g平台下，RAC的一些资源，在我的Blog：

RAC的一些概念性和原理性的知识

http://blog.csdn.net/xujinyang/article/details/6837273

里已经做了相关的说明。

分别看下这些进程：

（1）ora.asm：这个是asm实例的进程。在10g里，OCR和Voting disk是放在其他共享设备上的。11gR2里面，默认是放在ASM里面。在Clusterware启动的时候需要读取这些信息，所以在集群启动的时候需要先启动ASM实例。

（2）ora.crsd，ora.cssd和ora.evmd：

这三个进程是Clusterware中最重要的3个进程.

在10g中，在安装clusterware的最后阶段，会要求在每个节点执行root.sh脚本，这个脚本会在/etc/inittab文件的最后把这3个进程加入启动项，这样以后每次系统启动时，Clusterware也会自动启动，其中EVMD和CRSD两个进程如果出现异常，则系统会自动重启这两个进程，如果是CSSD进程异常，系统会立即重启。

在11gR2中，只会将ohasd写入/etc/inittab文件。

[grid@racnode1 init.d]$ cat /etc/inittab

h1:35:respawn:/etc/init.d/init.ohasd run >/dev/null 2>&1

所以在10g中常用的/etc/init.d/init.crs之类的命令都没有了。就剩下一个/etc/init.d/init.ohasd进程。

OCSSD：这个进程是Clusterware最关键的进程，如果这个进程出现异常，会导致系统重启，这个进程提供CSS(Cluster Synchronization Service)服务。CSS服务通过多种心跳机制实时监控集群状态，提供脑裂保护等基础集群服务功能。

CRSD：是实现"高可用性(HA)"的主要进程，它提供的服务叫作CRS(Cluster Ready Service)服务。所有需要高可用性的组件，都会在安装配置的时候，以CRS Resource的形式登记到OCR中，而CRSD进程就是根据OCR中的内容，决定监控哪些进程，如何监控，出现问题时又如何解决。也就是说，CRSD进程负责监控CRS Resource的运行状态，并要启动，停止，监控，Failover这些资源。默认情况下，CRS会自动尝试重启资源5次，如果还是失败，则放弃尝试。

CRS Resource包括GSD(Global Serveice Daemon),ONS(Oracle Notification Service),VIP, Database, Instance和Service.

EVMD：负责发布CRS产生的各种事件(Event).这些Event可以通过2种方式发布给客户：ONS和Callout Script.

这三个进程各自的作用，具体参考

RAC的一些概念性和原理性的知识

http://blog.csdn.net/xujinyang/article/details/6837273

中的说明。

（3）Grid Plug and Play (GPNPD):

Provides access to the Grid Plug and Play profile, and coordinates updates to the profile among the nodes of the cluster to ensure that all of the nodes have the most recent profile.

（4）Grid Interprocess Communication (GIPC):

A support daemon that enables Redundant Interconnect Usage.

（5）ora.mdns

Used by Grid Plug and Play to locate profiles in the cluster, as well as by GNS to perform name resolution. The mDNS process is a background process on Linux and UNIX, and a service on Windows.

（6）Cluster Time Synchronization Service (CTSS):

Provides time management in a cluster for Oracle Clusterware.在上面的查询结果中，我们看到CTSS的状态是OBSERVER。即旁观者。

在11gR2中，RAC在安装的时候，时间同步可以用两种方式来实现，一是NTP，还有就是CTSS.当安装程序发现NTP协议处于非活动状态时，安装集群时间同步服务将以活动模式自动进行安装并通过所有节点的时间。如果发现配置了NTP，则以观察者模式启动集群时间同步服务，Oracle Clusterware不会在集群中进行活动的时间同步。

（7）Automatic Storage Management Cluster File System (Oracle ACFS)：

Oracle Automatic Storage Management Cluster File System (Oracle ACFS) is a multi-platform, scalable file system, and storage management technology that extends Oracle Automatic Storage Management (Oracle ASM) functionality to support customer files maintained outside of Oracle Database.Oracle ACFS supports many database and application files, including executables, database trace files, database alert logs, application reports, BFILEs, and configuration files. Other supported files are video, audio, text, images, engineering drawings, and other general-purpose application file data.

An Oracle ACFS file system is a layer on Oracle ASM and is configured with Oracle ASM storage, as shown inFigure 5-1. Oracle ACFS leverages Oracle ASM functionality that enables:

·Oracle ACFS dynamic file system resizing

·Maximized performance through direct access to Oracle ASM disk group storage

·Balanced distribution of Oracle ACFS across Oracle ASM disk group storage for increased I/O parallelism

·Data reliability through Oracle ASM mirroring protection mechanisms

更多内容参考：

http://download.oracle.com/docs/cd/E11882_01/server.112/e16102/asmfilesystem.htm#OSTMG31000

三.查看CRS资源

在11.2中，对CRSD资源进行了重新分类：Local Resources和Cluster Resources。OHASD指的就是Cluster Resource.

[grid@racnode1 ~]$ crsctl stat res -t

---------------------------------------------------------

NAMETARGETSTATESERVERSTATE_DETAILS

---------------------------------------------------------

Local Resources

---------------------------------------------------------

ora.CRS.dg

ONLINEONLINEracnode1

ONLINEONLINEracnode2

ora.DATA.dg

ONLINEONLINEracnode1

ONLINEONLINEracnode2

ora.FRA.dg

ONLINEONLINEracnode1

ONLINEONLINEracnode2

ora.LISTENER.lsnr

ONLINEONLINEracnode1

ONLINEONLINEracnode2

ora.asm

ONLINEONLINEracnode1Started

ONLINEONLINEracnode2Started

ora.eons

ONLINEONLINEracnode1

ONLINEONLINEracnode2

ora.gsd

OFFLINE OFFLINEracnode1

OFFLINE OFFLINEracnode2

ora.net1.network

ONLINEONLINEracnode1

ONLINEONLINEracnode2

ora.ons

ONLINEONLINEracnode1

ONLINEONLINEracnode2

ora.registry.acfs

ONLINEUNKNOWNracnode1

ONLINEONLINEracnode2

---------------------------------------------------------

Cluster Resources

---------------------------------------------------------

ora.LISTENER_SCAN1.lsnr

1ONLINEONLINEracnode2

ora.oc4j

1OFFLINE OFFLINE

ora.racdb.db

1ONLINEONLINEracnode1Open

2ONLINEONLINEracnode2Open

ora.racnode1.vip

1ONLINEONLINEracnode1

ora.racnode2.vip

1ONLINEONLINEracnode2

ora.scan1.vip

1ONLINEONLINEracnode2

[grid@racnode1 ~]$

从上面的查询结果可以看出，在11gR2中把network,disgroup,eons,和asm也作为了一种资源。

还有一点需要注意：就是gsd和oc4j这两资源，他们是offlie的。说明如下：

ora.gsdis OFFLINE by default if there is no 9i database in the cluster.

ora.oc4j is OFFLINE in 11.2.0.1 as Database Workload Management(DBWLM) is unavailable.these can be ignored in 11gR2 RAC.

也可用如下命令来查看进程：

[root@racnode1 ~]# crs_stat -t

NameTypeTargetStateHost

------------------------------------------------------------

ora.CRS.dgora....up.typeONLINEONLINEracnode1

ora.DATA.dgora....up.typeONLINEONLINEracnode1

ora.FRA.dgora....up.typeONLINEONLINEracnode1

ora....ER.lsnrora....er.typeONLINEONLINEracnode1

ora....N1.lsnrora....er.typeONLINEONLINEracnode2

ora.asmora.asm.typeONLINEONLINEracnode1

ora.eonsora.eons.typeONLINEONLINEracnode1

ora.gsdora.gsd.typeOFFLINEOFFLINE

ora....networkora....rk.typeONLINEONLINEracnode1

ora.oc4jora.oc4j.typeOFFLINEOFFLINE

ora.onsora.ons.typeONLINEONLINEracnode1

ora.racdb.dbora....se.typeONLINEONLINEracnode1

ora....SM1.asmapplicationONLINEONLINEracnode1

ora....E1.lsnrapplicationONLINEONLINEracnode1

ora....de1.gsdapplicationOFFLINEOFFLINE

ora....de1.onsapplicationONLINEONLINEracnode1

ora....de1.vipora....t1.typeONLINEONLINEracnode1

ora....SM2.asmapplicationONLINEONLINEracnode2

ora....E2.lsnrapplicationONLINEONLINEracnode2

ora....de2.gsdapplicationOFFLINEOFFLINE

ora....de2.onsapplicationONLINEONLINEracnode2

ora....de2.vipora....t1.typeONLINEONLINEracnode2

ora....ry.acfsora....fs.typeONLINEONLINEracnode2

ora.scan1.vipora....ip.typeONLINEONLINEracnode1

ora.scan2.vipora....ip.typeONLINEONLINEracnode2

[root@racnode1 ~]#

四.查看各种资源之间的依赖关系

比如DG resource依赖于ASM，VIP依赖于network。这些可以从资源的详细属性看出：

[root@racnode1 ~]# crsctl stat res ora.DATA.dg -p

NAME=ora.DATA.dg

TYPE=ora.diskgroup.type

ACL=owner:grid:rwx,pgrp:oinstall:rwx,other::r--

ACTION_FAILURE_TEMPLATE=

ACTION_SCRIPT=

AGENT_FILENAME=%CRS_HOME%/bin/oraagent%CRS_EXE_SUFFIX%

ALIAS_NAME=

AUTO_START=never

CHECK_INTERVAL=300

CHECK_TIMEOUT=600

DEFAULT_TEMPLATE=

DEGREE=1

DESCRIPTION=CRS resource type definition for ASM disk group resource

ENABLED=1

LOAD=1

LOGGING_LEVEL=1

NLS_LANG=

NOT_RESTARTING_TEMPLATE=

OFFLINE_CHECK_INTERVAL=0

PROFILE_CHANGE_TEMPLATE=

RESTART_ATTEMPTS=5

SCRIPT_TIMEOUT=60

START_DEPENDENCIES=hard(ora.asm) pullup(ora.asm)

START_TIMEOUT=900

STATE_CHANGE_TEMPLATE=

STOP_DEPENDENCIES=hard(intermediate:ora.asm)

STOP_TIMEOUT=180

UPTIME_THRESHOLD=1d

USR_ORA_ENV=

USR_ORA_OPI=false

USR_ORA_STOP_MODE=

VERSION=11.2.0.1.0

[grid@racnode1 ~]$ crsctl stat res ora.racnode1.vip -p

NAME=ora.racnode1.vip

TYPE=ora.cluster_vip_net1.type

ACL=owner:root:rwx,pgrp:root:r-x,other::r--,group:oinstall:r-x,user:grid:r-x

ACTION_FAILURE_TEMPLATE=

ACTION_SCRIPT=

ACTIVE_PLACEMENT=1

AGENT_FILENAME=%CRS_HOME%/bin/orarootagent%CRS_EXE_SUFFIX%

AUTO_START=restore

CARDINALITY=1

CHECK_INTERVAL=1

DEFAULT_TEMPLATE=PROPERTY(RESOURCE_CLASS=vip)

DEGREE=1

DESCRIPTION=Oracle VIP resource

ENABLED=1

FAILOVER_DELAY=0

FAILURE_INTERVAL=0

FAILURE_THRESHOLD=0

HOSTING_MEMBERS=racnode1

LOAD=1

LOGGING_LEVEL=1

NLS_LANG=

NOT_RESTARTING_TEMPLATE=

OFFLINE_CHECK_INTERVAL=0

PLACEMENT=favored

PROFILE_CHANGE_TEMPLATE=

RESTART_ATTEMPTS=0

SCRIPT_TIMEOUT=60

SERVER_POOLS=*

START_DEPENDENCIES=hard(ora.net1.network) pullup(ora.net1.network)

START_TIMEOUT=0

STATE_CHANGE_TEMPLATE=

STOP_DEPENDENCIES=hard(ora.net1.network)

STOP_TIMEOUT=0

UPTIME_THRESHOLD=1h

USR_ORA_ENV=

USR_ORA_VIP=racnode1-vip

VERSION=11.2.0.1.0

------------------------------------------------------------------------------