12c 中Agent问题的描述及解决办法！

问题描述：在另一个节点安装完agent后OMS莫名其妙的down掉。

view sqlnet.log
Directory does not exist for read/write [/app/oracle/Middleware/agent/core/12.1.0.1.0/log] []

check the agents logs and find some java exception from the logs.
bash-3.2$ hostname
mcsp-prod-oem-1
bash-3.2$ pwd
/app/oracle/Middleware/oms/sysman/prov/agentpush/logs
bash-3.2$ /app/oracle/Middleware/oms/bin/emctl status oms
Oracle Enterprise Manager Cloud Control 12c Release 12.1.0.1.0
Copyright (c) 1996, 2011 Oracle Corporation. All rights reserved.
WebTier is Up
Oracle Management Server is Down
bash-3.2$ /app/oracle/Middleware/oms/bin/emctl start oms
Oracle Enterprise Manager Cloud Control 12c Release 12.1.0.1.0
Copyright (c) 1996, 2011 Oracle Corporation. All rights reserved.
Starting WebTier...
WebTier Successfully Started
Starting Oracle Management Server...
Oracle Management Server Successfully Started
Oracle Management Server is intializing.
Context not fully initialized yet.
成功启动了。
过一会再看：

bash-3.2$ /app/oracle/Middleware/oms/bin/emctl status oms
Oracle Enterprise Manager Cloud Control 12c Release 12.1.0.1.0
Copyright (c) 1996, 2011 Oracle Corporation. All rights reserved.
WebTier is Up
Oracle Management Server is Down
自己停了下来。
bash-3.2$ netstat -an |grep 4889
tcp 0 0 0.0.0.0:4889 0.0.0.0:* LISTEN
unix 2 [ ] DGRAM 4889
bash-3.2$ /app/oracle/Middleware/oms/bin/emctl status oms
Oracle Enterprise Manager Cloud Control 12c Release 12.1.0.1.0
Copyright (c) 1996, 2011 Oracle Corporation. All rights reserved.
WebTier is Up
Oracle Management Server is Down
bash-3.2$ /app/oracle/Middleware/oms/bin/emctl stop oms
Oracle Enterprise Manager Cloud Control 12c Release 12.1.0.1.0
Copyright (c) 1996, 2011 Oracle Corporation. All rights reserved.
Stopping WebTier...
WebTier Successfully Stopped
Stopping Oracle Management Server...
Oracle Management Server Already Stopped
Oracle Management Server is Down
bash-3.2$ /app/oracle/Middleware/oms/bin/emctl status oms
Oracle Enterprise Manager Cloud Control 12c Release 12.1.0.1.0
Copyright (c) 1996, 2011 Oracle Corporation. All rights reserved.
WebTier is Down
Oracle Management Server is Down
bash-3.2$ netstat -an |grep 4889
tcp 0 0 141.146.155.23:62700 141.146.155.23:4889 TIME_WAIT
unix 2 [ ] DGRAM 4889
bash-3.2$ netstat -an |grep 4900
bash-3.2$ netstat -an |grep 7799
bash-3.2$ netstat -an |grep 7403
tcp 0 0 141.146.155.23:7403 0.0.0.0:* LISTEN
tcp 0 0 141.146.155.23:50744 141.146.155.23:7403 TIME_WAIT
bash-3.2$ netstat -an |grep 7202
bash-3.2$ netstat -an |grep 7788
bash-3.2$ netstat -an |grep 3872
tcp 0 0 0.0.0.0:3872 0.0.0.0:* LISTEN
bash-3.2$ netstat -an |grep 7101
tcp 0 0 141.146.155.23:7101 0.0.0.0:* LISTEN
tcp 0 0 141.146.155.23:7101 141.146.155.23:35386 ESTABLISHED
tcp 0 0 141.146.155.23:35386 141.146.155.23:7101 ESTABLISHED
bash-3.2$ netstat -an |grep 7301
查看下面 logs，发现有Failed to get repository details类的错误提示。
/app/oracle/Middleware/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/sysman/log/emoms.log
/app/oracle/Middleware/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/sysman/log/emoms.trc
/app/oracle/Middleware/gc_inst/user_projects/domains/GCDomain/servers/EMGC_ADMINSERVER/logs/GCDomain.log
/app/oracle/Middleware/gc_inst/user_projects/domains/GCDomain/servers/EMGC_ADMINSERVER/logs/EMGC_ADMINSERVER.out
解决办法：

这里面的repository 用的是另一个库，而且这个库是RAC。

所以怀疑RAC库上有一些问题，查看上面的alert log 确实发现在有类似下面的错误信息。

Active Session History (ASH) performed an emergency flush.。。。。。。。。 This may mean that ASH is undersized.
和 Dumping diagnostic data in directory=[cdmp_20111108063439] 。。。。。。。。。。。。。。。。。
和Fatal NI connect error 12170.。。。。。。。。。。。。。。。。。。。。。。。。。。。。。

我通过修改/app/oracle/Middleware/gc_inst/em/EMGC_OMS1/sysman/ocm/emoms.properties.

来让respository 不使用scan listener 连接数据而是通过一个实例连接。

结果就没有问题了。

然后又试一下。停止2上的代理，停止1上的OMS，再打开OMS。再启动2上的代理。

结果到目前为止一切正常。没有再出再问题！