OEM 12c不能正常启动，同时repository数据库大量library cache lock等待事件

问题现象：
1，登录OEM 12c页面，显示'Backend WLS or EM application seems to be down' 。

2，登录服务器查看oms状态，报错Unable to retrive repository credentials from credential store. Verify that repository credentials are set in credential store.

[oem@dtydb0 config]$ emctl status oms

Oracle Enterprise Manager Cloud Control 12c Release 12.1.0.1.0  
Copyright (c) 1996, 2011 Oracle Corporation.  All rights reserved.
WebTier is Up
Oracle Management Server is not functioning because of the following reason:
Unable to retrive repository credentials from credential store. Verify that repository credentials are set in credential store.

3，此时登录数据库，发现大量library cache lock等待事件。同时username列为空

  

SID    SERIAL# OSUSER   USERNAME SVRPROC                  PROCESS                  EVENT                                        P1         P2         P3
------ ---------- -------- -------- ------------------------ ------------------------ ------------------------- --------------------- ---------- ----------
   162         15 oem               21504                    1234                     library cache lock                   6960226624 6892354168    5177347
   182         27 oem               21522                    1234                     library cache lock                   6960226624 6892961464    5177347
   221         27 oem               21557                    1234                     library cache lock                   6960226624 6941525184    5177347
   283         45 oem               22297                    1234                     library cache lock                   6960226624 6920965688    5177347
   303         31 oem               21453                    1234                     library cache lock                   6960226624 6892523840    5177347
   322         17 oem               21698                    1234                     library cache lock                   6960226624 6892486128    5177347
   363         11 oem               21593                    1234                     library cache lock                   6960226624 6892415392    5177347
   381         25 oem               21745                    1234                     library cache lock                   6960226624 6909425416    5177347
   403          9 oem               21651                    1234                     library cache lock                   6960226624 6959746000    5177347
    62         29 oem               22434                    1234                     library cache lock                   6960226624 6892094904    5177347
   123         83 oem               21700                    1234                     library cache lock                   6960226624 6909294952    5177347
   
   .....

4，查看数据库日志，从本机登录的用户大量超时的错误

Fatal NI connect error 12170.

  VERSION INFORMATION:
        TNS for Linux: Version 11.2.0.2.0 - Production
        Oracle Bequeath NT Protocol Adapter for Linux: Version 11.2.0.2.0 - Production
        TCP/IP NT Protocol Adapter for Linux: Version 11.2.0.2.0 - Production
  Time: 07-SEP-2012 16:58:44
  Tracing not turned on.
  Tns error struct:
    ns main err code: 12535

TNS-12535: TNS:operation timed out
    ns secondary err code: 12606
    nt main err code: 0
    nt secondary err code: 0
    nt OS err code: 0
  Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=10.4.124.239)(PORT=17859))
WARNING: inbound connection timed out (ORA-3136)

原因分析：
通过查阅网上参考资料的相关文章，初步定位为由于sysman用户密码不对，导致OMS 12c不能正常使用。同时，由于sysman用户不停登录，加上oracle 11gr2版本的登录限制(可以参考附录的相关文章），导致登录用户产生大量library cache lock等待事件。当用户长时间登录不上数据库时，产生alert.log的TNS-12535报错

解决办法
1，停止oms
emctl stop oms
2，修改sysman的密码
emctl config oms -change_repos_pwd -use_sys_pwd -sys_pwd "ora#2013" -new_pwd "oem12c.1"
[oem@dtydb0 config]$ emctl config oms -change_repos_pwd -use_sys_pwd -sys_pwd "ora#2013" -new_pwd "oem12c.1"
Oracle Enterprise Manager Cloud Control 12c Release 12.1.0.1.0  
Copyright (c) 1996, 2011 Oracle Corporation.  All rights reserved.

Changing passwords in backend ...
Passwords changed in backend successfully.
Updating repository password in Credential Store...
Successfully updated Repository password in Credential Store.
Restart all the OMSs using 'emctl stop oms -all' and 'emctl start oms'.
Successfully changed repository password.
3. Stop the Admin server on the primary OMS machine and re-start all the OMS:

cd <OMS_HOME>/bin
emctl stop oms -all
emctl start oms

4,再次登录，oem 12c已经恢复正常

参考资料：
    12C Cloud Control: Accessing the Console URL Shows Error Message 'Backend WLS or EM application seems to be down' [ID 1367997.1]
    12c Cloud Control: How to Modify the Password for SYSMAN and other Enterprise Manager Users at the OMS Level and Repository Database? [ID 1365930.1]
    http://www.cnblogs.com/fangwenyu/archive/2012/03/10/2389235.html
    http://yangtingkun.net/?p=279 数据库登录出现library cache lock等待（一）