遇到一个问题,在CDH中一个结点内存不够用,于是关闭CDH服务后,并且把虚拟机关闭了。重启虚拟机后
在执行 /opt/cloudera-manager/cm-5.4.3/etc/init.d/cloudera-scm-server start 这个命令后,显示
Starting cloudera-scm-server: [FAILED]
查看日志:
2017-06-29 23:34:18,403 ERROR main:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Table 'scm.CM_VERSION' doesn't exist
2017-06-29 23:39:02,803 ERROR MainThread:com.cloudera.server.web.cmf.cloud.EC2MetadataFetcher: Request to EC2 metadata failed: I/O error: Connection refused; nested exception is java.net.ConnectException: Connection refused
2017-06-30 00:36:39,549 ERROR 1257786676@scm-web-27:com.cloudera.server.web.cmf.HealthCheckController: Exception occurred when checking the role health of mgmt-HOSTMONITOR-a87cd6b7373577914b89cd2a4f6c0215
2017-06-30 00:42:11,945 ERROR ParcelUpdateService:com.cloudera.parcel.components.ParcelDownloaderImpl: Unable to retrieve remote parcel repository manifest
2017-06-30 00:42:23,200 ERROR 848994288@scm-web-39:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 00:42:23,203 ERROR HeartbeatInfoCache:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 00:42:23,203 ERROR 1984362683@scm-web-24:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 00:42:23,204 ERROR CommandPusher:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 00:43:05,212 ERROR DatabaseSizeGauge-0:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 00:43:05,213 ERROR DatabaseSizeGauge-0:com.cloudera.enterprise.DatabaseSizeGauge: Failed to execute db size query.
2017-06-30 00:43:05,621 ERROR CMMetricsForwarder-0:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 00:43:05,625 ERROR 259754993@scm-web-37:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 00:43:05,939 ERROR 259754993@scm-web-37:com.cloudera.cmon.MgmtServiceLocator: Unable to retrieve HOST_MONITORING nozzle URL
2017-06-30 01:35:07,753 ERROR 981843565@agentServer-56:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 01:35:07,757 ERROR 2112951525@agentServer-57:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 01:35:07,761 ERROR 788984542@agentServer-55:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 01:35:07,762 ERROR 2087598531@agentServer-54:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 01:35:07,764 ERROR 393804251@agentServer-52:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 01:35:07,765 ERROR 1832883193@agentServer-53:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 01:35:13,585 ERROR 612713479@scm-web-90:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 01:35:13,600 ERROR 777110549@agentServer-59:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 01:35:13,605 ERROR 1470355044@agentServer-58:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 01:35:14,240 ERROR 395648131@scm-web-93:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 01:35:14,242 ERROR 1060225136@scm-web-79:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 01:35:14,245 ERROR 1351956370@scm-web-83:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: An attempt by a client to checkout a Connection has timed out.
2017-06-30 02:07:34,454 ERROR 1351956370@scm-web-83:com.cloudera.cmon.MgmtServiceLocator: Unable to retrieve HOST_MONITORING nozzle URL
2017-06-30 02:07:37,211 ERROR 1351956370@scm-web-83:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:38,013 ERROR 475660012@agentServer-47:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:38,467 ERROR 475660012@agentServer-47:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:40,268 ERROR 66826631@scm-web-118:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:41,149 ERROR CommandPusher:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:42,240 ERROR 1351956370@scm-web-83:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:43,290 ERROR 1351956370@scm-web-83:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:44,080 ERROR 1102170305@scm-web-120:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:44,099 ERROR 66826631@scm-web-118:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:44,129 ERROR 1351956370@scm-web-83:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:44,368 ERROR HeartbeatInfoCache:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:46,229 ERROR CommandPusher:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:46,267 ERROR 1351956370@scm-web-83:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:47,272 ERROR 1351956370@scm-web-83:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:50,346 ERROR DatabaseSizeGauge-0:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:50,357 ERROR DatabaseSizeGauge-0:com.cloudera.enterprise.DatabaseSizeGauge: Failed to execute db size query.
2017-06-30 02:07:50,372 ERROR 1351956370@scm-web-83:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:51,234 ERROR CommandPusher:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:52,213 ERROR 1351956370@scm-web-83:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:52,289 ERROR 66826631@scm-web-118:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:52,302 ERROR 729438994@scm-web-82:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:53,314 ERROR 1351956370@scm-web-83:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:53,968 ERROR 66826631@scm-web-118:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:54,097 ERROR 729438994@scm-web-82:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:54,118 ERROR 729438994@scm-web-82:com.cloudera.cmon.MgmtServiceLocator: (3 skipped) Unable to retrieve HOST_MONITORING nozzle URL
2017-06-30 02:07:54,328 ERROR 1351956370@scm-web-83:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:56,445 ERROR CommandPusher:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 02:07:56,449 ERROR 66826631@scm-web-118:org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Communications link failure
2017-06-30 03:12:47,902 ERROR WebServerImpl:com.cloudera.server.web.cmf.TsqueryAutoCompleter: Error getting predicates
问题的原因:
重启虚拟机后,发现jps 命令不能用 提示:-bash: jps: command not found,但是java -version ,java 命令都可以用,并且 cat /etc/profile 后也是配置java的环境的,jps 就突然不能用了。
并且 cd $JAVA_HOME/bin 下,运行 jps 也同样提示:
-bash: jps: command not found。但是 运行 ./jps 是可以的。
最后直接重新安装jdk 后,发现jps 就可以用
于是重新执行命令:
/opt/cloudera-manager/cm-5.4.3/etc/init.d/cloudera-scm-server start 后,就OK了。