Hadoop之Cloudera Manager安装问题总结

安装过程中，由于网络终端，导致下面问题：
问题1：安装停止在获取安装锁

/tmp/scm_prepare_node.tYlmPfrT 
using SSH_CLIENT to get the SCM hostname: 172.16.77.20 33950 22 
opening logging file descriptor 

正在启动安装脚本...正在获取安装锁...BEGIN flock 4 

    这段大概过了半个小时，一次卸载，一次等了快1个小时，终于过去了，
问题2：不能选择主机
    安装失败了，重新不能选主机

    解决方案，需要清理安装失败文件《 Hadoop之Cloudera Manager CDH4卸载》

问题3：DNS反向解析PTR localhost

   描述：DNS反向解析错误，不能正确解析Cloudera Manager Server主机名。如下日志：

Detecting Cloudera Manager Server...
Detecting Cloudera Manager Server...
BEGIN host -t PTR 192.168.1.198
198.1.168.192.in-addr.arpa domain name pointer localhost.
END (0)
using localhost as scm server hostname
BEGIN which python
/usr/bin/python
END (0)
BEGIN python -c 'import socket; import sys; s = socket.socket(socket.AF_INET); s.settimeout(5.0); s.connect((sys.argv[1], int(sys.argv[2]))); s.close();' localhost 7182
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "<string>", line 1, in connect
socket.error: [Errno 111] Connection refused
END (1)
could not contact scm server at localhost:7182, giving up
waiting for rollback request

   解决方案：将连不上的机器 /usr/bin/host 文件删掉,执行下面命令

sudo mv /usr/bin/host /usr/bin/host.bak

   说明：不明白cloudera的初衷，这里已经得到 Cloudera Manager Server的ip了，却还要把ip解析成主机名来连接。由于DNS反向解析没有配置好，根据Cloudera Manager Server 的ip解析主机名却得到了localhost，造成之后的连接错误。这里的解决方案是直接把/usr/bin/host删掉，这样Cloudera Manager就会直接使用 ip进行连接，就没有错了。

问题 4 NTP:

   问题描述： 

Bad Health --Clock Offset
The host's NTP service did not respond to a request for the clock offset.

   解决：配置NTP服务

   步骤参考：

• CentOS配置NTP Server:【http://www.hailiangchen.com/centos-ntp/】
• 国内常用NTP服务器地址及IP【http://www.douban.com/note/171309770/】

   修改配置文件：【vim /etc/ntp.conf】

# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server s1a.time.edu.cn prefer
server s1b.time.edu.cn
server s1c.time.edu.cn

restrict 172.16.1.0 mask 255.255.255.0 nomodify   <===放行局域网来源

   启动ntp服务【service ntpd restart 】   

   客户端同步时间(work02,work03）：【ntpdate work01】

   说明：NTP服务启动需要大约五分钟时间，服务启动之前，若客户端同步时间，则会出现错误“no server suitable for synchronization found”

   定时同步时间：在work02和 work03上配置crontab定时同步时间【crontab -e】

00 12 * * * root /usr/sbin/ntpdate 192.168.56.121 >> /root/ntpdate.log 2>&1

问题 5 

    描述：Clock Offset

• Ensure that the host's hostname is configured properly.
• Ensure that port 7182 is accessible on the Cloudera Manager Server (check firewall rules).
• Ensure that ports 9000 and 9001 are free on the host being added.
• Check agent logs in /var/log/cloudera-scm-agent/ on the host being added (some of the logs can be found in the installation details).

   问题定位：在对应host（work02、work03）上运行 'ntpdc -c loopinfo'

[root@work03 work]# ntpdc -c loopinfo
ntpdc: read: Connection refused

   解决：【开启ntp服务：三台机器都开机启动 ntp服务】

chkconfig ntpd on

问题 6 heartbeat:

    错误信息：【Installation failed. Failed to receive heartbeat from agent.】

    解决：关闭防火墙

问题 7 Unknow Health：

Unknow Health
重启后：Request to theHost Monitor failed.
service --status-all| grep clo
机器上查看scm-agent状态：cloudera-scm-agent dead but pid file exists

   解决：重启服务

service cloudera-scm-agent restart
service cloudera-scm-server restart

问题 8 canonial name hostname consistent：

Bad Health
The hostname and canonical name for this host are not consistent when checked from a Java process.
canonical name：
4092 Monitor-HostMonitor throttling_logger WARNING  (29 skipped) hostname work02 differs from the canonical name work02.xinzhitang.com

    解决：修改hosts 使FQDN和 hostname相同【ps：虽然解决了但是不明白为什么主机名和主机别名要一样】

/etc/hosts
192.168.1.185 work01 work01
192.168.1.141 work02 work02
192.168.1.198 work03 work03

问题 9 Concerning Health：

    Concerning Health Issue

    --  Network Interface Speed --

   描述：The host has 2 network interface(s) that appear to be operating at less than full speed. Warning threshold: any.

    详细：

This is a host health test that checks for network interfaces that appear to be operating at less than full speed.
A failure of this health test may indicate that network interface(s) may be configured incorrectly and may be causing performance problems. Use the ethtool command to check and configure the host's network interfaces to use the fastest available link speed and duplex mode.

   解决：本次测试修改了 Cloudera Manager 的配置，应该不算是真正的解决

转子：http://www.aboutyun.com/thread-9087-1-1.html