部署CDH以及相关服务启动时的问题与解决方法
部署CDH以及相关服务启动时的问题与解决方法
（一）系统配置问题
（1）磁盘空间不够
The I/O cache encountered an error while updating data in medium "ahci-0-0" (rc=VERR_BAD_EXE_FORMAT). Make sure there is enough free space on the disk and that the disk is working properly. Operation can be resumed afterwards.
解决：
三台机器的配置
192.168.1.(164~166)：内存4G，磁盘40G
（二）cloudera-scm-server服务
（1）cloudera-scm-server启动后马上down掉
# /etc/init.d/cloudera-scm-server status
cloudera-scm-server 已死，但 pid 文件仍存
解决：
1. cloudera-scm-server依赖Mysqld服务，启动必须先启动数据库服务
#/etc/init.d/mysqld start
#chkconfig mysqld on (实现开机自启动)
2. 解析域名vim /etc/hosts
IP 名称
（2）cloudera-scm-server 启动后挂掉
查看日志
# vim /var/log/cloudera-scm-server/cloudera-scm-server.log
ERROR main:com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean: Tables have unsupported engine type [MyISAM]. InnoDB is required.
数据库要求INNODB
解决：
# vim /etc/my.cnf
[mysqld]
default-storage-engine=INNODB
重启服务
#/etc/init.d/mysld restart
重置数据库
#/usr/share/cmf/schema/scm_prepare_database.sh mysql -h localhost -uroot -p'123456' --scm-host localhost scm scm scm
#用Cloudera Manager重新建集群
（三）安装CDH
(1) yum源报错（安装服务前提前搭建好http服务）
https://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.10.0/repodata/repomd.xml: [Errno 14] PYCURL ERROR 6 - "Couldn't resolve host 'archive.cloudera.com'"
原因：cloudera-agent的URL不正确
解决：搭建http服务器，创建的yum源
1. 搭建http服务
#service httpd start
#mkdir /var/www/html/parcel-repo/
#ln -s /opt/cloudera/parcel-repo/ /var/www/html/parcel-repo/
2.创建parcel的yum源
#yum install createrepo -y
#cd /var/www/html/parcel-repo/
#createrepo .
3.浏览器测试
http://192.168.3.82/parcel-repo/
解决cloudera Manager安装锁的问题
删掉/tmp下面的
#rm -rf scm_prepare_node.*
# rm -rf .scm_prepare_node.lock
无法连接HOST_Monitor的问题（安装完CDH）
（1）确定启动cloudera Managemnt服务
（2）agent无法连接HOST_Monitor的问题
1. 查看日志
# cat /var/log/cloudera-scm-agent/cloudera-scm-agent.log
INFO CM server guid: d31b5fae-56f4-466e-9f89-d639daa5038b
2. 删除cm_guid
#rm -rf /var/lib/cloudera-scm-agent/cm_guid
3. 重启服务
#/etc/init.d/cloudera-scm-agent restart
guid是agent首次启动时会从server处获得，之后一直保存在agent端。Host-Monitor服务是运行在server端。guid是agent连接server端Host-Monitor服务的一个标识
（3）测某台主机连接不上Host-Monitor
Error sending messages to firehose: mgmt-SERVICEMONITOR-ddc9b774d2da234ec1da3086c2a9355f
解决：
查看是否集群中所有IP与域名对应：vim /etc/hosts
重启agent： /etc/init.d/cloudera-scm-agent restart
(4)查询Host-Monitor内部出错
Could not fetch descriptor after 5 tries, exiting
解决：重启cloudera Manager服务
(5)集群安装报错
ERROR Failure due to stall on seeded torrent.
解决：重启agent服务
（五）oozie服务启动报错
E0103: Could not load service classes, Cannot load JDBC driver class 'com.mysql.jdbc.Driver'
解决：
软连接JDBC的驱动器
#ln -s /soft/mysql-connector-java-5.1.40-java.jar /opt/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/lib/oozie/lib/
（六）HDFS启动时DataNode异常
Initialization failed for Block pool
解决：
#vim /dfs/dn/current/VERSION
clusterID=cluster6 //保证在同一集群中
（七）Hive启动后Hive Meatstore server进程异常退出
1）数据库驱动的问题
错误1：Error creating transactional connection factory
cause by：Attempt to invoke the "BONECP" plugin to create a ConnectionPool gave an error : The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
解决：
#ln -s /soft/mysql-connector-java-5.1.40-bin.jar /opt/cloudera/parcels/CDH/lib/hive/auxlib
2）数据库字符问题
错误2：Specified key was too long; max key length is 767 bytes
原因：Mysql的utf8字符集最大长度限制，注意字符集设置为latin1
解决：
在CM的web UI界面设置Hive的数据库：
[Hive]-->[配置]-->[Hive Metastore数据库]-->[Hive Metastore数据库名称]
创建Metastore数据库
Mysql>mysql -uroot -p123456
mysql>create database hive character set latin1；
3）数据库权限问题
错误3：
Database access problem. Killing off this connection and all remaining connections in the connection pool. SQL State = HY000
解决：
grant all on hive.* to 'scm'@'192.168.1.164' identified by 'scm';
grant all on hive.* to 'scm'@'192.168.1.165' identified by 'scm';
grant all on hive.* to 'scm'@'192.168.1.166' identified by 'scm';
4）错误4：
MetaException(message:Version information not found in metastore. )
解决：
在CM的web UI界面配置Hive服务：
在hive配置文件中添加：
#cd /opt/cloudera/parcels/CDH/lib/hive/conf
# vim hive-site.xml