CDH安装

CDH安装步骤

0.机器配置

我在虚拟机和实际生产中配置CDH均已成功，下面简略谈一下整体的过程。本教程适合有经验的大数据从业人员安装CDH，不建议新手完全按照本文章学习。

• 物理网卡12G
• 三台虚拟机【Centos 7系统】，分别是192.168.211.3，192.168.211.4，192.168.211.5
• 物理硬盘100G【由三个虚拟机配置，server3=40G，server4=server5=30G】
• 三台虚拟机的分配内存分别是3G，2G，2G

1.环境预安装

step 1.关闭防火墙

#!/bin/bash
for HOST in `cat hosts`
do
ssh -t root@$HOST "systemctl status firewalld"
ssh -t root@$HOST "systemctl stop firewalld"
ssh -t root@$HOST "systemctl disable firewalld"
done

step 2.安装mysql

具体细节详见博客：https://blog.csdn.net/liu16659/article/details/81053412

step 3.配置yum源

[root@server3 transparent_hugepage]# yum install httpd  #因为需要浏览器访问得到，所以需要安装这个httpd服务
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.163.com
 * updates: mirrors.aliyun.com
Resolving Dependencies
--> Running transaction check
---> Package httpd.x86_64 0:2.4.6-80.el7.centos.1 will be installed
--> Processing Dependency: httpd-tools = 2.4.6-80.el7.centos.1 for package: httpd-2.4.6-80.el7.centos.1.x86_64
--> Processing Dependency: /etc/mime.types for package: httpd-2.4.6-80.el7.centos.1.x86_64
--> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-
····【此处省略输出信息】
Installed:
  httpd.x86_64 0:2.4.6-80.el7.centos.1                                                        

Dependency Installed:
  apr.x86_64 0:1.4.8-3.el7_4.1                         apr-util.x86_64 0:1.5.2-6.el7          
  httpd-tools.x86_64 0:2.4.6-80.el7.centos.1           mailcap.noarch 0:2.1.41-2.el7          

Complete!

• 安装createrepo，这个在后面的remote parcel repository时需要用到的

[root@server3 transparent_hugepage]# yum install yum-utils createrepo
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.163.com
 * updates: mirrors.aliyun.com
Resolving Dependencies
--> Running transaction check
---> Package createrepo.noarch 0:0.9.9-28.el7 will be installed
--> Processing Dependency: python-deltarpm for package: createrepo-0.9.9-28.el7.noarch
--> Processing Dependency: libxml2-python for package: createrepo
···【省略输出信息】

Dependency Installed:
  deltarpm.x86_64 0:3.6-3.el7                  libxml2-python.x86_64 0:2.9.1-6.el7_2.3       
  python-chardet.noarch 0:2.2.1-1.el7_1        python-deltarpm.x86_64 0:3.6-3.el7            
  python-kitchen.noarch 0:1.1.1-5.el7         

Complete!

• 执行命令createrepo .用以生成本地资源库

[root@server3 cm]# createrepo .
Spawning worker 0 with 7 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[root@server3 cm]# ls
cloudera-manager-agent-5.14.2-1.cm5142.p0.8.el7.x86_64.rpm
cloudera-manager-daemons-5.14.2-1.cm5142.p0.8.el7.x86_64.rpm
cloudera-manager-server-5.14.2-1.cm5142.p0.8.el7.x86_64.rpm
cloudera-manager-server-db-2-5.14.2-1.cm5142.p0.8.el7.x86_64.rpm
enterprise-debuginfo-5.14.2-1.cm5142.p0.8.el7.x86_64.rpm
jdk-6u31-linux-amd64.rpm
oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm
repodata

• 开启http服务，这样你安装httpd服务的主机就可以作为一个可访问的主机了

[root@server3 ~]# systemctl start httpd.service
[root@server3 ~]# systemctl status httpd.service
??? httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled)
   Active: active (running) since Sun 2018-07-15 05:34:16 EDT; 5s ago
     Docs: man:httpd(8)
           man:apachectl(8)
 Main PID: 1699 (httpd)
   Status: "Processing requests..."
   CGroup: /system.slice/httpd.service
           ?????€1699 /usr/sbin/httpd -DFOREGROUND
           ?????€1700 /usr/sbin/httpd -DFOREGROUND
           ?????€1701 /usr/sbin/httpd -DFOREGROUND
           ?????€1702 /usr/sbin/httpd -DFOREGROUND
           ?????€1703 /usr/sbin/httpd -DFOREGROUND
           ?????€1704 /usr/sbin/httpd -DFOREGROUND

Jul 15 05:34:16 server3 systemd[1]: Starting The Apache HTTP Server...
Jul 15 05:34:16 server3 httpd[1699]: AH00558: httpd: Could not reliably determine the s...age
Jul 15 05:34:16 server3 systemd[1]: Started The Apache HTTP Server.
Hint: Some lines were ellipsized, use -l to show in full.

step 4.配置ntp服务

step 5.关闭透明大页

step 6.设置交换区大小

step 7.

如下的操作需要在每台机器上都执行

[root@server3 ~]# echo never > /sys/kernel/mm/transparent_hugepage/enabled 
[root@server3 ~]# echo never > /sys/kernel/mm/transparent_hugepage/defrag

step 8.创建cm元数据库

create database hive default character set utf8;
create user 'hive'@'%' identified by '929751@Shen';
grant all privileges on hive.* to 'hive'@'%';

create database amon default character set utf8;
create user 'amon'@'%' identified by '929751@Shen';
grant all privileges on amon.* to 'amon'@'%';


create database rman default character set utf8;
create user 'rman'@'%' identified by '929751@Shen';
grant all privileges on rman.* to 'rman'@'%';


create database sentry default character set utf8;
create user 'sentry'@'%' identified by '929751@Shen';
grant all privileges on sentry.* to 'sentry'@'%';


create database nav default character set utf8;
create user 'nav'@'%' identified by '929751@Shen';
grant all privileges on nav.* to 'nav'@'%';


create database navms default character set utf8;
create user 'navms'@'%' identified by '929751@Shen';
grant all privileges on navms.* to 'navms'@'%';


create database cm default character set utf8;
create user 'cm'@'%' identified by '929751@Shen';
grant all privileges on cm.* to 'cm'@'%';


create database oozie default character set utf8;
create user 'oozie'@'%' identified by '929751@Shen';
grant all privileges on oozie.* to 'oozie'@'%';

flush privileges;

• 可能不是一下子安装成功，这里也给出删表/库的语句：

drop database hive;
drop database amon;
drop database rman;
drop database nav;
drop database navms;
drop database cm;
drop database oozie;
drop database sentry;


drop user hive;
drop user amon;
drop user rman;
drop user nav;
drop user navms;
drop user cm;
drop user oozie;
drop user sentry;

step 9.下载CDH

• 安装wget【用以从www上get资源】命令：

yum -y install wget

• 从cdh官网得到需要安装的rpm包，parcel包

wget -c -r -nd -np -k -L -A rpm http://archive-primary.cloudera.com/cm5/redhat/7/x86_64/cm/5.14.2/RPMS/x86_64/


下载parcel包
wget -c -r -nd -np -k -L -A rpm http://archive-primary.cloudera.com/cdh5/parcels/5.14.2/CDH-5.14.2-1.cdh5.14.2.p0.3-el7.parcel

step 10.安装CDH

```
[root@server3 ~]# /usr/share/cmf/schema/scm_prepare_database.sh -h server3 mysql cm cm 123789@Sflfg
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing:  /usr/java/jdk1.7.0_67-cloudera/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
Sun Jul 15 04:30:12 EDT 2018 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
[                          main] DbCommandExecutor              INFO  Successfully connected to database.
All done, your SCM database is configured correctly!

这一步的操作是将cdh连接到指定的mysql数据库。这样以后的一些安装信息就可以写到这些数据库中。我们可以在/usr/share/cloudera-scm-manager下找到db.properties文件，里面记录的便是此信息。

step 10.启动cloudera-scm-server服务，并验证是否启动成功

[root@server3 ~]# systemctl start  cloudera-scm-server 
[root@server3 ~]# systemctl status  cloudera-scm-server 
â cloudera-scm-server.service - LSB: Cloudera SCM Server
   Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled)
   Active: active (exited) since Sun 2018-07-15 04:33:04 EDT; 3min 45s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1452 ExecStart=/etc/rc.d/init.d/cloudera-scm-server start (code=exited, status=0/SUCCESS)

Jul 15 04:32:58 server3 systemd[1]: Starting LSB: Cloudera SCM Server...
Jul 15 04:32:59 server3 cloudera-scm-server[1452]: /etc/rc.d/init.d/cloudera-scm-server: line 109: pstree: comm...ound
Jul 15 04:32:59 server3 su[1476]: (to cloudera-scm) root on none
Jul 15 04:33:04 server3 cloudera-scm-server[1452]: Starting cloudera-scm-server: [  OK  ]
Jul 15 04:33:04 server3 systemd[1]: Started LSB: Cloudera SCM Server.
Hint: Some lines were ellipsized, use -l to show in full.

报错信息

1.错误

[root@server3 share]# /usr/share/cmf/schema/scm_prepare_database.sh -h server3 mysql cm cm 929751@Shen
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing:  /usr/java/jdk1.7.0_67-cloudera/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
[                          main] DbCommandExecutor              INFO  Unable to find JDBC driver for database type: MySQL
[                          main] DbCommandExecutor              ERROR JDBC Driver com.mysql.jdbc.Driver not found.
[                          main] DbCommandExecutor              ERROR Exiting with exit code 3
--> Error 3, giving up (use --force if you wish to ignore the error)

原因：这是因为没有将`mysql-java-connector.jar放到指定目录/usr/share/java/下。这个目录可能没有创建，需要手动创建一下。
需要注意的是，我们下载得到的mysql驱动包是有小版本信息的，需要将其改名才能放到上述的指定目录中。

2.错误

install agent时出错：报错信息如下：

Error: Package: MySQL-python-1.2.5-1.el7.x86_64 (base)
Requires: libmysqlclient.so.18()(64bit)
Error: Package: 2:postfix-2.10.1-6.el7.x86_64 (@anaconda)
Requires: libmysqlclient.so.18(libmysqlclient_18)(64bit)
Error: Package: 2:postfix-2.10.1-6.el7.x86_64 (@anaconda)
Requires: libmysqlclient.so.18()(64bit)
Error: Package: MySQL-python-1.2.5-1.el7.x86_64 (base)
Requires: libmysqlclient.so.18(libmysqlclient_18)(64bit)    

报错原因：在mysql5.7版本中，缺失这个libmysqlclient.so.18(64bit)这个文件
解决方法：安装mysql-community-libs-compat-5.7.12-1.el6.x86_64包即可

Suggested fix:
Add libmysqlclient.so.18 to the mysql-community-libs-compat package.

rpm -ivh mysql-community-libs-compat-5.7.12-1.el6.x86_64

3.错误

2018-07-15 07:51:31,255 WARN 2007250860@scm-web-18:com.cloudera.server.cmf.tsquery.TimeSeriesQueryService: Could not find a HOST_MONITORING nozzle from SCM.
com.cloudera.cmon.MgmtServiceLocatorException: Could not find a HOST_MONITORING nozzle from SCM.
    at com.cloudera.cmon.MgmtServiceLocator.getNozzleIPC(MgmtServiceLocator.java:141)
    at com.cloudera.server.cmf.tsquery.NozzleRequest.(NozzleRequest.java:50)
    at com.cloudera.server.cmf.tsquery.TimeSeriesMultiRequest.(TimeSeriesMultiRequest.java:48)
    at com.cloudera.server.cmf.tsquery.TimeSeriesQueryService.queryTimeSeries(TimeSeriesQueryService.java:516)
    at com.cloudera.server.web.cmf.charts.TimeSeriesQueryController.queryTimeSeriesHelper(TimeSeriesQueryController.java:328)

我在选择安装的时候使用的是统一的root用户，导致出现了找不到scm server的错误。这里我的建议是将其修改成使用ssh秘钥安装。

4.错误

2018-07-15 09:40:06,638 WARN 2015560880@scm-web-27:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING

因为我用的是虚拟机装的cdh环境。因为刚开始只是用了15G的物理内存装，于是就在激活parcel包的地方报错了。将虚拟机的物理内存调大一些即可。

5.错误

2018-07-15 08:57:12,074 INFO CommandPusher:com.cloudera.server.cmf.CommandPusherThread: Failed to update commands in batch mode.
java.lang.IllegalStateException: Transaction not active
        at org.hibernate.ejb.TransactionImpl.getRollbackOnly(TransactionImpl.java:131)
        at com.cloudera.enterprise.AbstractWrappedEntityManager.commit(AbstractWrappedEntityManager.java:109)
        at com.cloudera.cmf.persist.CmfEntityManager.commit(CmfEntityManager.java:375)
        at com.cloudera.server.cmf.CommandPusherThread.handleCommandBatched(CommandPusherThread.java:274)
        at com.cloudera.server.cmf.CommandPusherThread.innerLoop(CommandPusherThread.java:189)
        at com.cloudera.server.cmf.CommandPusherThread.run(CommandPusherThread.java:157)

这个错误的解决办法记不起来了。【待完善】

步骤截图

这里给出一些比较难的步骤的安装图片，如果这些地方控制的好了，就不会有什么错了。
- 建议不要使用用户密码的形式安装

• 可以使用http浏览器查看到parcel的本地目录。同理，后面的rpm包也需要放到资源库中
  

• 本地Parcel存储库路径不要修改，直接是默认值即可
  

• 选择parcel本地目录
  

总结

• 可以通过如下日志查看cloudera-scm-server服务进程的具体执行消息。日志路径：/var/log/messages【基本上默认的日志路径都是在/var/lib目录下】

参考链接

• http://archive-primary.cloudera.com/cdh5/parcels/5.14.2/CDH-5.14.2-1.cdh5.14.2.p0.3-el7.parcel
• https://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/5.14.2/RPMS/x86_64/
• https://bugs.mysql.com/bug.php?id=81384