centos7 安装 cdh5 集群超详细教程

系统环境：
主节点： master 10.18.1.117
从节点： slave1, slave2, slave3 10.18.1.118, 10.18.119, 10.18.1.120

一、准备工作，系统环境搭建

预先准备工作：

CDH 系统优化之关闭透明大页面及设置swappiness

http://blog.csdn.net/post_yuan/article/details/54967508

1. 关闭透明大页面

首先查看透明大页是否启用，[always] never表示已启用，always [never]表示已禁用

[root@master ~]# cat /sys/kernel/mm/transparent_hugepage/defrag 
[always] madvise never

[root@master ~]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@master ~]# echo never > /sys/kernel/mm/transparent_hugepage/enabled

直接运行上面两个命令暂时生效，重启后恢复。

2. 修改swappiness

Linux内核参数vm.swappiness，值的范围为0~100，表示系统什么时候开始进行物理内存与虚拟内存的交换。举个例子，系统总内存为64G，vm.swappiness为60，表示在系统内存使用64*0.4=25.6G的时候开始物理内存与虚拟内存的交换，这个动作势必会影响系统的性能。因此，Cloudera建议把这个值修改为1~10。

首先检查当前设置的vm.swappiness值，

[root@master ~]# cat /proc/sys/vm/swappiness 
30

临时修改swappiness（重启后恢复）

[root@master ~]# sysctl -w vm.swappiness=10
vm.swappiness = 10

永久生效（重启后不恢复）

[root@master ~]# echo "vm.swappiness=10" >> /etc/sysctl.conf

3. 修改hostname，并配置/etc/hosts 文件

修改hostname:

[root@master ~]# hostnamectl set-hostname master

从节点slave同样操作修改hostname

修改hosts文件：

[root@master ~]# vim /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

10.18.1.117 master
10.18.1.118 slave1
10.18.1.119 slave2
10.18.1.120 slave3

slave节点同时修改

4. 关闭防火墙和selinux

关闭防火墙

[root@master ~]# systemctl stop firewalld.service 
[root@master ~]# systemctl disable firewalld.service 
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.

从节点slave同样操作

关闭selinux

#临时关闭
[root@slave1 ~]# setenforce 0


#永久关闭，重启后生效
[root@slave1 ~]# vim /etc/selinux/config

SELINUX=disabled

其他从节点slave同样操作

5. 配置主节点ssh免密码登录从节点

生成ssh秘钥：

[root@master ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
f5:60:9c:d8:59:da:63:c6:9e:0d:9b:15:2e:57:b0:50 root@master
The key's randomart image is:
+--[ RSA 2048]----+
|            o.E..|
|         + B o + |
|        . X O =  |
|         o * @   |
|        S   * .  |
|                 |
|                 |
|                 |
|                 |
+-----------------+

将master公钥分发到从节点slave上：

[root@master ~]# scp ~/.ssh/id_rsa.pub root@slave1:/root/
The authenticity of host 'slave1 (10.18.1.118)' can't be established.
ECDSA key fingerprint is 8e:31:12:ba:90:92:37:ef:df:6e:9a:b5:ae:76:4d:fc.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'slave1,10.18.1.118' (ECDSA) to the list of known hosts.
root@slave1's password: 
id_rsa.pub 

配置从节点slave：

[root@master ~]# ssh root@slave1
root@slave1's password: 
Last login: Tue Jun  2 11:39:03 2020 from 10.18.24.127
#配置免密登录
root@slave1 ~]# ssh-keygen -t rsa
[root@slave1 ~]# cat id_rsa.pub >> ~/.ssh/authorized_keys

其他节点同样操作

免密测试：

[root@master ~]# ssh root@slave1
Last login: Wed Jun  3 16:13:22 2020 from 10.18.1.117

6. 安装jdk

首先删除CentOS自带的Open JDK

[root@slave1 ~]# yum remove java

下载并安装jdk

#rpm安装
[root@master home]# rpm -Uvh jdk-8u131-linux-x64.rpm

#配置java环境
vim /etc/profile
#增加：
export JAVA_HOME=/usr/java/jdk1.8.0_131
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

#然后执行：
[root@master home]# source /etc/profile

其他从节点相同操作

7. 安装MySQL(使用mysql5.6)

安装mysql5.7在远程建表时一直提示密码强度问题，修改数据库表中的密码强度不起作用，后改为mysql5.6.
在主节点master 上安装 mysql

#获取repo
[root@master home]# wget https://dev.mysql.com/get/mysql57-community-release-el7-9.noarch.rpm

#安装repo
[root@master home]# rpm -ivh mysql57-community-release-el7-9.noarch.rpm

#安装mysql-server
[root@master home]# yum install mysql-server

#启动mysql
[root@master home]# systemctl start mysqld

#开机启动
[root@master home]# systemctl enable mysqld

#查看状态
[root@master home]# systemctl status mysqld
● mysqld.service - MySQL Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2020-06-04 16:00:23 CST; 18s ago
     Docs: man:mysqld(8)
           http://dev.mysql.com/doc/refman/en/using-systemd.html
  Process: 27140 ExecStart=/usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid $MYSQLD_OPTS (code=exited, status=0/SUCCESS)
  Process: 27077 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS)
 Main PID: 27143 (mysqld)
   CGroup: /system.slice/mysqld.service
           └─27143 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid

Jun 04 16:00:06 slave1 systemd[1]: Starting MySQL Server...
Jun 04 16:00:23 slave1 systemd[1]: Started MySQL Server.


#配置mysql密码

#获取初始密码
[root@master home]# grep 'temporary password' /var/log/mysqld.log
2020-06-04T08:00:20.783779Z 1 [Note] A temporary password is generated for root@localhost: 9cVXGPq-W-.m

#登录
[root@master home]# mysql -uroot -p9cVXGPq-W-.m
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.7.30

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> 

#修改密码强度策略，否则简单密码会报错
mysql> set global validate_password_policy=0;
Query OK, 0 rows affected (0.00 sec)

#修改密码
mysql> SET PASSWORD = PASSWORD('passw0rd');
Query OK, 0 rows affected, 1 warning (0.00 sec)

#刷新权限
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.01 sec)

#退出
mysql> exit
Bye

#测试新密码
[root@master home]# mysql -uroot -ppassw0rd
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 9
Server version: 5.7.30 MySQL Community Server (GPL)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> 

#配置远程登录权限
mysql> grant all on *.* to 'root'@'%' identified by 'passw0rd' with grant option;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> grant all on *.* to 'root'@'master' identified by 'passw0rd' with grant option;
Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

mysql> exit
Bye

8. 配置ntp服务

在所有节点上安装并启动ntp服务（若没有可以通过yum安装）

[root@master home]# systemctl enable ntpd
Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.
[root@master home]# systemctl start ntpd
[root@master home]# systemctl status ntpd
● ntpd.service - Network Time Service
   Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2020-06-04 16:42:05 CST; 2min 53s left
  Process: 30359 ExecStart=/usr/sbin/ntpd -u ntp:ntp $OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 30360 (ntpd)
   CGroup: /system.slice/ntpd.service
           └─30360 /usr/sbin/ntpd -u ntp:ntp -g

Jun 04 16:42:04 master ntpd[30360]: Listen normally on 5 lo ::1 UDP 123
Jun 04 16:42:04 master ntpd[30360]: Listen normally on 6 eth0 fe80::8b24:3796:f7a2:a38a UDP 123
Jun 04 16:42:04 master ntpd[30360]: Listening on routing socket on fd #23 for interface updates
Jun 04 16:42:05 master systemd[1]: Started Network Time Service.
Jun 04 16:42:05 master ntpd[30360]: 0.0.0.0 c016 06 restart
Jun 04 16:42:05 master ntpd[30360]: 0.0.0.0 c012 02 freq_set kernel 0.000 PPM
Jun 04 16:42:05 master ntpd[30360]: 0.0.0.0 c011 01 freq_not_set
Jun 04 16:42:12 master ntpd[30360]: 0.0.0.0 c61c 0c clock_step -185.315932 s
Jun 04 16:39:06 master ntpd[30360]: 0.0.0.0 c614 04 freq_mode
Jun 04 16:39:07 master ntpd[30360]: 0.0.0.0 c618 08 no_sys_peer

#查看同步状态
[root@master home]# ntpstat 
synchronised to NTP server (111.230.189.174) at stratum 3
   time correct to within 148 ms
   polling server every 64 s

9. 搭建 CDH yum 源(待完善）

下载 cdh parcel 和 rpm 包

下载链接：

cdh parcel：https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_create_local_parcel_repo.html#cmig_topic_21_3_1

CDH - https://archive.cloudera.com/cdh5/parcels/

rpm tar包：http://archive.cloudera.com/cm5/repo-as-tarball/

-rw-r–r--. 1 root root 1934231261 Dec 19 09:15 CDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcel

-rw-r–r--. 1 root root 41 Dec 19 09:10 CDH-5.13.1-1.cdh5.13.1.p0.2-el7.parcel.sha1

drwxrwxr-x. 3 jenkins jenkins 121 Dec 7 06:32 cm

-rw-r–r--. 1 root root 951176044 Dec 19 09:13 cm5.13.1-centos7.tar.gz

-rw-r–r--. 1 root root 106 Dec 19 09:13 cm5.13.1-centos7.tar.gz.md5

-rw-r–r--. 1 root root 114 Dec 19 09:13 cm5.13.1-centos7.tar.gz.sha1

-rw-r–r--. 1 root root 73766 Dec 20 15:27 manifest.json

注意一定在下载parcel的时候，一定要下载目录下的manifest.json 文件 ， 否则cm-manager认不到parcel ！！！

在master和slave节点配置repo

[root@master home]# cd /etc/yum.repos.d/

[root@master yum.repos.d]# vim cloudera-cm.repo


[cloudera-cm]

# Packages for Cloudera's Distribution for cm, Version 5, on RedHat or CentOS 7 x86_64

name=Cloudera's Distribution for cm, Version 5

baseurl=http://10.18.218.14/bigdata/cm/5/

gpgkey = http://10.18.218.14/bigdata/cm/RPM-GPG-KEY-cloudera

gpgcheck = 1

二、开始部署CDH集群

1. 安装 cm-manager (主节点 master)

主节点master安装cloudera-manager

[root@master yum.repos.d]# yum install cloudera-manager-daemons.x86_64 cloudera-manager-server

安装MySQL JDBC Driver

[root@master yum.repos.d]# yum install mysql-connector-java.noarch

配置cm 数据库

[root@master yum.repos.d]# ls /usr/share/cmf/schema/scm_prepare_database.sh
/usr/share/cmf/schema/scm_prepare_database.sh

[root@master ~]# /usr/share/cmf/schema/scm_prepare_database.sh mysql -h master -uroot -ppassw0rd --scm-host master scm scm scm
JAVA_HOME=/usr/java/jdk1.8.0_131
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing:  /usr/java/jdk1.8.0_131/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
[                          main] DbCommandExecutor              INFO  Successfully connected to database.
All done, your SCM database is configured correctly!

若此处cm数据库配置报错，注意前面的数据库权限配置，我在这地方踩了好几个坑，mysql5.7的远程权限开不了，改成mysql5.6。在卸载时除了用yum remove外，还得删除mysql的安装文件，否则5.6无法启动。

启动cm-server

[root@master ~]# systemctl start cloudera-scm-server

https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_install_path_b.html#cmig_topic_6_6_1

这个过程需要花费一小会儿

Wait several minutes for the Cloudera Manager Server to start. To observe the startup process, run tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log on the Cloudera Manager Server host. If the Cloudera Manager Server does not start, see Troubleshooting Installation and Upgrade Problems.

In a web browser, enter http://Server host:7180, where Server host is the FQDN or IP address of the

启动成功后，会看见以下日志输出：

2020-06-11 16:21:40,852 INFO WebServerImpl:org.mortbay.log: jetty-6.1.26.cloudera.4
2020-06-11 16:21:40,854 INFO WebServerImpl:org.mortbay.log: Started [email protected]:7180
2020-06-11 16:21:40,854 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
2020-06-11 16:21:40,998 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Finished constructing repo:2020-06-11T08:21:40.998Z
2020-06-11 16:21:43,549 INFO ScmActive-0:com.cloudera.server.cmf.components.ScmActive: ScmActive completed successfully.

出现问题时注意查看日志

tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

2. 开始安装服务

在浏览器中输入mater地址：

10.18.1.117:7180
进入登录页面

输入用户名admin，密码admin，添加主机后进入安装界面

搜索安装节点，即maser和slave节点：

搜索结果：

选择parcel为自己的搭建的库：

集群安装时，选择自定义存储库：

提供主机密码：

进入安装环节：

安装完成：

parcel分发：

正确性检查：

选择需要安装的服务：

根据需求配置服务：

设置数据库：

此处需要去master的数据库进行设置：

#登录数据库
[root@master ~]# mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 365
Server version: 5.6.37 MySQL Community Server (GPL)

Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> 
#建立用户并赋予权限
#创建hive用户
mysql> grant all privileges on hive.* to 'hive'@'master' identified by 'hive';
Query OK, 0 rows affected (0.01 sec)

mysql> grant all privileges on hive.* to 'hive'@'%' identified by 'hive';
Query OK, 0 rows affected (0.00 sec)

#创建amon用户
mysql> grant all privileges on amon.* to 'amon'@'master' identified by 'amon';
Query OK, 0 rows affected (0.00 sec)

mysql> grant all privileges on amon.* to 'amon'@'%' identified by 'amon';
Query OK, 0 rows affected (0.00 sec)

#创建oozie用户
mysql> grant all privileges on oozie.* to 'oozie'@'master' identified by 'oozie';
Query OK, 0 rows affected (0.00 sec)

mysql> grant all privileges on oozie.* to 'oozie'@'%' identified by 'oozie';
Query OK, 0 rows affected (0.00 sec)

#创建hue用户
mysql> grant all privileges on hue.* to 'hue'@'master' identified by 'hue';
Query OK, 0 rows affected (0.00 sec)

mysql> grant all privileges on hue.* to 'hue'@'%' identified by 'hue';
Query OK, 0 rows affected (0.00 sec)

#刷新权限
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)


#创建数据库
#创建hive库
mysql> create database hive default character set utf8;
Query OK, 1 row affected (0.00 sec)

#创建amon库
mysql> create database amon default character set utf8;
Query OK, 1 row affected (0.01 sec)

#创建oozie库
mysql> create database oozie default character set utf8;
Query OK, 1 row affected (0.00 sec)

#创建hue库
mysql> create database hue default character set utf8;
Query OK, 1 row affected (0.00 sec)

#完成退出
mysql> exit
Bye

配置完数据库后进行连接测试：

选择配置后，进行安装：

若安装失败时，注意查看日志信息。
安装成功：

管理页面：

总结

安装时遇到各种坑，注意查看日志！！！