CDH6.2搭建之安装
环境准备：
宿主机 Centos Liunx release 7.6.1810 128G内存 2T硬盘
VirtualBox 6.0.x86_64
虚拟机： 3台；用户cdh
配置
内存：32G
存储：200G
系统：Centos Liunx release 7.6.1810
网络：
一、准备工作
1、网络设置，要求每台机器可以访问互联网，并且可以互通；
这里我的VirtualBox设置是：
网卡一设置为：网络地址转换（NAT）
网卡二设置为：桥接网卡，指定到联网的那个网卡（找不到，自行百度）;
2、主机名设置（所有机器）
vim /etc/hostname修改为 cdh.a.com
196.168.89.61 cdh.a.com 主节点
196.168.89.62 cdh.b.com
196.168.89.63 cdh.c.com
3、防火墙设置
sudo systemctl stop firewalld.service sudo systemctl disable firewalld.servicesudo vim /etc/selinux/config修改SELINUX=disabled
sudo sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config4、ssh免密登陆
所有节点 [cdh.a.com,cdh.b.com,cdh.c.com]
sudo ssh-keygen -t rsa主节点 cdh.a.com
[cdh@com ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/cdh/.ssh/id_rsa):
Created directory '/home/cdh/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/cdh/.ssh/id_rsa.
Your public key has been saved in /home/cdh/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:kggyD9C2N4dDzRjlxwPkzM9hFEO/81AwWOlUNFo7jiE [email protected]
The key's randomart image is:
+---[RSA 2048]----+
|.. .B+.=+++= |
|. o o=o+.oo= o |
|+..o .= Eo+ + |
| =..=..* +.* . |
| ...+o S = . |
| . + |
| . |
| |
| |
+----[SHA256]-----+
[cdh@com ~]$ ssh-copy-id -i ~/.ssh/id_rsa.pub [email protected]
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/cdh/.ssh/id_rsa.pub"
The authenticity of host 'cdh.a.com (196.168.89.61)' can't be established.
ECDSA key fingerprint is SHA256:fLmm8WivB6o5YzGaMuWPB2sZQvk1W+T3XXc6uCMKWi8.
ECDSA key fingerprint is MD5:33:8d:96:e2:1e:0e:76:fe:4e:04:fc:bb:24:47:0a:2a.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
[email protected]'s password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added.
[cdh@com ~]$ scp ~/.ssh/authorized_keys [email protected]:~/.ssh/
The authenticity of host 'cdh.b.com (196.168.89.62)' can't be established.
ECDSA key fingerprint is SHA256:fLmm8WivB6o5YzGaMuWPB2sZQvk1W+T3XXc6uCMKWi8.
ECDSA key fingerprint is MD5:33:8d:96:e2:1e:0e:76:fe:4e:04:fc:bb:24:47:0a:2a.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'cdh.b.com,196.168.89.62' (ECDSA) to the list of known hosts.
[email protected]'s password:
authorized_keys 100% 395 339.7KB/s 00:00
[cdh@com ~]$ scp ~/.ssh/authorized_keys [email protected]:~/.ssh/
The authenticity of host 'cdh.c.com (196.168.89.63)' can't be established.
ECDSA key fingerprint is SHA256:fLmm8WivB6o5YzGaMuWPB2sZQvk1W+T3XXc6uCMKWi8.
ECDSA key fingerprint is MD5:33:8d:96:e2:1e:0e:76:fe:4e:04:fc:bb:24:47:0a:2a.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'cdh.c.com,196.168.89.63' (ECDSA) to the list of known hosts.
[email protected]'s password:
scp: /home/cdh/.ssh/: Is a directory5、阿里 yum repo (所有节点 [cdh.a.com,cdh.b.com,cdh.c.com])
sudo mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.bak
[cdh@com ~]$ sudo curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2523 100 2523 0 0 45165 0 --:--:-- --:--:-- --:--:-- 458726、lib install (所有节点 [cdh.a.com,cdh.b.com,cdh.c.com])
sudo yum install net-tools vim -y7、安装NTP (所有节点 [cdh.a.com,cdh.b.com,cdh.c.com])
sudo yum install -y ntpsudo vim /etc/ntp.conf主节点（cdh.a.com）修改
server time.windows.com prefer
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org ibursttime.windows.com 是时间服务器，在内网环境可让用户提供统一的时间服务器地址
从节点 （ [cdh.b.com,cdh.c.com]）修改
server cdh.a.com
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst8、JDK安装 这里我选择使用oracle-jkd 所以卸载Centos自带的openjdk
官网JDK要求
JDK 安装，环境变量配置完成以后记得备份一下虚拟机，至此还没有和CDH产生一毛钱关系
二、CM安装 所有节点 [cdh.a.com,cdh.b.com,cdh.c.com]
官方安装说明
yum repo 地址
sudo wget https://archive.cloudera.com/cm6/6.2.0/redhat7/yum/cloudera-manager.repo -P /etc/yum.repos.d/sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server数据库安装配置：CM初始化需要执行数据库脚本，需要一个数据库 数据库说明
安装好以后需要创建一系列的数据库 （我这里使用的是一个已经存在的postgresql，因此没有安装数据库）需要准备一些库
Cloudera Manager Server包含一个可以为自己创建和配置数据库的脚本。该脚本可以：
- 创建Cloudera Manager Server数据库配置文件。
- （MariaDB，MySQL和PostgreSQL）为Cloudera Manager Server创建和配置数据库以供使用。
- （MariaDB，MySQL和PostgreSQL）为Cloudera Manager Server创建和配置用户帐户。
sudo /opt/cloudera/cm/schema/scm_prepare_database.sh postgresql -h 196.168.65.136 --scm-host cdh.a.com scm scm[cdh@com ~]$ sudo /opt/cloudera/cm/schema/scm_prepare_database.sh postgresql -h 196.168.65.136 --scm-host cdh.a.com scm scm
[sudo] cdh 的密码：
Enter SCM password:
JAVA_HOME=/usr/java/jdk1.8.0_181-amd64
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing: /usr/java/jdk1.8.0_181-amd64/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
[ main] DbCommandExecutor INFO Successfully connected to database.
All done, your SCM database is configured correctly!表示我们的数据库已经可以连上了
启动 Cloudera Manager Server
sudo systemctl start cloudera-scm-server查看启动日志
sudo tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log管网交代 当看到"INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server."时表示成功启动了
2019-04-23 15:16:14,568 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Finished constructing repo:2019-04-23T07:16:14.568Z
2019-04-23 15:16:15,596 INFO WebServerImpl:org.eclipse.jetty.server.Server: jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_181-b13
2019-04-23 15:16:15,772 INFO WebServerImpl:org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@78fdd57a{HTTP/1.1,[http/1.1]}{0.0.0.0:7180}
2019-04-23 15:16:15,791 INFO WebServerImpl:org.eclipse.jetty.server.Server: Started @175529ms
2019-04-23 15:16:15,791 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.千呼万唤始出来，OK!CDH至此安装成功
记得虚拟机在备份一下，鬼知道配置的时候会不会调到坑里出不来
http://
