CentOS7+Ambria2.6.1.0+HDP2.6.4.0安装详细步骤
Ambria安装
1.环境准备
（在master，slave1，slave2，slave3）
1.1. 环境说明
| master | slave1 | slave2 | slave3 | |
|---|---|---|---|---|
| 主机名 | master.ambria.com | slave1.ambria.com | slave2.ambria.com | slave3.ambria.com |
| IP | 192.168.18.140 | 192.168.18.141 | 192.168.18.42 | 192.168.18.43 |
| Netmask | 255.255.255.0 | 255.255.255.0 | 255.255.255.0 | 255.255.255.0 |
| Gateway | 192.168.18.2 | 192.168.18.2 | 192.168.18.2 | 192.168.18.2 |
| DNS1 | 192.168.18.2 | 192.168.18.2 | 192.168.18.2 | 192.168.18.2 |
1.2 修改主机名称及映射
[root@localhost X ~]# vi /etc/hostname
master
slave1
slave2
slave3
[root@X ~]# vi /etc/hosts
192.168.18.140 master
192.168.18.141 slave1
192.168.18.142 slave2
192.168.18.143 slave3
1.3免密登录 (X:代表所有)
要使Ambari Server自动在所有群集主机上安装Ambari代理，必须在Ambari Server主机与群集中所有其他主机之间设置无密码SSH连接。Ambari服务器主机使用SSH公钥身份验证来远程访问和安装Ambari代理。
[root@X ~]# ssh-keygen -t rsa //一直回车生成秘钥
# 将slave1，slave2，slave3公钥拷贝到master的authorized_keys中
[root@slaveX ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub master //在slave1，slave2，slave3中执行
[root@master ~]# cat ~/.ssh/id_rsa.pub >> authorized_keys
# 将master中的公钥拷贝到 slave1，slave2，slave3中
[root@master ~]# scp -p ~/.ssh/authorized_keys root@slave1:~/.ssh/authorized_keys
[root@master ~]# scp -p ~/.ssh/authorized_keys root@slave2:~/.ssh/authorized_keys
[root@master ~]# scp -p ~/.ssh/authorized_keys root@slave3:~/.ssh/authorized_keys
[root@master .ssh]# chmod 700 ~/.ssh
[root@master .ssh]# chmod 600 ~/.ssh/authorized_keys
[root@X ~]# systemctl restart sshd.service
2.基础搭建
2.1 设置network
#每台节点里配置FQDN
[root@X~]# vim /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=master
2.2. 修改ulimit
（最大进程数和最大文件打开数)
[root@X~]$ vim /etc/security/limits.conf
* soft nofile 204800
* hard nofile 204800
* soft nproc 204800
* hard nproc 204800
[root@X~]$ vim /etc/security/limits.d/20-nproc.conf
# * soft nproc 4096
* soft nproc 204800
* hard nproc 204800
root soft nproc unlimited
2.3. 安装与配置NTP
用主节点作为时间服务器
[root@X~]# yum install unzip wget ntp
[root@X~]# yum update
[root@X~]# yum install unzip wget ntp
[root@X~]# systemctl is-enabled ntpd
disabled
[root@X~]# systemctl enable ntpd
Created symlink from /etc/systemd/system/multi-u ser.target.wants/ntpd.service to /usr/lib/system d/system/ntpd.service.
[root@X~]# systemctl start ntpd
验证时间是否已同步
[root@master ~]# date
Mon Dec 9 09:13:55 EST 2019
2.4 关闭防火墙
[root@X~]# systemctl stop firewalld
[root@X~]# systemctl disable firewalld
2.5 禁用 Selinux
#修改/etc/selinux/config文件中设置SELINUX=disabled ，然后重启服务器
[root@X ~]# vim /etc/selinux/config
#修改内容：
SELINUX=disabled
重启机器才生效
检查 selinux 状态
3. 本地安装 Ambria，HDP
| 软件 | 版本 |
|---|---|
| Ambari | ambari-2.6.1.0-centos7 |
| HDP | HDP-2.6.4.0-centos7-rpm |
| HDP-UTILS | HDP-UTILS-1.1.0.22-centos7 |
| JDK | jdk-8u171-linux-x64 |
| JCE | jce_policy-8 |
| MYSQL | 5.7.28 |
3.1 安装 jdk
解压
[root@X~]# mkdir -p /opt/java/
[root@X~]# tar -zxvf jdk-8u171-linux-x64.tar.gz -C /opt/java/
配置环境变量
[root@X~] # vim /etc/profile
JAVA_HOME=/opt/java/jdk1.8.0_171
CLASSPATH=$JAVA_HOME/lib/
PATH=$PATH:$JAVA_HOME/bin
export PATH
export JAVA_HOME
export CLASSPATH
[root@X~] # source /etc/profile
[root@X~] # java -version
3.2 安装JCE
[root@master ~]# mkdir -p /usr/jce_policy-8
[root@master ~]# yum install unzip
[root@master ~]# unzip -o -j -q jce_policy-8.zip -d /usr/jce_policy-8
将解压后软件包分发到其它服务器
[root@master ~]# scp -r /usr/jce_policy-8 root@slave1:/usr/
[root@master ~]# scp -r /usr/jce_policy-8 root@slave2:/usr/
[root@master ~]# scp -r /usr/jce_policy-8 root@slave3:/usr/
复制Jar包
[root@X]# cp /usr/jce_policy-8/local_policy.jar /opt/java/jdk1.8.0_171/jre/lib/security/
[root@X]# cp /usr/jce_policy-8/US_export_policy.jar /opt/java/jdk1.8.0_171/jre/lib/security/
[root@X]# cp /usr/jce_policy-8/README.txt /opt/java/jdk1.8.0_171/jre/lib/security/
3.3 安装MySQL数据库
（只在Master上）
[root@master opt]# cd /opt
[root@master opt]# wget http://repo.mysql.com/mysql57-community-release-el7-8.noarch.rpm
[root@master opt]# yum install mysql57-community-release-el7-8.noarch.rpm
[root@master opt]# yum install mysql-server
启动mysql服务
[root@master opt]# systemctl start mysqld
[root@master opt]# systemctl enable mysqld
# 查看临时密码
[root@master opt]# grep 'temporary password' /var/log/mysqld.log
重置root密码(规范密码设置)
[root@master opt]# mysql -u root -pOg=ymk0V=d51 临时密码 keirGxno4a:7
mysql> set global validate_password_policy=0;
mysql> set global validate_password_length=3;
mysql> set global validate_password_mixed_case_count=0;
mysql> set global validate_password_number_count=0;
mysql> set global validate_password_special_char_count=0;
mysql> alter user 'root'@'localhost' identified by '123456';
mysql> flush privileges;
mysql> SHOW VARIABLES LIKE 'validate_password%';
开放数据库访问权限
mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123456' WITH GRANT OPTION;
mysql> FLUSH PRIVILEGES;
mysql> quit
3.4 安装mysql jdbc 驱动
[root@master ~]# tar -zxvf mysql-connector-java-5.1.48.tar.gz -C /opt/java
[root@master ~]# vim /etc/profile
export CLASSPATH=$CLASSPATH:/opt/java/mysql-connector-java-5.1.48/mysql-connector-java-5.1.48.jar
[root@master ~]# source /etc/profile
3.5 准备
安装制作本地源工具
[root@X~]# yum install yum-utils createrepo
创建一个HTTP服务器
[root@X~]# yum install httpd -y
[root@X~]# systemctl enable httpd && systemctl start httpd
master
3.6 解压 ambria+HDP
为Web服务器创建目录
[root@X~]# mkdir -p /var/www/html/hdp/
[root@X~]# tar -zxvf ambari-2.6.1.0-centos7.tar.gz -C /var/www/html/
[root@X~]# tar -zxvf HDP-2.6.4.0-centos7-rpm.tar.gz -C /var/www/html/hdp/
[root@X~]# tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/hdp/
[root@X~]# tar -zxvf HDP-GPL-2.6.4.0-centos7-rpm.tar.gz -C /var/www/html/hdp/
查看目录结构
[root@master ~]# tree -L 4 ./
测试
http://master/ambari/centos7/2.6.1.0-143/
http://master/hdp/HDP/centos7/2.6.4.0-91/
http://master/hdp/HDP-UTILS/centos7/1.1.0.22/
http://master/hdp/HDP-GPL/centos7/2.6.4.0-91/
3.7. 制作本地源的repo文件
目录 /var/www/html (在master上操作)
[root@X]# cp /var/www/html/ambari/centos7/2.6.1.0-143/ambari.repo /etc/yum.repos.d/
[root@X]# cp /var/www/html/hdp/HDP/centos7/2.6.4.0-91/hdp.repo /etc/yum.repos.d/
[root@X]# cp /var/www/html/hdp/HDP-GPL/centos7/2.6.4.0-91/hdp-gpl.repo /etc/yum.repos.d/
修改ambari.repo文件
[root@master]# vim /etc/yum.repos.d/ambari.repo
#VERSION_NUMBER=2.6.1.5-143
[ambari-2.6.1.0]
name=ambari Version - ambari-2.6.1.0
baseurl=http://master/ambari/centos7/2.6.1.0-143/
gpgcheck=1
gpgkey=http://master/ambari/centos7/2.6.1.0-143/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
修改hdp.repo文件
[root@master]# vim /etc/yum.repos.d/hdp.repo
#VERSION_NUMBER=2.6.4.0-91
[HDP-2.6.4.0]
name=HDP Version - HDP-2.6.4.0
baseurl=http://master/hdp/HDP/centos7/2.6.4.0-91/
gpgcheck=1
gpgkey=http://master/hdp/HDP/centos7/2.6.4.0-91/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
[HDP-UTILS-1.1.0.22]
name=HDP-UTILS Version - HDP-UTILS-1.1.0.22
baseurl=http://master/hdp/HDP-UTILS/centos7/1.1.0.22/
gpgcheck=1
gpgkey=http://master/hdp/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
修改hdp.gpl.repo文件
[root@master]# vim /etc/yum.repos.d/hdp.gpl.repo
#VERSION_NUMBER=2.6.4.0-91
[HDP-GPL-2.6.4.0]
name=HDP-GPL Version - HDP-GPL-2.6.4.0
baseurl=http://master/hdp/HDP-GPL/centos7/2.6.4.0-91/
gpgcheck=1
gpgkey=http://master//hdp/HDP-GPL/centos7/2.6.4.0-91/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
拷贝到其他节点（X代表：slave1，slave2，slave3）
# X 代表 1,2,3
[root@master]# scp -p /etc/yum.repos.d/ambari.repo root@slaveX:/etc/yum.repos.d/ambari.repo
[root@master]# scp -p /etc/yum.repos.d/hdp.repo root@slaveX:/etc/yum.repos.d/hdp.repo
[root@master]# scp -p /etc/yum.repos.d/hdp.gpl.repo root@slaveX:/etc/yum.repos.d/hdp.gpl.repo
# yum clean all
# yum makecache
确定yum软件包有效
# yum repolist
4、安装ambari
4.1安装ambari-server
# 仅master上安装 ambari-server
[root@master ~]# yum install ambari-server -y
# 所有的节点上都安装 ambari-agent
#[root@X ~]# yum install ambari-agent -y
4.2 设置mysql连接器
[root@master java]# ambari-server setup --jdbc-db=mysql --jdbc-driver=/opt/java/mysql-connector-java-5.1.48/mysql-connector-java-5.1.48.jar
4.3.创建数据库
[root@master java]# mysql -uroot -p123456
创建ambari数据库
mysql> create database ambari character set utf8;
mysql> create user 'ambari'@'%'IDENTIFIED BY '123456';
mysql> GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';
mysql> FLUSH PRIVILEGES;
# 如果要安装Hive,再创建Hive数据库和用户,再执行下面的语句：
mysql> create database hive character set utf8 ;
mysql> CREATE USER 'hive'@'%'IDENTIFIED BY '123456';
mysql> GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%';
mysql> FLUSH PRIVILEGES;
4.4 配置MySQL的JDBC驱动
[root@master ~]# cp /opt/java/mysql-connector-java-5.1.48/mysql-connector-java-5.1.48-bin.jar /opt/java/jdk1.8.0_171/lib
[root@master ~] mkdir /usr/share/java
[root@master ~]# cp /opt/java/mysql-connector-java-5.1.48/mysql-connector-java-5.1.48-bin.jar /usr/share/java/
[root@master ~]# vim /etc/ambari-server/conf/ambari.properties
server.jdbc.driver.path=/opt/java/jdk1.8.0_171/lib/mysql-connector-java-5.1.48-bin.jar
4.5 配置Ambari
[root@master java]# ambari-server setup
Using python /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'enabled'
SELinux mode is 'permissive'
WARNING: SELinux is set to 'permissive' mode and temporarily disabled.
OK to continue [y/n] (y)? y
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root): #回车
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? y
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1): 3
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /opt/java/jdk1.8.0_171
Validating JDK on Ambari Server...done.
Checking GPL software agreement...
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (3): 3
Hostname (master):
Port (3306):
Database name (ambari):
Username (root):
Enter Database Password (123456):
Configuring ambari database...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
ambari-admin-2.6.1.0.143.jar
...........
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.
4.6 将Ambari数据库脚本导入到数据库
[root@master java]# mysql -uroot -p123456
mysql> use ambari;
mysql> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
5. 启动测试
ambari-server start 启动
ambari-server stop 停止
ambari-server restart 重启
ambari-agent start 启动
ambari-agent stop 停止
ambari-agent status 查看状态
5.1 进入Web界面配置
http://master:8080
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-YcHiqR2n-1576205212398)(C:\Users\hezq\AppData\Roaming\Typora\typora-user-images\1576148697489.png)]
6.创建集群实例
首先、安装之前，在每个主机节点上执行下面命令，已清除缓存，避免一些repo原因导致的安装失败。
yum clean all
集群规划
| master | slave1 | slave2 | slave3 | |
|---|---|---|---|---|
| ambari server | 是 | |||
| ambari agent | 是 | 是 | 是 | 是 |
| NameNode | 是 | |||
| DataNode | 是 | 是 | 是 | |
| SNameNode | 是 | |||
| NodeManager | 是 | 是 | 是 | |
| Resourcemanager | 是 | |||
| journalnode | 是 | 是 | 是 | 是 |
| zookeeper | 是 | 是 | 是 | |
| NFSGateway | ||||
| spark | 是 | 是 | 是 | |
| Broker | 是 | |||
| flume | 是 | |||
| HregionServer | 是 | 是 | 是 |
1.创建
点击 Launch Install Wizard
2. 编辑集群名称
AerospaceBigData
3.选择的对应的版本及本地源路径
确保和配好的hdp.repo中保持一致
http://master/hdp/HDP/centos7/2.6.4.0-91/
http://master/hdp/HDP-GPL/centos7/2.6.4.0-91/
http://master/hdp/HDP-UTILS/centos7/1.1.0.22/
4.填写集群的主机名称及私钥
master
slave1
slave2
slave3
其中下面的为master主机的ssh私钥（~/.ssh/id_rsa）
注意：1. 确保master和slave之间能 ssh通
2. 在复制的时候不能有换行符，否则会出现不能识别主机名称的错误
3.出现以下警告，不用理会，然后点击OK
原因：主机名称映射没有按照FQDNs的格式映射，即 ambari.master.com 的格式
5.注册+ host确认
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-PdeMgXgF-1576205212401)(C:\Users\hezq\AppData\Roaming\Typora\typora-user-images\1576199622059.png)]
注册成功后点击上边的红色标记的连接，确保没有警告（最好），然后点击Next
6.选择要安装的服务
下边以zookeeper安装为例
7.各个服务Master配置
这里可以根据自己的需要进行调整服务安装在哪个服务节点上，然后点击Next
（这里Metrics Collector，Grafana，Activity Analyzer，Activity Explorer，HST Server服务位置不要动，其他自己选择的可以调整）
8.服务的Slaves 和 Clients节配置
9.定制服务
确保没有红色错误显示，然后点击Next
这里选择默认的密码admin
10.显示配置信息
在这里可以看到选择的服务安装的主机信息，确认后点击Deploy
11、开始安装
（安装各个服务，并且完成安装后会启动相关服务，安装过程比较长，如果中途出现错误，请根据具体提示或者log进行操作）
（1）正在安装，耐心等待
（2）如果其中出现警告不用理会，不影响正常的使用，然后点击Next
（3）系统的总结，点击Complete
（4）分析图形化展示
到这里离就都安装完成，可以查看安装的进程了
7.问题解决
问题1：
ERROR 2019-12-11 09:45:08,423 NetUtil.py:96 - EOF occurred in violation of protocol (_ssl.c:618)
ERROR 2019-12-11 09:45:08,423 NetUtil.py:97 - SSLError: Failed to connect. Please check openssl library versions.
解决方式：
修改 ambari-agent.ini 文件
vim /etc/ambari-agent/conf/ambari-agent.ini
在［security］标签下面增加一项
[security]
force_https_protocol=PROTOCOL_TLSv1_2
问题2：
在（11）安装失败后卸载警告应用
卸载应用前，先确定应用的安装路径
查找
[root@X ~]# rpm -qa|grep 应用名称
rpm -qa|grep hdp-select
rpm -qa|grep ambari-metrics-monitor
rpm -qa|grep ambari-metrics-grafana
rpm -qa|grep smartsense-hst
rpm -qa|grep ambari-metrics-hadoop-sink
卸载警告中应该不存在的应用
rpm -e smartsense-hst-1.4.4.2.6.1.0-143.x86_64
rpm -e hdp-select-2.6.4.0-91.noarch
rpm -e ambari-metrics-monitor-2.6.1.0-143.x86_64
rpm -e ambari-metrics-grafana-2.6.1.0-143.x86_64
rpm -e ambari-metrics-hadoop-sink-2.6.1.0-143.x86_64
删除 Ambari 自动创建的用户及用户组
userdel -r zookeeper
userdel -r ams
userdel -r ambari-qa
userdel -r hdfs
userdel -r yarn
userdel -r mapred
userdel -r activity_analyzer
userdel hcat
userdel kafka
userdel hbase
userdel tez
groupdel hdfs
groupdel hadoop
groupdel zookeeper
groupdel yarn
groupdel mapred
groupdel hcat
groupdel kafka
groupdel hbase
groupdel tez
问题3：
如果出现这类错误
The 'hadoop-hdfs-client' component did not advertise a version. This may indicate a problem with the component packaging.
请检查本地源路径是和否配置正确