基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台

1. 集群规划、配置说明

1.1 系统硬件说明

操作系统 主机名 内网IP 内存 CPU 系统盘容量 数据盘容量 数据盘挂载点
CentOS-7.8.2003 hdp01 10.0.0.105 44GB 4核 100GB 200GB /data
CentOS-7.8.2003 hdp02 10.0.0.106 24GB 4核 100GB 200GB /data
CentOS-7.8.2003 hdp03 10.0.0.107 20GB 4核 100GB 200GB /data
CentOS-7.8.2003 hdp04 10.0.0.108 20GB 4核 100GB 200GB /data

1.2 组件、角色规划

主机名 角色 安装组件或服务
hdp01 管理节点、数据节点 Ambari-Server、Ambari-Agent、Apache、MySQL
hdp02 数据节点 Ambari-Agent
hdp03 数据节点 Ambari-Agent
hdp04 数据节点 Ambari-Agent

1.3 软件准备

安装包文件名 下载地址
ambari-2.7.4.0-centos7.tar.gz http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.4.0/ambari-2.7.4.0-centos7.tar.gz
HDP-3.1.4.0-centos7-rpm.tar.gz http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.0/HDP-3.1.4.0-centos7-rpm.tar.gz
HDP-UTILS-1.1.0.22-centos7.tar.gz http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
HDP-GPL-3.1.4.0-centos7gpl.tar.gz http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.1.4.0/HDP-GPL-3.1.4.0-centos7gpl.tar.gz
mysql-connector-java-8.0.22.jar https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.22/mysql-connector-java-8.0.22.jar

2. 操作系统准备工作

2.1 准备一个具有sudo权限的普通用户

我这里创建了一个普通用户名为admin,并且具有sudo权限,4个节点都需要有这个用户。

[admin@hdp01 ~]$ sudo vim /etc/sudoers
## Allow root to run any commands anywhere 
root    ALL=(ALL)       ALL
# 添加这一行配置
admin   ALL=(ALL)       NOPASSWD:ALL

# admin用户使用sudo命令可以无密码切换到root用户
[admin@hdp01 ~]$ sudo su - 
Last login: Thu Oct 22 21:12:24 CST 2020 on pts/0
[root@hdp01 ~]#

2.2 hosts文件

4个节点的配置保持同步

# 文件:/etc/hosts
# 注意,前两行配置不要删除
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.0.105      hdp01
10.0.0.106      hdp02
10.0.0.107      hdp03
10.0.0.108      hdp04

2.3 关闭防火墙

4个节点都要操作

# 关闭防火墙
sudo systemctl stop firewalld
# 禁止防火墙开机启动
sudo systemctl disable firewalld
# 查看防火墙状态
sudo systemctl status firewalld

# 确认4个节点的防火墙都是 inactive 和 disabled 状态
[root@hdp01 ~]# sudo systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:firewalld(1)

注意:生产环境一般由专业运维人员来限制相关端口,而不是完全禁用防火墙

2.4 禁用SELINUX

4个节点都要操作

# 首先临时关闭SELINUX
sudo setenforce 0

# 然后修改配置文件,永久关闭SELINUX
# 文件:/etc/selinux/config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
# 确保 SELINUX=disabled
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

先临时修改,后永久修改的好处:

如果只是临时修改,那么重启机器之后设置就会失效;如果只是永久修改,那么必须重启机器才可能让设置生效,产线环境是不可能重启机器的;这样先临时修改,让其生效,让永久修改,那么在下一次服务器重启后也不会担心这个配置失效。

2.5 umask设置

4个节点都要操作

# 文件:/etc/profile
# 在最后一行添加如下内容
umask 0022

# 修改完之后使用source命令使其生效
source /etc/profile

# 在4个节点确保配置生效,0022是应该出现的结果
[root@hdp01 ~]# umask
0022

2.6 文件描述符配置

4个节点都要操作

# 文件:/etc/security/limits.conf
# 在文件末尾追加如下内容
*       soft    nofile  131072
*       hard    nofile  131072

注意:修改之后,重启服务器生效

2.7 SSH优化

4个节点都要操作

# 文件:/etc/ssh/sshd_config
# 此配置默认为yes,改为no
GSSAPIAuthentication no
# 此配置默认是注释掉的,放开即可
UseDNS no

# 保存退出之后重启ssh服务
sudo systemctl restart sshd

2.8 SSH免秘钥配置

设置hdp01机器远程连接自身以及远程连接其他3个机器免秘钥

[admin@hdp01 ~]$ ssh-keygen
Generating public/private rsa key pair.
# 直接Enter
Enter file in which to save the key (/home/admin/.ssh/id_rsa):
# 直接Enter
Enter passphrase (empty for no passphrase): 
# 直接Enter
Enter same passphrase again: 
Your identification has been saved in /home/admin/.ssh/id_rsa.
Your public key has been saved in /home/admin/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:tAH9HF2xMFTCpmxDN7vjXat1NI/oPJX1avACPTi0UMU admin@hdp01
The key's randomart image is:
+---[RSA 2048]----+
|      ..   **o+. |
|       .. + E+ . |
|        o* = o.  |
|       ..oO .   .|
|        S+ = . +o|
|          = *.oo=|
|           +.B.++|
|           o+ *..|
|            o=.  |
+----[SHA256]-----+

[admin@hdp01 ~]$ ssh-copy-id -i ~/.ssh/id_rsa.pub admin@hdp01
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/admin/.ssh/id_rsa.pub"
The authenticity of host 'hdp01 (10.0.0.105)' can't be established.
ECDSA key fingerprint is SHA256:/eVZsOkuW8gHUcJxKNvtRh9yJ3UYDz4p9hA4W4CpeEg.
ECDSA key fingerprint is MD5:8c:e1:a8:0c:c9:80:e5:28:7a:25:96:32:c7:5f:25:47.
# 输入yes
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
# 输入dmin@hdp01的密码
admin@hdp01's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'admin@hdp01'"
and check to make sure that only the key(s) you wanted were added.

# 继续给其他机器添加hdp01节点的公钥,过程略...
ssh-copy-id -i ~/.ssh/id_rsa.pub admin@hdp02
ssh-copy-id -i ~/.ssh/id_rsa.pub admin@hdp03
ssh-copy-id -i ~/.ssh/id_rsa.pub admin@hdp04

# 以上操作执行完成之后,修改.ssh目录和authorized_keys文件的权限,4个节点都要执行
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

# 测试,每次ssh到其他机器都不需要输入密码
[admin@hdp01 ~]$ ssh hdp01
Last login: Thu Oct 22 21:11:53 2020 from 10.0.0.1
[admin@hdp01 ~]$ exit
logout
Connection to hdp01 closed.
[admin@hdp01 ~]$ ssh hdp02
Last login: Thu Oct 22 21:11:53 2020 from 10.0.0.1
[admin@hdp02 ~]$ exit
logout
Connection to hdp02 closed.
[admin@hdp01 ~]$ ssh hdp03
Last login: Thu Oct 22 21:11:53 2020 from 10.0.0.1
[admin@hdp03 ~]$ exit
logout
Connection to hdp03 closed.
[admin@hdp01 ~]$ ssh hdp04
Last login: Thu Oct 22 21:11:53 2020 from 10.0.0.1
[admin@hdp04 ~]$ exit
logout
Connection to hdp04 closed.
[admin@hdp01 ~]$

2.9 时钟同步

  • 4个节点安装Chrony软件,CentOS7默认已经安装了,如果没有安装,执行以下命令

    sudo yum -y install chrony
    
  • 规划

    主机名 Chrony角色
    hdp01 Chrony Server
    hdp02 Chrony Client
    hdp03 Chrony Client
    hdp04 Chrony Client
  • hdp01节点修改配置

    # 文件:/etc/chrony.conf
    
    # 注释掉以下4行
    #server 0.centos.pool.ntp.org iburst
    #server 1.centos.pool.ntp.org iburst
    #server 2.centos.pool.ntp.org iburst
    #server 3.centos.pool.ntp.org iburst
    
    # 添加下面7行
    # 阿里云提供了7个NTP时间服务器
    server ntp1.aliyun.com iburst
    server ntp2.aliyun.com iburst
    server ntp3.aliyun.com iburst
    server ntp4.aliyun.com iburst
    server ntp5.aliyun.com iburst
    server ntp6.aliyun.com iburst
    server ntp7.aliyun.com iburst
    
    # 添加下面3行
    # 设置本机为Chrony Server
    server 127.127.1.1 iburst
    # 允许10.0.0.0网段的机器从本机同步时间
    allow 10.0.0.0/24
    # 允许本地同步时间
    allow 127/8
    
  • hdp02-hdp04节点配置

    # 文件:/etc/chrony.conf
    
    # 注释掉以下4行
    #server 0.centos.pool.ntp.org iburst
    #server 1.centos.pool.ntp.org iburst
    #server 2.centos.pool.ntp.org iburst
    #server 3.centos.pool.ntp.org iburst
    
    # 添加如下一行
    # 从hdp01服务器同步时间
    server hdp01 iburst
    
  • 4个节点启动Chrony服务,并设置Chrony服务开机自启动

    # 如果Chrony服务开启,应该重启
    sudo systemctl restart chronyd
    # 如果Chrony服务未开启,应该启动
    sudo systemctl start chronyd
    # 设置开机自启动
    sudo systemctl enable chronyd
    # 查看运行状态
    sudo systemctl status chronyd
    
  • 验证

    # ^* 代表机器正在从这个服务器同步时间,hdp01显示从120.25.115.20(这是一个阿里云时间服务器的地址)这个地址同步时间
    [admin@hdp01 ~]$ chronyc sources
    210 Number of sources = 3
    MS Name/IP address         Stratum Poll Reach LastRx Last sample               
    ===============================================================================
    ^+ 127.127.1.1                   3   6    77     2  -2979ns[-2979ns] +/-   23ms
    ^* 120.25.115.20                 2   6    77     6   -206us[  -35us] +/-   22ms
    ^+ 203.107.6.88                  2   6    77     6   +761us[ +761us] +/-   18ms
    
    # hdp02-hdp04从hdp01同步时间
    # chrony服务刚启动之后,是"^? hdp01"状态,需要等几分钟才能变成"^* hdp01"状态
    # hdp03、hdp04应该看到相同的情况,这里就不再贴代码了
    [admin@hdp02 ~]$ chronyc sources
    210 Number of sources = 1
    MS Name/IP address         Stratum Poll Reach LastRx Last sample               
    ===============================================================================
    ^* hdp01                         3   6    37    25   +373us[ -124us] +/-  101ms
    

2.10 安装JDK

在4个节点都安装最新稳定版JDK1.8,关于JDK的安装就不再赘述了

[admin@hdp01 ~]$ echo $JAVA_HOME
/usr/java/jdk

[admin@hdp01 ~]$ java -version
java version "1.8.0_251"
Java(TM) SE Runtime Environment (build 1.8.0_251-b08)
Java HotSpot(TM) 64-Bit Server VM (build 25.251-b08, mixed mode)

[admin@hdp01 ~]$ tail -4 /etc/profile
export JAVA_HOME=/usr/java/jdk
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib
export PATH=$JAVA_HOME/bin:$PATH

3. Ambari Server安装部署

3.1 安装MySQL

3.1小节的操作在hdp01节点进行

# HDP3.1要求MySQL的版本大于5.5,这里选择5.7
# 安装MySQL的Yum仓库
sudo yum -y localinstall https://dev.mysql.com/get/mysql57-community-release-el7-8.noarch.rpm
# 安装MySQL
sudo yum -y install mysql-community-server
# 创建数据目录
sudo mkdir -p /data/mysql
sudo chown -R mysql:mysql /data/mysql

# 文件:/etc/my.cnf,内容:
[client]
default-character-set=utf8mb4

[mysql]
socket=/var/lib/mysql/mysql.sock
default-character-set=utf8mb4

[mysqld]
socket=/var/lib/mysql/mysql.sock
symbolic-links=0
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
# 注意将数据目录修改为/data目录,/data目录的磁盘是独立挂载的大容量磁盘
datadir=/data/mysql
character_set_server=utf8mb4
character-set-client-handshake=FALSE
collation-server=utf8mb4_unicode_ci
max_connections=800
max_connect_errors=1000

# 设置开机自启动
sudo systemctl enable mysqld
# 启动MySQL
sudo systemctl start mysqld
# 查看运行状态
sudo systemctl status mysqld

# 获取临时密码,注意每个人的临时密码是不同的,我这里是"Vnyib*Yja9v/"
[root@hdp01 ~]# sudo grep 'temporary password' /var/log/mysqld.log
2020-10-26T13:51:29.134320Z 1 [Note] A temporary password is generated for root@localhost: Vnyib*Yja9v/

# 进入MySQL操作
[root@hdp01 ~]# mysql -uroot -p"Vnyib*Yja9v/"
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.7.32

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> set global validate_password_policy=0;
Query OK, 0 rows affected (0.00 sec)

mysql> set global validate_password_mixed_case_count=0;
Query OK, 0 rows affected (0.00 sec)

mysql> set global validate_password_number_count=3;
Query OK, 0 rows affected (0.00 sec)

mysql> set global validate_password_special_char_count=0;
Query OK, 0 rows affected (0.00 sec)

mysql> set global validate_password_length=3;
Query OK, 0 rows affected (0.00 sec)

# 在生产上建议把root密码设置的更加复杂,我这里设置的root用户只能本机连接
mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY '123456';
Query OK, 0 rows affected (0.00 sec)

mysql> exit
Bye

# 将mysql-connector-java-8.0.22.jar上传到/usr/share/java目录下
sudo mkdir -p /usr/share/java
sudo mv ~/mysql-connector-java-8.0.22.jar /usr/share/java/

3.2 准备本地yum源

3.2小节的操作在hdp01节点进行

#安装Apache
sudo yum -y install httpd
#讲Apache设置为开机启动
sudo systemctl enable httpd
#启动Apache
sudo systemctl start httpd

如果能看到如下界面Apache就安装好了

基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第1张图片

HDP安装包部署到Apache:

# 首先把下载好的ambari和HDP的3个安装包上传到home目录下

cd /var/www/html/
sudo mv ~/ambari-2.7.4.0-centos7.tar.gz /var/www/html/
sudo mv ~/HDP-3.1.4.0-centos7-rpm.tar.gz /var/www/html/
sudo mv ~/HDP-GPL-3.1.4.0-centos7-gpl.tar.gz /var/www/html/
sudo mv ~/HDP-UTILS-1.1.0.22-centos7.tar.gz /var/www/html/

#解压缩
sudo tar -zxvf ambari-2.7.4.0-centos7.tar.gz
sudo tar -zxvf HDP-3.1.4.0-centos7-rpm.tar.gz
sudo tar -zxvf HDP-GPL-3.1.4.0-centos7-gpl.tar.gz
sudo tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz

#安装包清理
sudo rm -r ambari-2.7.4.0-centos7.tar.gz
sudo rm -r HDP-3.1.4.0-centos7-rpm.tar.gz
sudo rm -r HDP-GPL-3.1.4.0-centos7-gpl.tar.gz
sudo rm -r HDP-UTILS-1.1.0.22-centos7.tar.gz

#更改目录权限
sudo chown -R admin:admin /var/www/html
sudo chmod -R ugo+rX /var/www/html/

最终,4个安装包都可以在页面访问,其URL分别为:

安装包 URL
ambari http://hdp01/ambari/centos7/2.7.4.0-118/
HDP http://hdp01/HDP/centos7/3.1.4.0-315/
HDP-GPL http://hdp01/HDP-GPL/centos7/3.1.4.0-315/
HDP-UTILS http://hdp01/HDP-UTILS/centos7/1.1.0.22/

配置本地yum源:

# 文件:/etc/yum.repos.d/ambari.repo,内容如下:
[ambari-repo]
name=ambari
baseurl=http://hdp01/ambari/centos7/2.7.4.0-118/
gpgcheck=0
enabled=1

# 验证,应该看到ambari-repo那一行
[admin@hdp01 yum.repos.d]$ yum repolist
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
repo id               repo name                                          status
ambari-repo           ambari                                             13
base/7/x86_64         CentOS-7 - Base - mirrors.aliyun.com               10,070
epel/x86_64           Extra Packages for Enterprise Linux 7 - x86_64     13,453
extras/7/x86_64       CentOS-7 - Extras - mirrors.aliyun.com             413
updates/7/x86_64      CentOS-7 - Updates - mirrors.aliyun.com            1,134
repolist: 25,083

这样,本地yum源就创建好了。

3.3 安装Ambari-Server

3.3小节的操作在hdp01节点进行

# 安装ambari-server
sudo yum -y install ambari-server
# 核心日志信息
Installed:
  ambari-server.x86_64 0:2.7.4.0-118

# 数据库操作
[admin@hdp01 ~]$ mysql -uroot -p123456
mysql> set global validate_password_policy=0;
mysql> set global validate_password_mixed_case_count=0;
mysql> set global validate_password_number_count=3;
mysql> set global validate_password_special_char_count=0;
mysql> set global validate_password_length=3;
mysql> create database ambari;
mysql> CREATE USER 'ambari'@'%' IDENTIFIED BY '123456';
mysql> GRANT ALL ON ambari.* TO 'ambari'@'%';
mysql> FLUSH PRIVILEGES;
mysql> use ambari;
mysql> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql;
mysql> exit

# 配置AmbariServer
[admin@hdp01 ~]$ sudo ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
# 是否自定义ambari-server的运行用户,我们保持默认root用户即可,所以输入n并回车
Customize user account for ambari-server daemon [y/n] (n)? n
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
==============================================================================
# 选择2,代表自定义JDK,然后回车
Enter choice (1): 2
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
# 指定自己安装的JDK的路径/usr/local/jdk,然后回车
Path to JAVA_HOME: /usr/java/jdk
Validating JDK on Ambari Server...done.
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
# 提示是否下载并安装LZO的包,我们输入y并回车
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? y  
Completing setup...
Configuring database...
# 是否进入数据库高级设置,我们输入y然后回车
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
# 在选择数据库的环节,我们选择3也就是MySQL,然后回车
Enter choice (1): 3
# MySQL的相关配置我们均采用默认值,默认用户和db都是ambari,密码输入123456,然后回车
Hostname (localhost): 
Port (3306): 
Database name (ambari): 
Username (ambari): 
Enter Database Password (bigdata): 
Re-enter password: 
Configuring ambari database...
# 我们需要指定MySQL的驱动包所在的路径/usr/share/java/mysql-connector-java-8.0.22.jar,然后回车
Enter full path to custom jdbc driver: /usr/share/java/mysql-connector-java-8.0.22.jar
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
# 选择可以远程连接该数据库(y),然后回车就配置完毕了
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
ambari-admin-2.7.4.0.118.jar
....
Ambari repo file doesn't contain latest json url, skipping repoinfos modification
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

# 启动Ambari-Server
sudo ambari-server start

# 如果出现错误,查看日志/var/log/ambari-server/ambari-server.log
# 出现如下信息就是启动成功了
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start.................................
Server started listening on 8080

DB configs consistency check: no errors and warnings were found.
Ambari Server 'start' completed successfully.

访问WEB-UI:

基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第2张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第3张图片

如果能看到上面的页面,那么Ambari-Server就安装成功了!

4. Ambari+HDP大数据平台部署

4.1 部署Ambari集群和HDFS集群

基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第4张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第5张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第6张图片

接着要把滚动条往下拉,把无用的其他操作系统的仓库删掉:

基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第7张图片

最终我们只保留"readhat7"的仓库,接下来要把直接部署yum本地仓库的地址填进去,同时不要勾选下面的两个高级选项,如下图:

基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第8张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第9张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第10张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第11张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第12张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第13张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第14张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第15张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第16张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第17张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第18张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第19张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第20张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第21张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第22张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第23张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第24张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第25张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第26张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第27张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第28张图片

4.2 部署YARN集群

基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第29张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第30张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第31张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第32张图片

基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第33张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第34张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第35张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第36张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第37张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第38张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第39张图片

4.3 部署HBase集群

后续一些重复的图片就不再贴出来了。

基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第40张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第41张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第42张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第43张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第44张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第45张图片

4.4 部署Hive

# MySQL操作
mysql> create database hive;
mysql> CREATE USER 'hive'@'%' IDENTIFIED BY '123456';
mysql> GRANT ALL ON hive.* TO 'hive'@'%';
mysql> FLUSH PRIVILEGES;
mysql> exit;

# 执行以下命令
sudo ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java-8.0.22.jar
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第46张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第47张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第48张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第49张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第50张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第51张图片

新服务部署后,可能会提示重启其他相关服务,重启即可

基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第52张图片

4.5 部署Kafka

基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第53张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第54张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第55张图片

4.6 部署Spark2+Sqoop

基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第56张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第57张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第58张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第59张图片
基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台_第60张图片

剩余的组件的安装都是一样的,有需要的话直接在页面操作安装即可,要注意的就是需要修改日志目录和数据目录。

至此,Ambari2.7+HDP3.1大数据平台就安装部署完成了!

你可能感兴趣的:(基于CentOS7.8安装Ambari2.7+HDP3.1大数据平台)