ubuntu 18.04 离线搭建HDP3.1

版本：
操作系统版本：
OS：ubuntu-18.04.5-live-server-amd64.iso
java：jdk1.8.0_141
mysql：mysql5.7（操作系统自带）
mysql-connect-java： mysql-connector-java-5.1.20-bin.jar
HDP：HDP-3.1.0.0-ubuntu18

节点拓扑：

192.168.72.17：Ambari-server/agent安装节点，mysql安装节点，deb数据源安装节点，http服务安装节点

192.168.72.18：Ambari-agent节点

192.168.72.29：Ambari-agent节点

1、设置ssh不超时，并配置hosts（3个节点都执行）
在/etc/profile中设置 export TMOUT=0
在sshd配置文件中添加这个：
ClientAliveInterval 300
ClientAliveCountMax 3

hosts文件配置（3个节点都执行）：

root@host002:~# cat /etc/hosts
127.0.0.1 localhost

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
192.168.72.17 host001
192.168.72.18 host002
192.168.72.29 host003

2、设置ssh互信（注意mater本机自身也要能够自动登录）
在192.168.72.17上执行：ssh-keygen
ssh-copy-id -i /root/.ssh/id_rsa.pub 192.168.72.17
ssh-copy-id -i /root/.ssh/id_rsa.pub 192.168.72.18
ssh-copy-id -i /root/.ssh/id_rsa.pub 192.168.72.29

3、以Ubuntu18.04为例配置阿里源并按照python开发包（3个节点都执行）
cp /etc/apt/sources.list /etc/apt/sources.list.bak 
vim /etc/apt/sources.list 
注释掉之前的官方安装源，添加阿里源
deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
执行：
sudo apt-get update

sudo apt-get upgrade

 安装python 开发工具（3个节点都执行）
Ambari的安装依赖于python2.7，进行python2.7的安装和配置
apt-get install python2.7
apt-get install python-pip
# 升级pip
python -m pip install --upgrade pip
# 配置为国内源
pip config set global.index-url http://mirrors.aliyun.com/pypi/simple/
# 安装python 开发包
apt-get install python2.7-dev
# 检查确认
python -V

4、安装http服务（192.168.72.17执行）
apt-get install apache2
systemctl status apache2
查看服务是否正常

5、放置安装源（192.168.72.17执行）
cd /var/www/html
tar zxvf ambari-2.7.3.0-ubuntu18.tar.gz -C /var/www/html
tar zxvf HDP-3.1.0.0-ubuntu18-deb.tar.gz -C /var/www/html
tar zxvf HDP-GPL-3.1.0.0-ubuntu18-gpl.tar.gz -C /var/www/html
tar zxvf HDP-UTILS-1.1.0.22-ubuntu18.tar.gz -C /var/www/html

6、配置hdp的镜像源（3个节点上都执行）
在/etc/apt/sources.list.d/路径下分别添加 ambari.list 和 ambari-hdp.list
cd /etc/apt/sources.list.d
# ambari.list
deb http://192.168.72.17/ambari/ubuntu18/2.7.3.0-139/ Ambari main
# ambari-hdp.list
deb http://192.168.72.17/HDP/ubuntu18/3.1.0.0-78/ HDP main
deb http://192.168.72.17/HDP-GPL/ubuntu18/3.1.0.0-78/ HDP-GPL main
deb http://192.168.72.17/HDP-UTILS/ubuntu18/1.1.0.22/ HDP-UTILS main

添加key，不然会出现public key校验失败
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys B9733A7A07513CAD

apt-get update

7、安装mysql（默认5.7版本）（192.168.72.17上执行）
apt-get install mysql-server
apt-get install mysql-client
apt-get install libmysqlclient-dev 
设置root密码：
root@mhost001:/etc/apt/sources.list.d# cat /etc/mysql/debian.cnf
# Automatically generated for Debian scripts. DO NOT TOUCH!
[client]
host     = localhost
user     = debian-sys-maint
password = LGyqRd8t5kZUq2Vi
socket   = /var/run/mysqld/mysqld.sock
[mysql_upgrade]
host     = localhost
user     = debian-sys-maint
password = LGyqRd8t5kZUq2Vi
socket   = /var/run/mysqld/mysqld.sock
登录并修改密码：
mysql -u debian-sys-maint -p  LGyqRd8t5kZUq2Vi

use mysql;
update mysql.user set authentication_string=password('root') where user='root' and Host ='localhost';
update user set plugin="mysql_native_password";
flush privileges;
exit;
service mysql restart
mysql -u root -p 输入密码登录验证

8、配置java环境变量（3个节点上都执行）
export JAVA_HOME=/usr/local/java/jdk1.8.0_141
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib:$CLASSPATH
export JAVA_PATH=${JAVA_HOME}/bin:${JRE_HOME}/bin
export PATH=$PATH:${JAVA_PATH}:${JAVA_HOME}:${JRE_HOME}

9、在 mysql 中执⾏ amabri 的建表脚本，同时创建对应的账号密码（192.168.72.17上执行）
SQL
-- 创建ambari专用的数据库
CREATE DATABASE `ambari`;
CREATE USER 'ambari'@'%' IDENTIFIED BY 'bigdata123';
grant all privileges on ambari.* to 'ambari'@'%';
-- 刷新权限
flush privileges;

10、修改mysql监听的ip地址（192.168.72.17上执行）
# localhost which is more compatible and is not less secure.
bind-address            = 0.0.0.0

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:3306            0.0.0.0:*               LISTEN      8718/mysqld
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      752/systemd-resolve
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1192/sshd

11、执行Ambari安装（192.168.72.17上执行）

apt-get install ambari-server
ambari-server setup

12、指定驱动的路径（192.168.72.17上执行）
mv  mysql-connector-java-5.1.20-bin.jar /usr/share/java/mysql-connector-java.jar
ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar

13、启动Ambari（192.168.72.17上执行）
ambari-server start

14、登录Ambari 
http://192.168.72.XX:8080/

FAQ：
出现这种情况属于之前setup Ambari连接数据库出现的问题，需要在进行数据库初始化
Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.6.2.v20151217-774c696): org.eclipse.persistence.exceptions.DatabaseException Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'ambari.metainfo' doesn't exis
采用ambari登录数据库，执行
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql