Centos7 Ambari集群环境搭建
Ambari集群环境搭建
1.1服务规划
序号 主机名 操作系统 Ip地址 服务
1 hbase100.yun Cent OS_7.6 172.21.xx.100 HTTP Server、amber-server、amber-agent
2 hbase101.yun Cent OS_7.6 172.21.xx.101 amber-agent
2 hbase102.yun Cent OS_7.6 172.21.xx.102 amber-agent
*****1.2用户规划*****
序号 用户名 所属群组 备注
1 root root
2 安装准备
2.0基本软件需求
Software requirement
yum,scp,curl,unzip,tar,wget,gcc*,OpenSSL(v1.01,build 16 or later),Python 2.7.12 (with python-devel*)
*2.1* /etc/hosts
Configuration File Location Value Description
/etc/hosts 172.21.xx.100 hbase100.yun
172.21.xx.101 hbase101.yun
172.21.xx.102 hbase102.yun
2.2 关闭防火墙以及SELinux
2.2.1 关闭
序号 Where Host comm
1 All host All systemctl disable firewalld
service firewalld stop
2.2.2 关闭SELinux
序号 Where Host comm
1 All host All setenforce 0
2.3 系统特性调整
2.3.1 关闭PackageKit(if have)
vi /etc/yum/pluginconf.d/refresh-packagekit.conf
enabled=0
2.3.2 关闭THP (hbase tune)
1 All host All
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
2.3.3 关闭NUMA回收 (hbase tune)
1 All host All
echo "0" > /proc/sys/vm/zone_reclaim_mode
2.3.4 关闭swappiness (hbase tune)
1 All host All echo "0">/proc/sys/vm/swappiness
echo "vm.swappiness=0" >> /etc/sysctl.conf
2.3.5 调整vm.min_free_kbytes(1G) (hbase tune)
1 All host All echo "1048576" > /proc/sys/vm/min_free_kbytes
3.0 HTTP 安装
序号 Where to install Install command
1 HTTP Server yum install httpd
2 HTTP Server sudo service httpd start
sudo chkconfig httpd
cd /etc
3.1 Yum_HDP
3.1.1 Server 资源配置
序号 Where to install Install command
1 HTTP Server tar -zxvf ambari-2.7.4.0-centos7.tar.gz
tar -zxvf HDP-3.1.0.0-centos7-rpm.tar.gz
tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz
2 HTTP Server sudo mkdir -p /var/www/html/hop
cd /var/www/html
mkdir -p Ambari-2.7.4.0/centos7/
mv amber/cnetos7/2.7.4.0-118 /var/www/html/Ambari-2.7.4.0/centos7/
mkdir -p hop/HDP/centos7/3.x/updates/
mv HDP/centos7/3.1.0.0-78 /var/www/html/hop/HDP/centos7/3.x/updates/
mkdir -p hop/HDP-UTILS-/repos/centos7
mv HDP-UTILS/centos7/1.1.0.22 /var/www/htmlhdp/HDP-UTILS-/repos/centos7
3.1.2 Server 配置
Configuration File Location Value Description
/etc/httpd/cone/httpd.conf <Files ~ "^\.ht">
Order allow, deny
Allow from all
Satisfy All
</Files>
3.1.3 Client 配置
/etc/yum.repos.d/ambari.repo Copy HTTP对应源中的repo
修改baseurl 和 gpgkey地址
/etc/yum.repos.d/HDP-3.1.0.0.repo Copy 源中的repo
修改baseurl 和 gpgkey地址
/etc/yum.repos.d/HDP-UTILS-1.1.0.20.repo Copy 源中的repo
修改baseurl 和 gpgkey地址
3.1.4 生效
1. all host
sudo yum clean all
2 NTP服务
all hosts
yum install ntpd
systemctl enable ntpd
3 文件句柄
All ulimit -n 65536
4.0 JDK安装
序号 Where to install Install command 备注
4.1 rpm -qa|grep jdk
rpm -e --nodeps java -- or yum remove *openjdk* 卸载自带jdk
4.2 cd /usr/java/
Tar -zxvf jdk-8-linux-x64.tar.gz
4.3 all host
unzip -o -j -q jce_policy-8.zip -d /usr/java/jdk1.8/jre/lib/security
4.4 all host
sudo vi /etc/profile
export JAVA_HOME=/usr/java/jdk1.8
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib
4.4 all host
source /etc/profile
4.6 java -version
5.0 server和agent之间ssh免密码登录
5.1 all hosts
ssh-keygen -t rsa
ssh-copy-id master.machine
6.0 安装amber
6.1 amber-server sudo yum install amber-server
6.2 amber-server sudo amber-server setup
6.3 amber-server Java路径手动配置，其他默认
6.4 运行amber
6.4.1
amber-serve sudo amber-server start
管理页面地址：http://<ambari-server>:8080
初始帐号密码：admin/admin
7.0资源下载
http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.0/HDP-3.1.4.0-centos7-rpm.tar.gz
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.4.0/ambari-2.7.4.0-centos7.tar.gz
https://download.oracle.com/otn-pub/java/jce/8/jce_policy-8.zip?AuthParam=1590129241_dd28336bc76071c7fc29a3a0b646768b
安装参考
https://docs.cloudera.com/HDPDocuments/Ambari-2.7.4.0/bk_ambari-installation/content/ch_Getting_Ready.html