ansible-playbook+shell实现自动化部署ambari（纯离线）
文章目录
- 一、业务场景
- 二、选用技术
- 三、准备
- 四、服务器准备
- 五、配置步骤
- 1、配置网卡（所有节点）
- 2、配置hosts映射（主节点）
- 3、配置主节点到各个机器的ssh（主节点）
- 4、安装ansible
- 5、配置ansible的hosts（ansible使用的ip和定义主机组的映射）
- 六、编写playbook
- 七、附件-脚本
- 1.closeTHP.sh
- 2.modifyLimits.sh
- 3.modifyNtpMaster.sh
- 4.modifyNtpSlave.sh
- 5.installJdk.sh
- 6. closeSelinux.sh
- 7.installMysql.sh
- 八、附件-配置文件
- 1. my.cnf
- 2. ambari.repo
- 3. HDP.repo
- 4. HDP-UTILS.repo
- 5. ambari.properties （注意改 node01 为 主节点 hostname）
- 九、遇到问题
- 1. 安装mysq修改密码问题
- 2.在openssl版本一致的情况python-2.7.5可能会出现证书验证失败
- 3.页面进行注册ambari-agent失败问题
一、业务场景
大规模集群（几十上百台服务器）搭建ambari，频繁更换登录服务器操作不便并且容易出现操作失误的情况，集群特点是环境基本一致，各节点的操作大部分相同，由此想到做一键部署脚本。
二、选用技术
- ansible：新出现的自动化运维工具，基于Python开发，集合了众多运维工具（puppet、cfengine、chef、func、fabric）的优点，实现了批量系统配置、批量程序部署、批量运行命令等功能。执行命令基于ssh
- shell ：安装各种依赖使用ansible调用shell脚本的方式
三、准备
这里我选用的纯离线安装方式，原因主要是受网络限制
需要下载的依赖包：
- ambari-2.5.0.3-centos7.tar.gz
- HDP-2.6.5.0-centos7-rpm.tar.gz
- HDP-UTILS-1.1.0.21-centos7.tar.gz
- ansible-2.4-rpms.el7.tar.gz（包含安装ansible各种依赖）
- mysql-5.7.27-el7-x86_64.tar.gz
- mysql-connector-java-5.1.47.jar
- jdk-8u201-linux-x64.tar.gz
（版本根据项目需求自己选择）
四、服务器准备
- 我这里用了10台centos7.2的没有装过任何软件的服务器
五、配置步骤
1、配置网卡（所有节点）
- “DEVICE”：网卡对应的设备别名，如ifcfg-eth0第一块网卡；
- “HWADDR”：网卡物理地址；
- “BOOTPROTO”：网卡获得ip地址的方式，Static（静态 ip地址）dhcp（通过dhcp协议获取ip） bootip通过bootp协议获得的ip地址；
- “IPADDR”： 网 卡 IP 地 址 ； “NETMASK”：网卡对应网络掩码；
- “NETWORK”：网卡对应的网络地址；
- “ONBOOT”：系统启动时是否设置此网络接口，设置为yes时，系统启动时激活此设备。默认设置 为yes；
- “DNS”：配置该网络环境下的DNS。
保存上述编辑内容，运行如下命令重启network服务。
service network restart
- 上传下载好的依赖包 我这边是放在/opt下了
[root@node01 opt]# ll
total ……
drwxr-xr-x. 2 root root 4096 May 20 17:37 playbook # 存放ansible执行playbook文件
drwxr-xr-x. 5 root root 4096 May 20 17:44 srcConfig # 存放实现准备好的配置文件（具体内容放在步骤里）
drwxr-xr-x. 6 root root 4096 May 20 11:57 srcPackage # 存放依赖包
drwxr-xr-x. 2 root root 4096 May 21 10:08 srcScript # 所有编写脚本文件夹（下面的所有脚本都放在这里）
2、配置hosts映射（主节点）
- 先配好hostname
- 配置/etc/hosts，编写以下脚本并执行
vim addHosts.sh
#! /bin/bash
echo "******** add host -> hosts ********"
# ip 和 hostname 应改为集群对应的
echo "192.168.111.201 node01" >> /etc/hosts
echo "192.168.111.202 node02" >> /etc/hosts
echo "192.168.111.203 node03" >> /etc/hosts
echo "192.168.111.204 node04" >> /etc/hosts
echo "192.168.111.205 node05" >> /etc/hosts
echo "192.168.111.206 node06" >> /etc/hosts
echo "192.168.111.207 node07" >> /etc/hosts
echo "192.168.111.208 node08" >> /etc/hosts
echo "192.168.111.209 node09" >> /etc/hosts
echo "192.168.111.210 node10" >> /etc/hosts
cp /etc/hosts /opt/srcConfig/system/
# 拷贝到配置文件源目录，这里只是方便管理统一的配置文件，非必要操作
echo "******** END ********"
3、配置主节点到各个机器的ssh（主节点）
- 编写ssh脚本
vim sshScript.sh
#! /bin/bash
ssh-keygen -t rsa
# 192.168.111 改为集群所在网段（和hosts中配置对应）
ALL_CLIENTS=`cat /etc/hosts| grep "192.168.111" | awk '{print $2}'`
for client in $ALL_CLIENTS
do
echo "=============copy-ssh-id $client============="
ssh-copy-id $client
done
- 此脚本执行的时候需要与人员交互（输入各节点服务器密码），机器比较多的时候可以考虑使用expect
4、安装ansible
- 依赖包位置
/opt/srcPackage/ansible/ansible-2.4-rpms.el7.tar.gz
- 编写安装ansible脚本
vim installAnsible.sh
tar -xzvf /opt/srcPackage/ansible/ansible-2.4-rpms.el7.tar.gz -C /opt/srcPackage/ansible/
cd /opt/srcPackage/ansible/ansible-2.4-rpms.el7
rpm -ivh PyYAML*rpm libyaml*rpm python-babel*rpm python-backports*rpm python-backports-ssl_match_hostname*rpm python-cffi*rpm python-enum34*rpm python-httplib2*rpm python-idna*rpm python-ipaddress*rpm python-jinja2*rpm python-markupsafe*rpm python-paramiko*rpm python-passlib*rpm python-ply*rpm python-pycparser*rpm python-setuptools*rpm python-six*rpm python2-cryptography*rpm python2-jmespath*rpm python2-pyasn1*rpm sshpass*rpm --nodeps --force
rpm -ivh ansible-2.4.2.0-2.el7.noarch.rpm
ansible --version
5、配置ansible的hosts（ansible使用的ip和定义主机组的映射）
- 编写配置脚本
vim addAnsibleHosts.sh
#! /bin/bash
echo "******** add ansible host -> ansible hosts ********"
# ip 和 hostname 应改为集群对应的
echo "[test]" >> /etc/ansible/hosts # 主机组（所有节点）名称
echo "192.168.111.201" >> /etc/ansible/hosts
echo "192.168.111.202" >> /etc/ansible/hosts
echo "192.168.111.203" >> /etc/ansible/hosts
echo "192.168.111.204" >> /etc/ansible/hosts
echo "192.168.111.205" >> /etc/ansible/hosts
echo "192.168.111.206" >> /etc/ansible/hosts
echo "192.168.111.207" >> /etc/ansible/hosts
echo "192.168.111.208" >> /etc/ansible/hosts
echo "192.168.111.209" >> /etc/ansible/hosts
echo "192.168.111.210" >> /etc/ansible/hosts
echo "[testmaster]" >> /etc/ansible/hosts # 主机组（主节点）名称
echo "192.168.111.201" >> /etc/ansible/hosts
echo "[testslave]" >> /etc/ansible/hosts # 主机组（从节点）名称
echo "192.168.111.202" >> /etc/ansible/hosts
echo "192.168.111.203" >> /etc/ansible/hosts
echo "192.168.111.204" >> /etc/ansible/hosts
echo "192.168.111.205" >> /etc/ansible/hosts
echo "192.168.111.206" >> /etc/ansible/hosts
echo "192.168.111.207" >> /etc/ansible/hosts
echo "192.168.111.208" >> /etc/ansible/hosts
echo "192.168.111.209" >> /etc/ansible/hosts
echo "192.168.111.210" >> /etc/ansible/hosts
echo "******** END ********"
六、编写playbook
- playbook支持yaml文件格式，语法也遵循yaml格式
vim installAmbari.yaml
---
- hosts: test # 所有机器
remote_user: root
tasks:
# 配置host文件
- name: copy hosts
copy: src=/opt/srcConfig/system/hosts dest=/etc/hosts
# 执行脚本统一存放路径,有的脚本需要在所有机器本地执行，在所有机器上创建一个脚本存放路径，方便管理
- name: mkdir shFile dir
shell: mkdir -p /opt/shFile
# 关闭大页
# 脚本内容见 下面 七 中的closeTHP.sh
- name: copy closeTHP file
copy: src=/opt/srcScript/closeTHP.sh dest=/opt/shFile/closeTHP.sh
# 执行脚本
- name: closeTHP.sh
shell: sh /opt/shFile/closeTHP.sh
# 修改linux限制
# 脚本内容见 下面 七 中的modifyLimits.sh
- name: copy modifyLimits file
copy: src=/opt/srcScript/modifyLimits.sh dest=/opt/shFile/modifyLimits.sh
# 执行脚本
- name: modifyLimits.sh
shell: sh /opt/shFile/modifyLimits.sh
- hosts: testmaster # 主节点
remote_user: root
# 配置ntp服务 安装http
tasks:
# ntp server配置
# 脚本内容见 下面 七 中的modifyNtpMaster.sh
- name: copy modifyNtpMaster file # 脚本需要修改 ip网段
copy: src=/opt/srcScript/modifyNtpMaster.sh dest=/opt/shFile/modifyNtpMaster.sh
# 执行脚本并安装http服务
- name: modifyNtpMaster.sh install http
shell: sh /opt/shFile/modifyNtpMaster.sh
- shell: yum install httpd -y
- shell: systemctl start httpd
- shell: systemctl enable httpd
- hosts: testslave # 从节点
remote_user: root
# 配置ntp服务
tasks:
# ntp agent配置
# 脚本内容见 下面 七 中的modifyNtpSlave.sh
- name: copy modifyNtpSlave file # 脚本中 变量masterIp需要修改 网段也需要修改
copy: src=/opt/srcScript/modifyNtpSlave.sh dest=/opt/shFile/modifyNtpSlave.sh
# 执行脚本
- name: modifyNtpSlave.sh
shell: sh /opt/shFile/modifyNtpSlave.sh
- hosts: test # 所有节点
remote_user: root
# 关闭防火墙 开启ntp服务
tasks:
- name: stop disable firewalld & start enable ntpd
shell: systemctl stop firewalld.service
- shell: systemctl disable firewalld.service
- shell: systemctl start ntpd
- shell: systemctl enable ntpd
# 安装 openssl jdk
- name: install openssl
shell: yum install openssl -y
# 修改路径 注意修改脚本中对应的路径
- name: copy jdk
copy: src=/opt/srcPackage/jdk dest=/opt
# 脚本内容见 下面 七 中的installJdk.sh
- name: copy installJdk.sh
copy: src=/opt/srcScript/installJdk.sh dest=/opt/shFile/installJdk.sh
- name: chmod installJdk.sh
shell: chmod u+x /opt/shFile/installJdk.sh
# 执行脚本 并刷新环境变量
- name: sh installJdk.sh
shell: sh /opt/shFile/installJdk.sh
register: jdk_status
# 返回安装jdk状态
- name: show
debug: var=jdk_status verbosity=0
# 关闭selinux
# 脚本内容见 下面 七 中的closeSelinux.sh
- name: copy closeSelinux.sh
copy: src=/opt/srcScript/closeSelinux.sh dest=/opt/shFile/closeSelinux.sh
- name: sh closeSelinux.sh
shell: sh /opt/shFile/closeSelinux.sh
- hosts: testmaster # 主节点
remote_user: root
tasks:
# 安装mysql
# 脚本内容见 下面 七 中的installMysql.sh 安装过程中遇到了比较恶心的问题有兴趣可以看一下 九 问题mysql部分
# 脚本中需要使用一个my.cnf 配置文件 内容见 八.1
- name: copy installMysql.sh
copy: src=/opt/srcScript/installMysql.sh dest=/opt/shFile/installMysql.sh
# 执行脚本
- name: sh installMysql.sh
shell: sh /opt/shFile/installMysql.sh
register: installMysql_status
# 返回安装mysql状态
- name: show
debug: var=installMysql_status verbosity=0
# 配置ambari HDP HDP-UTILS yum源
# 解压ambari tar 包
- name: tar -zxvf ambari-2.5.0.3-centos7.tar.gz
shell: tar -zxvf /opt/srcPackage/ambari/ambari-2.5.0.3-centos7.tar.gz -C /var/www/html/
# 使用提前准备好的配置文件 注意路径
# /opt/srcConfig/ambari/ambari.repo 是我提前写好 存放的路径
# 配置文件内容见 八 中 ambari.repo
- name: copy ambari.repo # 需要修改ip
copy: src=/opt/srcConfig/ambari/ambari.repo dest=/etc/yum.repos.d/ambari.repo
# 配置文件内容见 八 中 HDP.repo
- name: copy HDP.repo # 需要修改ip
copy: src=/opt/srcConfig/ambari/HDP.repo dest=/etc/yum.repos.d/HDP.repo
- name: tar -zxvf ambari-2.5.0.3-centos7.tar.gz
shell: tar -zxvf /opt/srcPackage/ambari/HDP-2.6.5.0-centos7-rpm.tar.gz -C /var/www/html/
# - name: copy HDP-UTILS-1.1.0.21-centos7.tar.gz
# copy: src=/opt/srcPackage/ambari/HDP-UTILS-1.1.0.21-centos7.tar.gz dest=/var/www/html/HDP-UTILS-1.1.0.21-centos7.tar.gz
# 配置文件内容见 八 中 HDP-UTILS.repo
- name: copy HDP-UTILS.repo # 需要修改ip
copy: src=/opt/srcConfig/ambari/HDP-UTILS.repo dest=/etc/yum.repos.d/HDP-UTILS.repo
- name: tar -zxvf HDP-UTILS-1.1.0.21-centos7.tar.gz
shell: mkdir -p /var/www/html/HDP-UTILS
- shell: tar -zxvf /opt/srcPackage/ambari/HDP-UTILS-1.1.0.21-centos7.tar.gz -C /var/www/html/HDP-UTILS
# 配置 mysql-connector
- name: cp mysql-connector-java-5.1.47.jar
copy: src=/opt/srcPackage/mysql/mysql-connector-java-5.1.47.jar dest=/usr/share/java/mysql-connector-java.jar
# 安装 ambari-server
- name: install ambari-server
shell: yum install ambari-server -y
# 配置mysql 注意mysql密码为自己修改的
- name: Configure MySQL
shell: ln -s /usr/local/mysql/bin/mysql /usr/bin
- shell: mysql -e"use mysql;GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123456' WITH GRANT OPTION" -p123456 -h127.0.0.1
- shell: mysql -e"CREATE USER ambari IDENTIFIED BY '123456'" -p123456 -h127.0.0.1
- shell: mysql -e"CREATE DATABASE ambari" -p123456 -h127.0.0.1
# 执行ambari的sql脚本 路径默认是一样的
- shell: mysql -uroot -p123456 -h127.0.0.1 ambari < /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
# 赋权
- shell: mysql -e"grant ALL on ambari.* to ambari;flush privileges" -uroot -p123456 -h127.0.0.1
# 这一部分是我第一次 ambari-server setup的时候使用expect 进行配置的
# 但是由于每次setup步骤不是固定的，老是出现问题 我这里就换成使用配置文件的方式了
# install expect
# - name: install expect
# shell: yum install expect -y
# ambari-server setup 启动ambari-server
# - name: copy ambari-server setup Script
# copy: src=/opt/srcScript/ambariSetup-expect.sh dest=/opt/shFile/ambariSetup-expect.sh
# - name: ambari-server setup
# shell: expect /opt/shFile/ambariSetup-expect.sh
# - shell: ambari-server start
# 配置文件内容见 八 中的 ambari.properties 注意该文件的权限为 -rwxr-xr-x.
# password.dat 文件内容为 mysql密码 权限为 -rw-r-----
- name: ambari-server setup
shell: cat /opt/srcConfig/ambari/ambari.properties > /etc/ambari-server/conf/ambari.properties
- shell: cp /opt/srcConfig/ambari/password.dat /etc/ambari-server/conf/
- name: start ambari-server
shell: ambari-server start
register: ambari_server_status
# 返回ambari-server启动状态
- name: show
debug: var=ambari_server_status verbosity=0
- hosts: testslave # 从节点
remote_user: root
tasks:
- name: copy ambari.repo
copy: src=/opt/srcConfig/ambari/ambari.repo dest=/etc/yum.repos.d/ambari.repo
- name: copy HDP.repo
copy: src=/opt/srcConfig/ambari/HDP.repo dest=/etc/yum.repos.d/HDP.repo
- name: copy HDP-UTILS.repo
copy: src=/opt/srcConfig/ambari/HDP-UTILS.repo dest=/etc/yum.repos.d/HDP-UTILS.repo
# - name: install ambari-agent
# shell: yum install ambari-agent -y
- hosts:test # 所有节点
remote_user: root
tasks:
- name: install ambari-agent
shell: yum install ambari-agent -y
- name: ambari-agent.ini
shell: sed -i 's/localhost/node01/g' /etc/ambari-agent/conf/ambari-agent.ini # 应把 node01 改为 ambari-server 节点主机名
- shell: sed -i 's/verify=platform_default/verify=disable/' /etc/python/cert-verification.cfg
- shell: sed -i "53 iforce_https_protocol=PROTOCOL_TLSv1_2" /etc/ambari-agent/conf/ambari-agent.ini
- name: start ambari-agent
shell: /etc/init.d/ambari-agent start
register: ambari_agent_status
#返回ambari-agent启动状态
- name: show
debug: var=ambari_agent_status verbosity=0
七、附件-脚本
1.closeTHP.sh
#! /bin/bash
echo "******** CLOSE THP *********"
echo "if test -f /sys/kernel/mm/transparent_hugepage/enabled; then" >> /etc/rc.d/rc.local
echo "echo never > /sys/kernel/mm/transparent_hugepage/enabled" >> /etc/rc.d/rc.local
echo "fi" >> /etc/rc.d/rc.local
echo "if test -f /sys/kernel/mm/transparent_hugepage/defrag; then" >> /etc/rc.d/rc.local
echo "echo never > /sys/kernel/mm/transparent_hugepage/defrag" >> /etc/rc.d/rc.local
echo "fi" >> /etc/rc.d/rc.local
chmod +x /etc/rc.d/rc.local
echo "******** END ********"
2.modifyLimits.sh
#! /bin/bash
echo "******** Modify system limits ********"
echo "* soft nofile 1024000" >> /etc/security/limits.conf
echo "* hard nofile 1024000" >> /etc/security/limits.conf
echo "hive - nofile 1024000" >> /etc/security/limits.conf
echo "hive - nproc 1024000" >> /etc/security/limits.conf
echo "******** END ********"
3.modifyNtpMaster.sh
#! /bin/bash
#文件具体怎么修改可参考 我前面安装ntp负的文章
echo "******** MODIFY NTP MASTER CONFIG ********"
localIp=`ifconfig -a|grep inet|grep -v 127.0.0.1|grep -v inet6|awk '{print $2}'|tr -d "addr:"|grep "192.168.111"`
file=/etc/ntp.conf
# 行数需按具体情况修改
sed -i "21,24s/^/#/" $file
sed '13 irestrict '$localIp' nomodify notrap nopeer noquery' -i $file
sed '20 irestrict 192.168.111.1 mask 255.255.255.0 nomodify notrap' -i $file
sed '27 iserver 127.127.1.0' -i $file
sed '28 iFudge 127.127.1.0 stratum 10' -i $file
echo "******** END ********"
4.modifyNtpSlave.sh
#! /bin/bash
echo "******** MODIFY NTP MASTER CONFIG ********"
# ip需改为 ntpserver节点ip
masterIp="10.20.10.129"
# 10.20.10 需改为集群节点所在网段
localIp=`ifconfig -a|grep inet|grep -v 127.0.0.1|grep -v inet6|awk '{print $2}'|tr -d "addr:"|grep "10.20.10"`
file=/etc/ntp.conf
sed -i "21,24s/^/#/" $file
sed '13 irestrict '$localIp' nomodify notrap nopeer noquery' -i $file
# 10.20.10.1 需改为集群网关
sed '20 irestrict 10.20.10.1 mask 255.255.255.0 nomodify notrap' -i $file
sed '27 iserver '$masterIp -i $file
sed '28 iFudge '$masterIp' stratum 10' -i $file
echo "******** END ********"
5.installJdk.sh
#! /bin/bash
echo "******** JDK ********"
tar -xvf /opt/jdk/jdk-8u201-linux-x64.tar.gz -C /opt/jdk
# 我这里是没有配置过环境变量的新机器，所以可以直接在/etc/profile 后追加
# 如果已经有环境变量可根据具体情况使用sed -i等命令配置
echo "export JAVA_HOME=/opt/jdk/jdk1.8.0_201" >> /etc/profile
echo "export PATH=\$PATH:\$JAVA_HOME/bin" >> /etc/profile
echo "export CLASSPATH=.:\$JAVA_HOME/lib/dt.jar:\$JAVA_HOME/lib/tools.jar" >> /etc/profile
rm -rf /usr/bin/java
rm -rf /usr/bin/javac
#hostname=`"hostname"`
#ssh $hostname "source /etc/profile;java -version;exit"
source /etc/profile
echo "******* JDK installation completed ********"
6. closeSelinux.sh
#! /bin/bash
echo "******** CLOSE SELINUX *********"
setenforce 0
file=/etc/selinux/config
sed -i "7s/^/#/" $file
sed '7 iSELINUX=disabled' -i $file
echo "******** END ********"
7.installMysql.sh
#! /bin/bash
echo "******** INSTALL MYSQL *********"
#卸载原有的mariadb
OLD_MYSQL=`rpm -qa|grep mariadb`
profile=/etc/profile
for mariadb in $OLD_MYSQL
do
rpm -e --nodeps $mariadb
done
#删除原有的my.cnf
rm -rf /etc/my.cnf
#添加用户组 用户
groupadd mysql
useradd -g mysql mysql
#解压mysql包并修改名称
tar -zxvf /opt/srcPackage/mysql/mysql-5.7.27-el7-x86_64.tar.gz -C /usr/local
mv /usr/local/mysql-5.7.27-el7-x86_64 /usr/local/mysql
#更改所属的组和用户
chown -R mysql /usr/local/mysql
chgrp -R mysql /usr/local/mysql
mkdir -p /usr/local/mysql/data
chown -R mysql:mysql /usr/local/mysql/data
#粘贴配置文件my.cnf 内容见八 中的 my.cnf
cp /opt/srcConfig/mysql/my.cnf /usr/local/mysql/
# 安装mysql
/usr/local/mysql/bin/mysql_install_db --user=mysql --basedir=/usr/local/mysql/ --datadir=/usr/local/mysql/data/
#设置文件及目录权限：
cp /usr/local/mysql/support-files/mysql.server /etc/init.d/mysqld
chown 777 /usr/local/mysql/my.cnf
chmod +x /etc/init.d/mysqld
mkdir /var/lib/mysql
chmod 777 /var/lib/mysql
#启动mysql
/etc/init.d/mysqld start
#设置开机启动
chkconfig --level 35 mysqld on
chmod +x /etc/rc.d/init.d/mysqld
chkconfig --add mysqld
#修改环境变量
#行数需根据实际情况修改
sed '78s/$/&:\/usr\/local\/mysql\/bin/' -i $profile
mysqlPw=`sed -n 2p /root/.mysql_secret`
mysqlPwTMP=`sed -n 2p /root/.mysql_secret`1
mysqlNewPw=123456
hostname=`"hostname"`
#ssh $hostname "source /etc/profile;java -version"
ssh $hostname "source /etc/profile;mysqladmin -h127.0.0.1 -uroot -p'$mysqlPw' password '$mysqlPwTMP';mysqladmin -h127.0.0.1 -uroot -p'$mysqlPwTMP' password '$mysqlNewPw';exit"
echo "******** MYSQL installation completed ********"
八、附件-配置文件
1. my.cnf
[mysql]
socket=/var/lib/mysql/mysql.sock
# set mysql client default chararter
default-character-set=utf8
[mysqld]
socket=/var/lib/mysql/mysql.sock
# set mysql server port
port = 3306
# set mysql install base dir
basedir=/usr/local/mysql
# set the data store dir
datadir=/usr/local/mysql/data
# set the number of allow max connnection
max_connections=200
# set server charactre default encoding
character-set-server=utf8
# the storage engine
default-storage-engine=INNODB
lower_case_table_names=1
max_allowed_packet=16M
explicit_defaults_for_timestamp=true
[mysql.server]
user=mysql
basedir=/usr/local/mysql
2. ambari.repo
[ambari-2.5.0.3]
# 注意把 192.168.111.201 为 自己集群 主节点的ip
name=ambari Version - ambari-2.5.0.3
#baseurl=http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.5.0.3
baseurl=http://192.168.111.201/ambari/centos7/
gpgcheck=1
#gpgkey=http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.5.0.3/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
gpgkey=http://192.168.111.201/ambari/centos7/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
3. HDP.repo
[HDP-2.6]
# 注意把 192.168.111.201 为 自己集群 主节点的ip
name=HDP-2.6
baseurl=http://192.168.111.201/HDP/centos7/2.6.5.0-292
path=/
enabled=1
gpgcheck=0
4. HDP-UTILS.repo
[HDP-UTILS-1.1.0.21]
# 注意把 192.168.111.201 为 自己集群 主节点的ip
name=HDP-UTILS-1.1.0.21
baseurl=http://192.168.111.201/HDP-UTILS/
path=/
enabled=1
gpgcheck=0
5. ambari.properties （注意改 node01 为 主节点 hostname）
#
# Copyright 2011 The Apache Software Foundation
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#Wed May 20 15:47:56 CST 2020
ulimit.open.files=65536
views.http.x-content-type-options=nosniff
server.persistence.type=remote
jdk1.8.jcpol-url=http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-8.zip
java.releases.ppc64le=
recommendations.artifacts.lifetime=1w
http.pragma=no-cache
jdk1.8.url=http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-8u112-linux-x64.tar.gz
server.execution.scheduler.misfire.toleration.minutes=480
java.home=/opt/jdk/jdk1.8.0_201
security.server.disabled.ciphers=TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384|TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384|TLS_RSA_WITH_AES_256_CBC_SHA256|TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384|TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384|TLS_DHE_RSA_WITH_AES_256_CBC_SHA256|TLS_DHE_DSS_WITH_AES_256_CBC_SHA256|TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA|TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA|TLS_RSA_WITH_AES_256_CBC_SHA|TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA|TLS_ECDH_RSA_WITH_AES_256_CBC_SHA|TLS_DHE_RSA_WITH_AES_256_CBC_SHA|TLS_DHE_DSS_WITH_AES_256_CBC_SHA|TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256|TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256|TLS_RSA_WITH_AES_128_CBC_SHA256|TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256|TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256|TLS_DHE_RSA_WITH_AES_128_CBC_SHA256|TLS_DHE_DSS_WITH_AES_128_CBC_SHA256|TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA|TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA|TLS_RSA_WITH_AES_128_CBC_SHA|TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA|TLS_ECDH_RSA_WITH_AES_128_CBC_SHA|TLS_DHE_RSA_WITH_AES_128_CBC_SHA|TLS_DHE_DSS_WITH_AES_128_CBC_SHA|TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA|TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA|TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA|TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA|SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA|SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA|TLS_EMPTY_RENEGOTIATION_INFO_SCSV|TLS_DH_anon_WITH_AES_256_CBC_SHA256|TLS_ECDH_anon_WITH_AES_256_CBC_SHA|TLS_DH_anon_WITH_AES_256_CBC_SHA|TLS_DH_anon_WITH_AES_128_CBC_SHA256|TLS_ECDH_anon_WITH_AES_128_CBC_SHA|TLS_DH_anon_WITH_AES_128_CBC_SHA|TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA|SSL_DH_anon_WITH_3DES_EDE_CBC_SHA|SSL_RSA_WITH_DES_CBC_SHA|SSL_DHE_RSA_WITH_DES_CBC_SHA|SSL_DHE_DSS_WITH_DES_CBC_SHA|SSL_DH_anon_WITH_DES_CBC_SHA|SSL_RSA_EXPORT_WITH_DES40_CBC_SHA|SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA|SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA|SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA|TLS_RSA_WITH_NULL_SHA256|TLS_ECDHE_ECDSA_WITH_NULL_SHA|TLS_ECDHE_RSA_WITH_NULL_SHA|SSL_RSA_WITH_NULL_SHA|TLS_ECDH_ECDSA_WITH_NULL_SHA|TLS_ECDH_RSA_WITH_NULL_SHA|TLS_ECDH_anon_WITH_NULL_SHA|SSL_RSA_WITH_NULL_MD5|TLS_KRB5_WITH_3DES_EDE_CBC_SHA|TLS_KRB5_WITH_3DES_EDE_CBC_MD5|TLS_KRB5_WITH_DES_CBC_SHA|TLS_KRB5_WITH_DES_CBC_MD5|TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA|TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5
server.jdbc.hostname=node01
shared.resources.dir=/usr/lib/ambari-server/lib/ambari_commons/resources
server.jdbc.connection-pool.max-idle-time=14400
resources.dir=/var/lib/ambari-server/resources
custom.action.definitions=/var/lib/ambari-server/resources/custom_action_definitions
views.request.connect.timeout.millis=5000
jdk1.7.desc=Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
server.jdbc.driver=com.mysql.jdbc.Driver
security.server.keys_dir=/var/lib/ambari-server/keys
server.jdbc.rca.user.name=ambari
webapp.dir=/usr/lib/ambari-server/web
views.http.pragma=no-cache
server.os_family=redhat7
server.jdbc.user.passwd=/etc/ambari-server/conf/password.dat
server.execution.scheduler.isClustered=false
views.ambari.request.connect.timeout.millis=30000
server.jdbc.database=mysql
server.jdbc.connection-pool=c3p0
server.jdbc.database_name=ambari
server.jdbc.rca.url=jdbc:mysql://node01:3306/ambari
bootstrap.script=/usr/lib/python2.6/site-packages/ambari_server/bootstrap.py
server.version.file=/var/lib/ambari-server/resources/version
jdk1.8.dest-file=jdk-8u112-linux-x64.tar.gz
server.task.timeout=1200
user.inactivity.timeout.role.readonly.default=0
server.jdbc.connection-pool.max-age=0
java.releases=jdk1.8,jdk1.7
recommendations.dir=/var/run/ambari-server/stack-recommendations
agent.stack.retry.tries=5
server.os_type=centos7
server.python.log.level=INFO
server.execution.scheduler.maxDbConnections=5
views.ambari.request.read.timeout.millis=45000
views.http.cache-control=no-store
rolling.upgrade.skip.packages.prefixes=
jdk1.8.home=/usr/jdk64/
jdk1.7.home=/usr/jdk64/
agent.task.timeout=900
bootstrap.setup_agent.script=/usr/lib/python2.6/site-packages/ambari_server/setupAgent.py
server.jdbc.rca.driver=com.mysql.jdbc.Driver
jdk1.7.dest-file=jdk-7u67-linux-x64.tar.gz
agent.package.install.task.timeout=1800
server.jdbc.port=3306
http.strict-transport-security=max-age=31536000
common.services.path=/var/lib/ambari-server/resources/common-services
agent.threadpool.size.max=25
ambari.python.wrap=ambari-python-wrap
skip.service.checks=false
server.jdbc.connection-pool.idle-test-interval=7200
ambari-server.user=root
jdk1.8.desc=Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
views.http.strict-transport-security=max-age=31536000
http.x-content-type-options=nosniff
jdk1.7.re=(jdk.*)/jre
metadata.path=/var/lib/ambari-server/resources/stacks
jdk1.8.jcpol-file=jce_policy-8.zip
views.skip.home-directory-check.file-system.list=wasb,adls,adl
server.python.log.name=ambari-server-command.log
stackadvisor.script=/var/lib/ambari-server/resources/scripts/stack_advisor.py
http.x-xss-protection=1; mode=block
bootstrap.dir=/var/run/ambari-server/bootstrap
server.connection.max.idle.millis=900000
server.jdbc.rca.user.passwd=/etc/ambari-server/conf/password.dat
views.http.x-frame-options=SAMEORIGIN
server.jdbc.connection-pool.acquisition-size=5
http.x-frame-options=DENY
jce.download.supported=true
jdk.download.supported=true
jdk1.7.url=http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-7u67-linux-x64.tar.gz
server.jdbc.user.name=ambari
mpacks.staging.path=/var/lib/ambari-server/resources/mpacks
pid.dir=/var/run/ambari-server
user.inactivity.timeout.default=0
agent.stack.retry.on_repo_unavailability=false
extensions.path=/var/lib/ambari-server/resources/extensions
views.request.read.timeout.millis=10000
jdk1.7.jcpol-file=UnlimitedJCEPolicyJDK7.zip
server.tmp.dir=/var/lib/ambari-server/data/tmp
server.execution.scheduler.maxThreads=5
server.jdbc.url=jdbc:mysql://node01:3306/ambari
server.fqdn.service.url=http://169.254.169.254/latest/meta-data/public-hostname
views.http.x-xss-protection=1; mode=block
server.http.session.inactive_timeout=1800
server.stages.parallel=true
kerberos.keytab.cache.dir=/var/lib/ambari-server/data/cache
jdk1.8.re=(jdk.*)/jre
http.cache-control=no-store
client.threadpool.size.max=25
jdk1.7.jcpol-url=http://public-repo-1.hortonworks.com/ARTIFACTS/UnlimitedJCEPolicyJDK7.zip
server.jdbc.connection-pool.max-idle-time-excess=0
九、遇到问题
1. 安装mysq修改密码问题
报错信息
Access denied for user 'root'@'localhost' (using password: YES)
问题原因应该是密码设置策略问题，因为我之间使用临时密码，设置新密码为很简单的密码123456才会报错
设置新密码临时密码后面随便加个字母，就可以成功，然后再重新设置为简单密码，这样就可以了
脚本如下
mysqlPw=`sed -n 2p /root/.mysql_secret`
mysqlPwTMP=`sed -n 2p /root/.mysql_secret`1
mysqlNewPw=123456
hostname=`"hostname"`
#ssh $hostname "source /etc/profile;java -version"
ssh $hostname "source /etc/profile;mysqladmin -h127.0.0.1 -uroot -p'$mysqlPw' password '$mysqlPwTMP';mysqladmin -h127.0.0.1 -uroot -p'$mysqlPwTMP' password '$mysqlNewPw';exit"
使用ssh修改的原因是，在当前shell中，前面刚刚source环境变量之后，当前shell拿不到最新的环境变量也就是会报错
mysqladmin 命令找不到的错误
2.在openssl版本一致的情况python-2.7.5可能会出现证书验证失败
解决办法
sed -i 's/verify=platform_default/verify=disable/' /etc/python/cert-verification.cfg
3.页面进行注册ambari-agent失败问题
解决办法
sed -i "53 iforce_https_protocol=PROTOCOL_TLSv1_2" /etc/ambari-agent/conf/ambari-agent.ini