使用ansible部署CDH 5.15.1大数据集群

　　　　　　　　　　　　　　　　使用ansible离线部署CDH 5.15.1大数据集群

　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　作者：尹正杰 

版权声明：原创作品，谢绝转载！否则将追究法律责任。

 

　　在此之前，我之前分享过使用shell自定义脚本部署大数据集群，不管是部署CDH集群还是部署HDP集群基本上套路都一样，调优的方式也都大同小异。只是操作界面上可能存在一定的差异。最近赶上公司搞自动化运维项目，我们用到了ansible的一些功能，本篇博客主要记录ansible部署CDH服务。

 

 

一.安装ansible

1>.操作环境介绍

　　注意，请检查你的虚拟机是否可以正常联网，如果不能联网的话，本篇博客估计也就只能成为你的一个参考部署方案了！由于Mac内存有限，我只开了3台虚拟机，使用的操作系统是CentOS 7.2发行版的，内存分配每个节点4G，大家根据自己的机器适当调大或调小内存。详细信息如下：

[root@node101 ~]# hostname
node101.yinzhengjie.org.cn　　　　　　　　　　　　#这是本篇博客所说的主节点，该节点配置必须高于其他节点配置或者配置不能低于2G内存！否则你在后期安装服务时，可能会遇到各种坑！（我测试时，三台主机分配过4，1，1，或者3，3，3基本上都可以正常运行！）
[root@node101 ~]# 
[root@node101 ~]# hostname -i
172.30.1.101
[root@node101 ~]# 
[root@node101 ~]# cat /etc/redhat-release          
CentOS Linux release 7.2.1511 (Core) 
[root@node101 ~]# 
[root@node101 ~]# 
[root@node101 ~]# free  -h   
              total        used        free      shared  buff/cache   available
Mem:           5.9G        120M        5.5G        6.7M        213M        713M
Swap:          2.0G          0B        2.0G
[root@node101 ~]# 
[root@node101 ~]# df -h      
Filesystem                    Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root   50G  3.8G   43G   9% /
devtmpfs                      480M     0  480M   0% /dev
tmpfs                         495M     0  495M   0% /dev/shm
tmpfs                         495M  6.7M  488M   2% /run
tmpfs                         495M     0  495M   0% /sys/fs/cgroup
/dev/sda1                     477M  133M  315M  30% /boot
/dev/mapper/VolGroup-lv_home   12G   41M   11G   1% /home
Home                          234G  156G   79G  67% /media/psf/Home
tmpfs                          99M     0   99M   0% /run/user/0
[root@node101 ~]# 
[root@node101 ~]# cat /etc/hosts | grep yinzhengjie
172.30.1.101 node101.yinzhengjie.org.cn
172.30.1.102 node102.yinzhengjie.org.cn
172.30.1.103 node103.yinzhengjie.org.cn
[root@node101 ~]# 
[root@node101 ~]# 

2>.在主节点（node101.yinzhengjie.org.cn）配置至各个节点的免密钥登录

[root@node101 ~]# ssh-keygen  -t rsa -P '' -f ~/.ssh/id_rsa
Generating public/private rsa key pair.
Created directory '/root/.ssh'.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
7b:9a:3a:cd:0b:7b:b5:5e:da:ad:b8:dd:aa:0a:b1:fb [email protected]
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|                 |
|                 |
|                 |
|      . S        |
|       o ..      |
|      +o.....    |
|      .=++.* o   |
|      +=E+*o=oo  |
+-----------------+
[root@node101 ~]# 

生成密钥（[root@node101 ~]# ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa）

[root@node101 ~]# ssh-copy-id [email protected]
The authenticity of host 'node101.yinzhengjie.org.cn (172.30.1.101)' can't be established.
ECDSA key fingerprint is 72:2c:15:15:05:ba:4a:73:6d:5b:88:c4:bd:ab:c2:66.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
[email protected]'s password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added.

[root@node101 ~]# 
[root@node101 ~]# who
root     pts/0        2019-01-10 16:19 (172.30.1.2)
[root@node101 ~]# 
[root@node101 ~]# ssh node101.yinzhengjie.org.cn
Last login: Thu Jan 10 16:19:18 2019 from 172.30.1.2
[root@node101 ~]# 
[root@node101 ~]# who
root     pts/0        2019-01-10 16:19 (172.30.1.2)
root     pts/1        2019-01-10 16:30 (node101.yinzhengjie.org.cn)
[root@node101 ~]# 
[root@node101 ~]# exit
logout
Connection to node101.yinzhengjie.org.cn closed.
[root@node101 ~]# 
[root@node101 ~]# who
root     pts/0        2019-01-10 16:19 (172.30.1.2)
[root@node101 ~]# 
[root@node101 ~]# 

与本机配置免密登录（[root@node101 ~]# ssh-copy-id [email protected]）

[root@node101 ~]# ssh-copy-id [email protected]
The authenticity of host 'node102.yinzhengjie.org.cn (172.30.1.102)' can't be established.
ECDSA key fingerprint is 85:f4:84:e1:7a:13:45:31:22:7b:12:49:5d:1b:56:ec.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
[email protected]'s password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added.

[root@node101 ~]# 
[root@node101 ~]# ssh node102.yinzhengjie.org.cn
Last login: Thu Jan 10 15:46:07 2019 from 172.30.1.2
[root@node102 ~]# 
[root@node102 ~]# who
root     pts/0        2019-01-10 15:46 (172.30.1.2)
root     pts/1        2019-01-10 16:31 (172.30.1.101)
[root@node102 ~]# 
[root@node102 ~]# exit
logout
Connection to node102.yinzhengjie.org.cn closed.
[root@node101 ~]# 
[root@node101 ~]# 

与node102.yinzhengjie.org.cn节点配置免密登录（[root@node101 ~]# ssh-copy-id [email protected]）

[root@node101 ~]# ssh-copy-id [email protected]
The authenticity of host 'node103.yinzhengjie.org.cn (172.30.1.103)' can't be established.
ECDSA key fingerprint is 62:e7:04:30:ce:10:0d:55:9d:e5:24:9f:19:ab:4c:e7.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
[email protected]'s password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added.

[root@node101 ~]# 
[root@node101 ~]# ssh node103.yinzhengjie.org.cn             
Last login: Thu Jan 10 15:46:08 2019 from 172.30.1.2
[root@node103 ~]# 
[root@node103 ~]# who
root     pts/0        2019-01-10 15:46 (172.30.1.2)
root     pts/1        2019-01-10 16:32 (172.30.1.101)
[root@node103 ~]# 
[root@node103 ~]# exit 
logout
Connection to node103.yinzhengjie.org.cn closed.
[root@node101 ~]# 
[root@node101 ~]# 

与node103.yinzhengjie.org.cn节点配置免密登录（[root@node101 ~]# ssh-copy-id [email protected]）

3>.使用yum工具包安装ansible

　　安装ansible依赖python2-pyasn1，PyYAML，python-jinja2等模块，使用yum方式安装，他会自动帮我们处理依赖关系，我们配置好yum源即可。

[root@node101 ~]# yum -y install wget
Loaded plugins: fastestmirror
base                                                                                                                                        | 3.6 kB  00:00:00     
extras                                                                                                                                      | 3.4 kB  00:00:00     
updates                                                                                                                                     | 3.4 kB  00:00:00     
updates/7/x86_64/primary_db    FAILED                                          
http://mirrors.cqu.edu.cn/CentOS/7.6.1810/updates/x86_64/repodata/1c8b4fadfe3ee3539b33616064daaae4a96b2d659235607042e44c7cc389e0dc-primary.sqlite.bz2: [Errno 14] curl#56 - "Recv failure: Connection reset by peer"
Trying other mirror.
(1/4): extras/7/x86_64/primary_db                                                                                                           | 156 kB  00:00:00     
(2/4): base/7/x86_64/group_gz                                                                                                               | 166 kB  00:00:02     
(3/4): updates/7/x86_64/primary_db                                                                                                          | 2.4 MB  00:00:03     
(4/4): base/7/x86_64/primary_db                                                                                                             | 6.0 MB  00:00:08     
Determining fastest mirrors
 * base: mirrors.tuna.tsinghua.edu.cn
 * extras: mirrors.aliyun.com
 * updates: mirrors.tuna.tsinghua.edu.cn
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 0:1.14-18.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

===================================================================================================================================================================
 Package                             Arch                                  Version                                       Repository                           Size
===================================================================================================================================================================
Installing:
 wget                                x86_64                                1.14-18.el7                                   base                                547 k

Transaction Summary
===================================================================================================================================================================
Install  1 Package

Total download size: 547 k
Installed size: 2.0 M
Downloading packages:
warning: /var/cache/yum/x86_64/7/base/packages/wget-1.14-18.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
Public key for wget-1.14-18.el7.x86_64.rpm is not installed
wget-1.14-18.el7.x86_64.rpm                                                                                                                 | 547 kB  00:00:00     
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Importing GPG key 0xF4A80EB5:
 Userid     : "CentOS-7 Key (CentOS 7 Official Signing Key) <security@centos.org>"
 Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5
 Package    : centos-release-7-2.1511.el7.centos.2.10.x86_64 (@anaconda)
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : wget-1.14-18.el7.x86_64                                                                                                                         1/1 
  Verifying  : wget-1.14-18.el7.x86_64                                                                                                                         1/1 

Installed:
  wget.x86_64 0:1.14-18.el7                                                                                                                                        

Complete!
[root@node101 ~]# 

安装下载工具（[root@node101 ~]# yum -y install wget）

[root@node101 ~]# wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
--2019-01-10 17:14:45--  http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
Resolving dl.fedoraproject.org (dl.fedoraproject.org)... 209.132.181.23, 209.132.181.24
Connecting to dl.fedoraproject.org (dl.fedoraproject.org)|209.132.181.23|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 15080 (15K) [application/x-rpm]
Saving to: ‘epel-release-latest-7.noarch.rpm’

100%[================================================================================================================================>] 15,080      78.2KB/s   in 0.2s   

2019-01-10 17:14:46 (78.2 KB/s) - ‘epel-release-latest-7.noarch.rpm’ saved [15080/15080]

[root@node101 ~]# 

下载epel源（[root@node101 ~]# wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm）

[root@node101 ~]# ll
total 16
-rw-r--r--. 1 root root 15080 Jan 10 17:00 epel-release-latest-7.noarch.rpm
[root@node101 ~]# 
[root@node101 ~]# rpm -ivh epel-release-latest-7.noarch.rpm 
warning: epel-release-latest-7.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:epel-release-7-11                ################################# [100%]
[root@node101 ~]# 
[root@node101 ~]# ll /etc/yum.repos.d/ | grep epel
-rw-r--r--. 1 root root  951 Oct  3  2017 epel.repo
-rw-r--r--. 1 root root 1050 Oct  3  2017 epel-testing.repo
[root@node101 ~]# 

安装epel源（[root@node101 ~]# rpm -ivh epel-release-latest-7.noarch.rpm ）

[root@node101 ~]# cat /etc/yum.conf  | grep -v ^# |grep  -v ^$
[main]
cachedir=/var/cache/yum/$basearch/$releasever
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
exactarch=1
obsoletes=1
gpgcheck=1
plugins=1
installonly_limit=5
bugtracker_url=http://bugs.centos.org/set_project.php?project_id=23&ref=http://bugs.centos.org/bug_report_page.php?category=yum
distroverpkg=centos-release
[root@node101 ~]# 
[root@node101 ~]# sed -i 's#keepcache=0#keepcache=1#' /etc/yum.conf
[root@node101 ~]# 
[root@node101 ~]# cat /etc/yum.conf  | grep -v ^# |grep  -v ^$     
[main]
cachedir=/var/cache/yum/$basearch/$releasever
keepcache=1
debuglevel=2
logfile=/var/log/yum.log
exactarch=1
obsoletes=1
gpgcheck=1
plugins=1
installonly_limit=5
bugtracker_url=http://bugs.centos.org/set_project.php?project_id=23&ref=http://bugs.centos.org/bug_report_page.php?category=yum
distroverpkg=centos-release
[root@node101 ~]# 

修改yum的配置文件，使得下载rpm安装包后不删除，方便咱们以后自建内网的yum仓库（[root@node101 ~]# sed -i 's#keepcache=0#keepcache=1#' /etc/yum.conf）

[root@node101 ~]# yum  clean all
Loaded plugins: fastestmirror, langpacks
Cleaning repos: epel
Cleaning up everything
[root@node101 ~]# 

清空之前的缓存（[root@node101 ~]# yum clean all）

[root@node101 ~]# yum makecache
Loaded plugins: fastestmirror, langpacks
base                                                                                                                                               | 3.6 kB  00:00:00     
epel/x86_64/metalink                                                                                                                               | 8.1 kB  00:00:00     
epel                                                                                                                                               | 3.2 kB  00:00:00     
extras                                                                                                                                             | 3.4 kB  00:00:00     
updates                                                                                                                                            | 3.4 kB  00:00:00     
(1/18): epel/x86_64/group_gz                                                                                                                       |  88 kB  00:00:00     
(2/18): base/7/x86_64/other_db                                                                                                                     | 2.6 MB  00:00:06     
(3/18): base/7/x86_64/group_gz                                                                                                                     | 166 kB  00:00:08     
(4/18): epel/x86_64/prestodelta                                                                                                                    |  803 B  00:00:11     
(5/18): epel/x86_64/updateinfo                                                                                                                     | 948 kB  00:00:13     
(6/18): epel/x86_64/filelists                                                                                                                      |  10 MB  00:00:20     
(7/18): epel/x86_64/other                                                                                                                          | 2.3 MB  00:00:06     
(8/18): base/7/x86_64/filelists_db                                                                                                                 | 7.1 MB  00:00:27     
(9/18): extras/7/x86_64/primary_db                                                                                                                 | 156 kB  00:00:01     
(10/18): extras/7/x86_64/other_db                                                                                                                  | 107 kB  00:00:01     
(11/18): base/7/x86_64/primary_db                                                                                                                  | 6.0 MB  00:00:33     
(12/18): updates/7/x86_64/prestodelta                                                                                                              | 183 kB  00:00:03     
(13/18): extras/7/x86_64/filelists_db                                                                                                              | 189 kB  00:00:22     
(14/18): updates/7/x86_64/filelists_db                                                                                                             | 1.4 MB  00:00:14     
(15/18): updates/7/x86_64/primary_db                                                                                                               | 1.3 MB  00:00:14     
extras/7/x86_64/prestodelta    FAILED                                          ========================================================-] 200 kB/s |  36 MB  00:00:01 ETA 
http://mirrors.njupt.edu.cn/centos/7.6.1810/extras/x86_64/repodata/9718255a5fa57675311c5ac73719bee470f94b10d38b5aa988511b9ddd860025-prestodelta.xml.gz: [Errno 12] Timeout on http://10.10.254.10/cache/4/02/mirrors.njupt.edu.cn/4f9b5de52f938c8378fba16c668a0a55/9718255a5fa57675311c5ac73719bee470f94b10d38b5aa988511b9ddd860025-prestodelta.xml.gz: (28, 'Connection timed out after 30884 milliseconds')
Trying other mirror.
(16/18): extras/7/x86_64/prestodelta                                                                                                               |  36 kB  00:00:00     
(17/18): epel/x86_64/primary                                                                                                                       | 3.6 MB  00:00:38     
updates/7/x86_64/other_db      FAILED                                          
http://mirrors.njupt.edu.cn/centos/7.6.1810/updates/x86_64/repodata/1fab468ed066d5469823091845179165040deb4951923d854a025e3b5c903051-other.sqlite.bz2: [Errno 12] Timeout on http://10.10.254.10/cache/5/02/mirrors.njupt.edu.cn/08d3e890295133ac7140ae59fc9dee3f/1fab468ed066d5469823091845179165040deb4951923d854a025e3b5c903051-other.sqlite.bz2: (28, 'Connection timed out after 30866 milliseconds')
Trying other mirror.
(18/18): updates/7/x86_64/other_db                                                                                                                 | 191 kB  00:00:00     
Determining fastest mirrors
 * base: mirror.jdcloud.com
 * epel: mirrors.yun-idc.com
 * extras: mirror.jdcloud.com
 * updates: mirrors.tuna.tsinghua.edu.cn
epel                                                                                                                                                          12852/12852
epel                                                                                                                                                          12852/12852
epel                                                                                                                                                          12852/12852
Metadata Cache Created
[root@node101 ~]# 

创建yum缓存（[root@node101 ~]# yum makecache）

[root@node101 ~]# yum -y install  ansible
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirror.jdcloud.com
 * epel: mirrors.yun-idc.com
 * extras: mirror.jdcloud.com
 * updates: mirrors.tuna.tsinghua.edu.cn
Resolving Dependencies
--> Running transaction check
---> Package ansible.noarch 0:2.7.5-1.el7 will be installed
--> Processing Dependency: sshpass for package: ansible-2.7.5-1.el7.noarch
--> Processing Dependency: python2-jmespath for package: ansible-2.7.5-1.el7.noarch
--> Processing Dependency: python-paramiko for package: ansible-2.7.5-1.el7.noarch
--> Processing Dependency: python-keyczar for package: ansible-2.7.5-1.el7.noarch
--> Processing Dependency: python-jinja2 for package: ansible-2.7.5-1.el7.noarch
--> Processing Dependency: python-httplib2 for package: ansible-2.7.5-1.el7.noarch
--> Processing Dependency: python-crypto for package: ansible-2.7.5-1.el7.noarch
--> Processing Dependency: PyYAML for package: ansible-2.7.5-1.el7.noarch
--> Running transaction check
---> Package PyYAML.x86_64 0:3.10-11.el7 will be installed
--> Processing Dependency: libyaml-0.so.2()(64bit) for package: PyYAML-3.10-11.el7.x86_64
---> Package python-httplib2.noarch 0:0.9.2-1.el7 will be installed
---> Package python-jinja2.noarch 0:2.7.2-2.el7 will be installed
--> Processing Dependency: python-babel >= 0.8 for package: python-jinja2-2.7.2-2.el7.noarch
--> Processing Dependency: python-markupsafe for package: python-jinja2-2.7.2-2.el7.noarch
---> Package python-keyczar.noarch 0:0.71c-2.el7 will be installed
---> Package python-paramiko.noarch 0:2.1.1-9.el7 will be installed
--> Processing Dependency: python2-pyasn1 for package: python-paramiko-2.1.1-9.el7.noarch
---> Package python2-crypto.x86_64 0:2.6.1-15.el7 will be installed
--> Processing Dependency: libtomcrypt.so.0()(64bit) for package: python2-crypto-2.6.1-15.el7.x86_64
---> Package python2-jmespath.noarch 0:0.9.0-3.el7 will be installed
---> Package sshpass.x86_64 0:1.06-2.el7 will be installed
--> Running transaction check
---> Package libtomcrypt.x86_64 0:1.17-26.el7 will be installed
--> Processing Dependency: libtommath >= 0.42.0 for package: libtomcrypt-1.17-26.el7.x86_64
--> Processing Dependency: libtommath.so.0()(64bit) for package: libtomcrypt-1.17-26.el7.x86_64
---> Package libyaml.x86_64 0:0.1.4-11.el7_0 will be installed
---> Package python-babel.noarch 0:0.9.6-8.el7 will be installed
---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed
---> Package python-pyasn1.noarch 0:0.1.6-2.el7 will be obsoleted
---> Package python2-pyasn1.noarch 0:0.1.9-7.el7 will be obsoleting
--> Running transaction check
---> Package libtommath.x86_64 0:0.42.0-6.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==========================================================================================================================================================================
 Package                                       Arch                               Version                                       Repository                           Size
==========================================================================================================================================================================
Installing:
 ansible                                       noarch                             2.7.5-1.el7                                   epel                                 11 M
 python2-pyasn1                                noarch                             0.1.9-7.el7                                   base                                100 k
     replacing  python-pyasn1.noarch 0.1.6-2.el7
Installing for dependencies:
 PyYAML                                        x86_64                             3.10-11.el7                                   base                                153 k
 libtomcrypt                                   x86_64                             1.17-26.el7                                   extras                              224 k
 libtommath                                    x86_64                             0.42.0-6.el7                                  extras                               36 k
 libyaml                                       x86_64                             0.1.4-11.el7_0                                base                                 55 k
 python-babel                                  noarch                             0.9.6-8.el7                                   base                                1.4 M
 python-httplib2                               noarch                             0.9.2-1.el7                                   extras                              115 k
 python-jinja2                                 noarch                             2.7.2-2.el7                                   base                                515 k
 python-keyczar                                noarch                             0.71c-2.el7                                   epel                                218 k
 python-markupsafe                             x86_64                             0.11-10.el7                                   base                                 25 k
 python-paramiko                               noarch                             2.1.1-9.el7                                   updates                             269 k
 python2-crypto                                x86_64                             2.6.1-15.el7                                  extras                              477 k
 python2-jmespath                              noarch                             0.9.0-3.el7                                   extras                               39 k
 sshpass                                       x86_64                             1.06-2.el7                                    extras                               21 k

Transaction Summary
==========================================================================================================================================================================
Install  2 Packages (+13 Dependent packages)

Total download size: 15 M
Downloading packages:
(1/15): PyYAML-3.10-11.el7.x86_64.rpm                                                                                                              | 153 kB  00:00:00     
(2/15): libtomcrypt-1.17-26.el7.x86_64.rpm                                                                                                         | 224 kB  00:00:00     
(3/15): python-httplib2-0.9.2-1.el7.noarch.rpm                                                                                                     | 115 kB  00:00:00     
(4/15): python-jinja2-2.7.2-2.el7.noarch.rpm                                                                                                       | 515 kB  00:00:00     
(5/15): libtommath-0.42.0-6.el7.x86_64.rpm                                                                                                         |  36 kB  00:00:01     
(6/15): python-markupsafe-0.11-10.el7.x86_64.rpm                                                                                                   |  25 kB  00:00:00     
(7/15): python-paramiko-2.1.1-9.el7.noarch.rpm                                                                                                     | 269 kB  00:00:00     
(8/15): python2-crypto-2.6.1-15.el7.x86_64.rpm                                                                                                     | 477 kB  00:00:00     
(9/15): python2-jmespath-0.9.0-3.el7.noarch.rpm                                                                                                    |  39 kB  00:00:00     
(10/15): python2-pyasn1-0.1.9-7.el7.noarch.rpm                                                                                                     | 100 kB  00:00:00     
(11/15): sshpass-1.06-2.el7.x86_64.rpm                                                                                                             |  21 kB  00:00:00     
(12/15): python-babel-0.9.6-8.el7.noarch.rpm                                                                                                       | 1.4 MB  00:00:04     
(13/15): libyaml-0.1.4-11.el7_0.x86_64.rpm                                                                                                         |  55 kB  00:00:05     
warning: /var/cache/yum/x86_64/7/epel/packages/python-keyczar-0.71c-2.el7.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY7 kB/s | 9.4 MB  00:00:06 ETA 
Public key for python-keyczar-0.71c-2.el7.noarch.rpm is not installed
(14/15): python-keyczar-0.71c-2.el7.noarch.rpm                                                                                                     | 218 kB  00:00:09     
(15/15): ansible-2.7.5-1.el7.noarch.rpm                                                                                                            |  11 MB  00:00:17     
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                     870 kB/s |  15 MB  00:00:17     
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
Importing GPG key 0x352C64E5:
 Userid     : "Fedora EPEL (7) "
 Fingerprint: 91e9 7d7c 4a5e 96f1 7f3e 888f 6a2f aea2 352c 64e5
 Package    : epel-release-7-11.noarch (installed)
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
  Installing : python2-pyasn1-0.1.9-7.el7.noarch                                                                                                                     1/16 
  Installing : python-paramiko-2.1.1-9.el7.noarch                                                                                                                    2/16 
  Installing : python-babel-0.9.6-8.el7.noarch                                                                                                                       3/16 
  Installing : python-httplib2-0.9.2-1.el7.noarch                                                                                                                    4/16 
  Installing : sshpass-1.06-2.el7.x86_64                                                                                                                             5/16 
  Installing : libtommath-0.42.0-6.el7.x86_64                                                                                                                        6/16 
  Installing : libtomcrypt-1.17-26.el7.x86_64                                                                                                                        7/16 
  Installing : python2-crypto-2.6.1-15.el7.x86_64                                                                                                                    8/16 
  Installing : python-keyczar-0.71c-2.el7.noarch                                                                                                                     9/16 
  Installing : python-markupsafe-0.11-10.el7.x86_64                                                                                                                 10/16 
  Installing : python-jinja2-2.7.2-2.el7.noarch                                                                                                                     11/16 
  Installing : python2-jmespath-0.9.0-3.el7.noarch                                                                                                                  12/16 
  Installing : libyaml-0.1.4-11.el7_0.x86_64                                                                                                                        13/16 
  Installing : PyYAML-3.10-11.el7.x86_64                                                                                                                            14/16 
  Installing : ansible-2.7.5-1.el7.noarch                                                                                                                           15/16 
  Erasing    : python-pyasn1-0.1.6-2.el7.noarch                                                                                                                     16/16 
  Verifying  : python-keyczar-0.71c-2.el7.noarch                                                                                                                     1/16 
  Verifying  : libyaml-0.1.4-11.el7_0.x86_64                                                                                                                         2/16 
  Verifying  : python-jinja2-2.7.2-2.el7.noarch                                                                                                                      3/16 
  Verifying  : python2-jmespath-0.9.0-3.el7.noarch                                                                                                                   4/16 
  Verifying  : python-paramiko-2.1.1-9.el7.noarch                                                                                                                    5/16 
  Verifying  : python-markupsafe-0.11-10.el7.x86_64                                                                                                                  6/16 
  Verifying  : python2-crypto-2.6.1-15.el7.x86_64                                                                                                                    7/16 
  Verifying  : libtommath-0.42.0-6.el7.x86_64                                                                                                                        8/16 
  Verifying  : sshpass-1.06-2.el7.x86_64                                                                                                                             9/16 
  Verifying  : python-httplib2-0.9.2-1.el7.noarch                                                                                                                   10/16 
  Verifying  : ansible-2.7.5-1.el7.noarch                                                                                                                           11/16 
  Verifying  : PyYAML-3.10-11.el7.x86_64                                                                                                                            12/16 
  Verifying  : python-babel-0.9.6-8.el7.noarch                                                                                                                      13/16 
  Verifying  : libtomcrypt-1.17-26.el7.x86_64                                                                                                                       14/16 
  Verifying  : python2-pyasn1-0.1.9-7.el7.noarch                                                                                                                    15/16 
  Verifying  : python-pyasn1-0.1.6-2.el7.noarch                                                                                                                     16/16 

Installed:
  ansible.noarch 0:2.7.5-1.el7                                                     python2-pyasn1.noarch 0:0.1.9-7.el7                                                    

Dependency Installed:
  PyYAML.x86_64 0:3.10-11.el7                libtomcrypt.x86_64 0:1.17-26.el7         libtommath.x86_64 0:0.42.0-6.el7         libyaml.x86_64 0:0.1.4-11.el7_0          
  python-babel.noarch 0:0.9.6-8.el7          python-httplib2.noarch 0:0.9.2-1.el7     python-jinja2.noarch 0:2.7.2-2.el7       python-keyczar.noarch 0:0.71c-2.el7      
  python-markupsafe.x86_64 0:0.11-10.el7     python-paramiko.noarch 0:2.1.1-9.el7     python2-crypto.x86_64 0:2.6.1-15.el7     python2-jmespath.noarch 0:0.9.0-3.el7    
  sshpass.x86_64 0:1.06-2.el7               

Replaced:
  python-pyasn1.noarch 0:0.1.6-2.el7                                                                                                                                      

Complete!
[root@node101 ~]# 

安装ansible（[root@node101 ~]# yum -y install ansible）

4>.配置ansible的配置文件

　　我根据我的需求，将主机分为2个角色，大家都知道，hdfs集群又NameNode和DataNode之分，因此我将配置相对较高的那台机器配置为NameNode节点，然后把DataNode服务安置在所有的节点上。

[root@node101 ~]# tail -8 /etc/ansible/hosts 
#Add by yinzhengjie
[namenode]
node101.yinzhengjie.org.cn

[datanode]
node101.yinzhengjie.org.cn
node102.yinzhengjie.org.cn
node103.yinzhengjie.org.cn
[root@node101 ~]# 
[root@node101 ~]# 

5>.安装相关依赖插件，解决cryptography 0.8.2 版本兼容性（如果该步骤不做的话，在你执行“ansible all -m ping”命令时可能会报错！）

　　5.1>.安装pip

[root@node101 ~]# wget  https://files.pythonhosted.org/packages/45/ae/8a0ad77defb7cc903f09e551d88b443304a9bd6e6f124e75c0fbbf6de8f7/pip-18.1.tar.gz
--2019-01-10 18:11:31--  https://files.pythonhosted.org/packages/45/ae/8a0ad77defb7cc903f09e551d88b443304a9bd6e6f124e75c0fbbf6de8f7/pip-18.1.tar.gz
Resolving files.pythonhosted.org (files.pythonhosted.org)... 151.101.109.63, 2a04:4e42:36::319
Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.109.63|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1259370 (1.2M) [binary/octet-stream]
Saving to: ‘pip-18.1.tar.gz’

100%[================================================================================================================================>] 1,259,370   64.7KB/s   in 22s    

2019-01-10 18:11:55 (55.4 KB/s) - ‘pip-18.1.tar.gz’ saved [1259370/1259370]

[root@node101 ~]# 

下载pip（[root@node101 ~]# wget https://files.pythonhosted.org/packages/45/ae/8a0ad77defb7cc903f09e551d88b443304a9bd6e6f124e75c0fbbf6de8f7/pip-18.1.tar.gz） 

[root@node101 ~]# tar -zxf pip-18.1.tar.gz 
[root@node101 ~]# ll
total 1252
-rw-r--r--. 1 root root    15080 Oct  3  2017 epel-release-latest-7.noarch.rpm
drwxr-xr-x. 6  501 games    4096 Jan 10 18:13 pip-18.1
-rw-r--r--. 1 root root  1259370 Oct  5 19:20 pip-18.1.tar.gz
[root@node101 ~]# 
[root@node101 ~]# cd pip-18.1/
[root@node101 pip-18.1]# 
[root@node101 pip-18.1]# python setup.py  install
.........(内容输出过多，我这里就统一复制了！)
Installing pip2.7 script to /usr/bin
Installing pip2 script to /usr/bin

Installed /usr/lib/python2.7/site-packages/pip-18.1-py2.7.egg
Processing dependencies for pip==18.1
Finished processing dependencies for pip==18.1
[root@node101 pip-18.1]#  

安装pip工具（[root@node101 pip-18.1]# python setup.py install）

　　5.2>.安装wheel 

[root@node101 ~]# wget  https://files.pythonhosted.org/packages/d8/55/221a530d66bf78e72996453d1e2dedef526063546e131d70bed548d80588/wheel-0.32.3.tar.gz
--2019-01-10 18:16:37--  https://files.pythonhosted.org/packages/d8/55/221a530d66bf78e72996453d1e2dedef526063546e131d70bed548d80588/wheel-0.32.3.tar.gz
Resolving files.pythonhosted.org (files.pythonhosted.org)... 151.101.109.63, 2a04:4e42:1a::319
Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.109.63|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 18585 (18K) [binary/octet-stream]
Saving to: ‘wheel-0.32.3.tar.gz’

100%[================================================================================================================================>] 18,585      --.-K/s   in 0.1s    

2019-01-10 18:16:38 (162 KB/s) - ‘wheel-0.32.3.tar.gz’ saved [18585/18585]

[root@node101 ~]# ll
total 1272
-rw-r--r--. 1 root root    15080 Oct  3  2017 epel-release-latest-7.noarch.rpm
drwxr-xr-x. 6  501 games    4096 Jan 10 18:13 pip-18.1
-rw-r--r--. 1 root root  1259370 Oct  5 19:20 pip-18.1.tar.gz
-rw-r--r--. 1 root root    18585 Nov 19 08:26 wheel-0.32.3.tar.gz
[root@node101 ~]# 
[root@node101 ~]# 

下载wheel（[root@node101 ~]# wget https://files.pythonhosted.org/packages/d8/55/221a530d66bf78e72996453d1e2dedef526063546e131d70bed548d80588/wheel-0.32.3.tar.gz）

[root@node101 ~]# ll
total 1272
-rw-r--r--. 1 root root    15080 Oct  3  2017 epel-release-latest-7.noarch.rpm
drwxr-xr-x. 6  501 games    4096 Jan 10 18:13 pip-18.1
-rw-r--r--. 1 root root  1259370 Oct  5 19:20 pip-18.1.tar.gz
-rw-r--r--. 1 root root    18585 Nov 19 08:26 wheel-0.32.3.tar.gz
[root@node101 ~]# 
[root@node101 ~]# tar -zxf wheel-0.32.3.tar.gz 
[root@node101 ~]# 
[root@node101 ~]# ll
total 1276
-rw-r--r--. 1 root root    15080 Oct  3  2017 epel-release-latest-7.noarch.rpm
drwxr-xr-x. 6  501 games    4096 Jan 10 18:13 pip-18.1
-rw-r--r--. 1 root root  1259370 Oct  5 19:20 pip-18.1.tar.gz
drwxrwxr-x. 4 2000  2000    4096 Nov 19 08:25 wheel-0.32.3
-rw-r--r--. 1 root root    18585 Nov 19 08:26 wheel-0.32.3.tar.gz
[root@node101 ~]# 
[root@node101 ~]# cd wheel-0.32.3/
[root@node101 wheel-0.32.3]# ll
total 32
-rw-rw-r--. 1 2000 2000 1125 Nov 19 08:25 LICENSE.txt
-rw-rw-r--. 1 2000 2000  244 Nov 19 08:25 MANIFEST.in
-rw-rw-r--. 1 2000 2000 2207 Nov 19 08:25 PKG-INFO
-rw-rw-r--. 1 2000 2000  799 Nov 19 08:25 README.rst
-rw-rw-r--. 1 2000 2000  276 Nov 19 08:25 setup.cfg
-rw-rw-r--. 1 2000 2000 2133 Nov 19 08:25 setup.py
drwxrwxr-x. 3 2000 2000 4096 Nov 19 08:25 wheel
drwxrwxr-x. 2 2000 2000 4096 Nov 19 08:25 wheel.egg-info
[root@node101 wheel-0.32.3]#
[root@node101 wheel-0.32.3]# python setup.py  install
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'project_urls'
  warnings.warn(msg)
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
  warnings.warn(msg)
running install
running bdist_egg
running egg_info
writing requirements to wheel.egg-info/requires.txt
writing wheel.egg-info/PKG-INFO
writing top-level names to wheel.egg-info/top_level.txt
writing dependency_links to wheel.egg-info/dependency_links.txt
writing entry points to wheel.egg-info/entry_points.txt
reading manifest file 'wheel.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'wheel/*.txt'
warning: no files found matching '*.sh'
warning: no files found matching '*.py' under directory 'wheel/test'
warning: no files found matching 'wheel/test/test-1.0-py2.py3-none-win32.whl'
warning: no files found matching 'wheel/test/headers.dist/header.h'
warning: no files found matching 'wheel/test/pydist-schema.json'
no previously-included directories found matching 'wheel/test/*/dist'
no previously-included directories found matching 'wheel/test/*/build'
writing manifest file 'wheel.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/wheel
copying wheel/wheelfile.py -> build/lib/wheel
copying wheel/pep425tags.py -> build/lib/wheel
copying wheel/__init__.py -> build/lib/wheel
copying wheel/util.py -> build/lib/wheel
copying wheel/metadata.py -> build/lib/wheel
copying wheel/__main__.py -> build/lib/wheel
copying wheel/bdist_wheel.py -> build/lib/wheel
copying wheel/pkginfo.py -> build/lib/wheel
creating build/lib/wheel/cli
copying wheel/cli/install.py -> build/lib/wheel/cli
copying wheel/cli/pack.py -> build/lib/wheel/cli
copying wheel/cli/__init__.py -> build/lib/wheel/cli
copying wheel/cli/convert.py -> build/lib/wheel/cli
copying wheel/cli/unpack.py -> build/lib/wheel/cli
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/wheelfile.py -> build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/pep425tags.py -> build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/__init__.py -> build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/util.py -> build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/metadata.py -> build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/__main__.py -> build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/bdist_wheel.py -> build/bdist.linux-x86_64/egg/wheel
copying build/lib/wheel/pkginfo.py -> build/bdist.linux-x86_64/egg/wheel
creating build/bdist.linux-x86_64/egg/wheel/cli
copying build/lib/wheel/cli/install.py -> build/bdist.linux-x86_64/egg/wheel/cli
copying build/lib/wheel/cli/pack.py -> build/bdist.linux-x86_64/egg/wheel/cli
copying build/lib/wheel/cli/__init__.py -> build/bdist.linux-x86_64/egg/wheel/cli
copying build/lib/wheel/cli/convert.py -> build/bdist.linux-x86_64/egg/wheel/cli
copying build/lib/wheel/cli/unpack.py -> build/bdist.linux-x86_64/egg/wheel/cli
byte-compiling build/bdist.linux-x86_64/egg/wheel/wheelfile.py to wheelfile.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/pep425tags.py to pep425tags.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/util.py to util.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/metadata.py to metadata.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/__main__.py to __main__.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/bdist_wheel.py to bdist_wheel.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/pkginfo.py to pkginfo.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/cli/install.py to install.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/cli/pack.py to pack.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/cli/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/cli/convert.py to convert.pyc
byte-compiling build/bdist.linux-x86_64/egg/wheel/cli/unpack.py to unpack.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying wheel.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying wheel.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying wheel.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying wheel.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying wheel.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO
copying wheel.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying wheel.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
creating dist
creating 'dist/wheel-0.32.3-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing wheel-0.32.3-py2.7.egg
creating /usr/lib/python2.7/site-packages/wheel-0.32.3-py2.7.egg
Extracting wheel-0.32.3-py2.7.egg to /usr/lib/python2.7/site-packages
Adding wheel 0.32.3 to easy-install.pth file
Installing wheel script to /usr/bin

Installed /usr/lib/python2.7/site-packages/wheel-0.32.3-py2.7.egg
Processing dependencies for wheel==0.32.3
Finished processing dependencies for wheel==0.32.3
[root@node101 wheel-0.32.3]# 

安装wheel工具（[root@node101 wheel-0.32.3]# python setup.py install）

　　5.3>.安装cryptography 

[root@node101 ~]# wget  https://files.pythonhosted.org/packages/7f/ba/383b51cc26e3141c689ce988814385c7659f5ba01c4b5f2de38233010b5f/cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl
--2019-01-10 18:23:46--  https://files.pythonhosted.org/packages/7f/ba/383b51cc26e3141c689ce988814385c7659f5ba01c4b5f2de38233010b5f/cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl
Resolving files.pythonhosted.org (files.pythonhosted.org)... 151.101.73.63, 2a04:4e42:11::319
Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.73.63|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2138565 (2.0M) [binary/octet-stream]
Saving to: ‘cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl’

100%[================================================================================================================================>] 2,138,565   47.2KB/s   in 24s    

2019-01-10 18:24:11 (85.8 KB/s) - ‘cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl’ saved [2138565/2138565]

[root@node101 ~]# 

下载cryptography（[root@node101 ~]# wget https://files.pythonhosted.org/packages/7f/ba/383b51cc26e3141c689ce988814385c7659f5ba01c4b5f2de38233010b5f/cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl）

[root@node101 ~]# pip install cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl 
Processing ./cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting idna>=2.1 (from cryptography==2.4.2)
  Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)
    100% |████████████████████████████████| 61kB 248kB/s 
Requirement already satisfied: enum34; python_version < "3" in /usr/lib/python2.7/site-packages (from cryptography==2.4.2) (1.0.4)
Collecting cffi!=1.11.3,>=1.7 (from cryptography==2.4.2)
  Downloading https://files.pythonhosted.org/packages/14/dd/3e7a1e1280e7d767bd3fa15791759c91ec19058ebe31217fe66f3e9a8c49/cffi-1.11.5-cp27-cp27mu-manylinux1_x86_64.whl (407kB)
    100% |████████████████████████████████| 409kB 147kB/s 
Requirement already satisfied: six>=1.4.1 in /usr/lib/python2.7/site-packages (from cryptography==2.4.2) (1.9.0)
Collecting ipaddress; python_version < "3" (from cryptography==2.4.2)
  Downloading https://files.pythonhosted.org/packages/fc/d0/7fc3a811e011d4b388be48a0e381db8d990042df54aa4ef4599a31d39853/ipaddress-1.0.22-py2.py3-none-any.whl
Collecting asn1crypto>=0.21.0 (from cryptography==2.4.2)
  Downloading https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl (101kB)
    100% |████████████████████████████████| 102kB 292kB/s 
Requirement already satisfied: pycparser in /usr/lib/python2.7/site-packages (from cffi!=1.11.3,>=1.7->cryptography==2.4.2) (2.14)
Installing collected packages: idna, cffi, ipaddress, asn1crypto, cryptography
  Found existing installation: cffi 0.8.6
    Uninstalling cffi-0.8.6:
      Successfully uninstalled cffi-0.8.6
  Found existing installation: cryptography 0.8.2
    Uninstalling cryptography-0.8.2:
      Successfully uninstalled cryptography-0.8.2
Successfully installed asn1crypto-0.24.0 cffi-1.11.5 cryptography-2.4.2 idna-2.8 ipaddress-1.0.22
[root@node101 ~]# 

安装cryptography（[root@node101 ~]# pip install cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl ）

6>.使用ansible同步文件到其他节点

[root@node101 ~]# ansible all -m ping 
node102.yinzhengjie.org.cn | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
node103.yinzhengjie.org.cn | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
node101.yinzhengjie.org.cn | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
[root@node101 ~]# 

检查所有节点是否在线（[root@node101 ~]# ansible all -m ping ）

[root@node101 ~]# ansible datanode -m copy -a 'src=/etc/hosts dest=/etc/hosts'
node101.yinzhengjie.org.cn | SUCCESS => {
    "changed": false, 
    "checksum": "6b35ffc3c04d3ec2d3464b74ea0f78c20f27131a", 
    "dest": "/etc/hosts", 
    "gid": 0, 
    "group": "root", 
    "mode": "0644", 
    "owner": "root", 
    "path": "/etc/hosts", 
    "secontext": "system_u:object_r:net_conf_t:s0", 
    "size": 279, 
    "state": "file", 
    "uid": 0
}
node102.yinzhengjie.org.cn | CHANGED => {
    "changed": true, 
    "checksum": "6b35ffc3c04d3ec2d3464b74ea0f78c20f27131a", 
    "dest": "/etc/hosts", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "494c828fc321572368ec771a390f90c1", 
    "mode": "0644", 
    "owner": "root", 
    "secontext": "system_u:object_r:net_conf_t:s0", 
    "size": 279, 
    "src": "/root/.ansible/tmp/ansible-tmp-1547116266.5-263369209847793/source", 
    "state": "file", 
    "uid": 0
}
node103.yinzhengjie.org.cn | CHANGED => {
    "changed": true, 
    "checksum": "6b35ffc3c04d3ec2d3464b74ea0f78c20f27131a", 
    "dest": "/etc/hosts", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "494c828fc321572368ec771a390f90c1", 
    "mode": "0644", 
    "owner": "root", 
    "secontext": "system_u:object_r:net_conf_t:s0", 
    "size": 279, 
    "src": "/root/.ansible/tmp/ansible-tmp-1547116266.52-44859007354579/source", 
    "state": "file", 
    "uid": 0
}
[root@node101 ~]# 
[root@node101 ~]# ansible all -m shell -a 'cat /etc/hosts|grep yinzhengjie'   
node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
172.30.1.101 node101.yinzhengjie.org.cn
172.30.1.102 node102.yinzhengjie.org.cn
172.30.1.103 node103.yinzhengjie.org.cn

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
172.30.1.101 node101.yinzhengjie.org.cn
172.30.1.102 node102.yinzhengjie.org.cn
172.30.1.103 node103.yinzhengjie.org.cn

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
172.30.1.101 node101.yinzhengjie.org.cn
172.30.1.102 node102.yinzhengjie.org.cn
172.30.1.103 node103.yinzhengjie.org.cn

[root@node101 ~]# 

将namenode节点的数据同步到其他节点中（[root@node101 ~]# ansible datanode -m copy -a 'src=/etc/hosts dest=/etc/hosts'）

 

二.安装jdk环境

1>.卸载jdk1.7版本

[root@node101 ~]# ansible all -m shell -a 'rpm -qa | grep java'
 [WARNING]: Consider using the yum, dnf or zypper module rather than running rpm.  If you need to use command because yum, dnf or zypper is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
javapackages-tools-3.4.1-11.el7.noarch
tzdata-java-2015g-1.el7.noarch
java-1.7.0-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64
python-javapackages-3.4.1-11.el7.noarch
java-1.7.0-openjdk-1.7.0.91-2.6.2.3.el7.x86_64

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
javapackages-tools-3.4.1-11.el7.noarch
tzdata-java-2015g-1.el7.noarch
java-1.7.0-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64
python-javapackages-3.4.1-11.el7.noarch
java-1.7.0-openjdk-1.7.0.91-2.6.2.3.el7.x86_64

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
javapackages-tools-3.4.1-11.el7.noarch
tzdata-java-2015g-1.el7.noarch
java-1.7.0-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64
python-javapackages-3.4.1-11.el7.noarch
java-1.7.0-openjdk-1.7.0.91-2.6.2.3.el7.x86_64

[root@node101 ~]# 

使用ansible检查各个节点是否安装jdk（[root@node101 ~]# ansible all -m shell -a 'rpm -qa | grep java'）

[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps javapackages-tools-3.4.1-11.el7.noarch'
 [WARNING]: Consider using the yum, dnf or zypper module rather than running rpm.  If you need to use command because yum, dnf or zypper is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 ~]# 

卸载javapackages软件包（ [root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps javapackages-tools-3.4.1-11.el7.noarch'）

[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps tzdata-java-2015g-1.el7.noarch'        
 [WARNING]: Consider using the yum, dnf or zypper module rather than running rpm.  If you need to use command because yum, dnf or zypper is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 ~]# 

卸载tzdata软件包（[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps tzdata-java-2015g-1.el7.noarch' ）

[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps java-1.7.0-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64'
 [WARNING]: Consider using the yum, dnf or zypper module rather than running rpm.  If you need to use command because yum, dnf or zypper is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 ~]# 

卸载openjdk软件包（[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps java-1.7.0-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64'）

[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps python-javapackages-3.4.1-11.el7.noarch'                
 [WARNING]: Consider using the yum, dnf or zypper module rather than running rpm.  If you need to use command because yum, dnf or zypper is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 ~]#

[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps python-javapackages-3.4.1-11.el7.noarch'

[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps java-1.7.0-openjdk-1.7.0.91-2.6.2.3.el7.x86_64'
 [WARNING]: Consider using the yum, dnf or zypper module rather than running rpm.  If you need to use command because yum, dnf or zypper is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 ~]# 

[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps java-1.7.0-openjdk-1.7.0.91-2.6.2.3.el7.x86_64'

[root@node101 ~]# ansible all -m shell -a  'rpm -qa | grep java' 
 [WARNING]: Consider using the yum, dnf or zypper module rather than running rpm.  If you need to use command because yum, dnf or zypper is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node103.yinzhengjie.org.cn | FAILED | rc=1 >>
non-zero return code

node102.yinzhengjie.org.cn | FAILED | rc=1 >>
non-zero return code

node101.yinzhengjie.org.cn | FAILED | rc=1 >>
non-zero return code

[root@node101 ~]# 

检查jdk1.7相关软件包是否卸载完成（[root@node101 ~]# ansible all -m shell -a 'rpm -qa | grep java' ）

2>.安装jdk1.8版本几以上版本

 　　大家可以自行去Oracle官网下载相应的jdk，我这里给出下载地址：https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html（推荐下载tar.gz的安装包）。

[root@node101 ~]# ansible all -m shell -a 'mkdir  -pv /yinzhengjie/softwares'
 [WARNING]: Consider using the file module with state=directory rather than running mkdir.  If you need to use command because file is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
mkdir: created directory ‘/yinzhengjie’
mkdir: created directory ‘/yinzhengjie/softwares’

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
mkdir: created directory ‘/yinzhengjie’
mkdir: created directory ‘/yinzhengjie/softwares’

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
mkdir: created directory ‘/yinzhengjie’
mkdir: created directory ‘/yinzhengjie/softwares’

[root@node101 ~]# 

创建指定的软件存放目录，这个目录大家可以自定义创建目录（[root@node101 ~]# ansible all -m shell -a 'mkdir -pv /yinzhengjie/softwares'）

[root@node101 ~]# ansible namenode -m shell -a 'tar -zxf ~/jdk-8u131-linux-x64.tar.gz -C /yinzhengjie/softwares'
 [WARNING]: Consider using the unarchive module rather than running tar.  If you need to use command because unarchive is insufficient you can add warn=False to this
command task or set command_warnings=False in ansible.cfg to get rid of this message.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 ~]# ls /yinzhengjie/softwares/
jdk1.8.0_131
[root@node101 ~]# 
[root@node101 ~]# ll /yinzhengjie/softwares/jdk1.8.0_131/
total 25884
drwxr-xr-x. 2 10 143     4096 Mar 15  2017 bin
-r--r--r--. 1 10 143     3244 Mar 15  2017 COPYRIGHT
drwxr-xr-x. 4 10 143     4096 Mar 15  2017 db
drwxr-xr-x. 3 10 143     4096 Mar 15  2017 include
-rwxr-xr-x. 1 10 143  5097105 Mar 15  2017 javafx-src.zip
drwxr-xr-x. 5 10 143     4096 Mar 15  2017 jre
drwxr-xr-x. 5 10 143     4096 Mar 15  2017 lib
-r--r--r--. 1 10 143       40 Mar 15  2017 LICENSE
drwxr-xr-x. 4 10 143     4096 Mar 15  2017 man
-r--r--r--. 1 10 143      159 Mar 15  2017 README.html
-rw-r--r--. 1 10 143      526 Mar 15  2017 release
-rw-r--r--. 1 10 143 21115141 Mar 15  2017 src.zip
-rwxr-xr-x. 1 10 143    63933 Mar 15  2017 THIRDPARTYLICENSEREADME-JAVAFX.txt
-r--r--r--. 1 10 143   177094 Mar 15  2017 THIRDPARTYLICENSEREADME.txt
[root@node101 ~]# 

解压jdk软件（[root@node101 ~]# ansible namenode -m shell -a 'tar -zxf ~/jdk-8u131-linux-x64.tar.gz -C /yinzhengjie/softwares'）

[root@node101 ~]# ansible  datanode -m copy -a 'src=/yinzhengjie/softwares/jdk1.8.0_131  dest=/yinzhengjie/softwares'       
node101.yinzhengjie.org.cn | CHANGED => {
    "changed": true, 
    "dest": "/yinzhengjie/softwares/", 
    "src": "/yinzhengjie/softwares/jdk1.8.0_131"
}
node103.yinzhengjie.org.cn | CHANGED => {
    "changed": true, 
    "dest": "/yinzhengjie/softwares/", 
    "src": "/yinzhengjie/softwares/jdk1.8.0_131"
}
node102.yinzhengjie.org.cn | CHANGED => {
    "changed": true, 
    "dest": "/yinzhengjie/softwares/", 
    "src": "/yinzhengjie/softwares/jdk1.8.0_131"
}
[root@node101 ~]# 

将主节点的java目录同步到其他节点（[root@node101 ~]# ansible datanode -m copy -a 'src=/yinzhengjie/softwares/jdk1.8.0_131 dest=/yinzhengjie/softwares'）

[root@node101 ~]# ansible datanode -m shell -a 'ln -s /yinzhengjie/softwares/jdk1.8.0_131 /yinzhengjie/softwares/jdk'             
 [WARNING]: Consider using the file module with state=link rather than running ln.  If you need to use command because file is insufficient you can add warn=False to
this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 ~]# 

创建符号链接，即软连接（[root@node101 ~]# ansible datanode -m shell -a 'ln -s /yinzhengjie/softwares/jdk1.8.0_131 /yinzhengjie/softwares/jdk' ） 

3>.配置环境java的系统环境变量

[root@node101 ~]# tail -3 /etc/profile
#ADD JAVA_PATH
JAVA_HOME=/yinzhengjie/softwares/jdk
PATH=$PATH:$JAVA_HOME/bin
[root@node101 ~]# 
[root@node101 ~]# ansible datanode -m copy -a 'src=/etc/profile dest=/etc/profile'
node103.yinzhengjie.org.cn | CHANGED => {
    "changed": true, 
    "checksum": "5b3a42ee16d78583b834dc0dea67d282b5fa5323", 
    "dest": "/etc/profile", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "cd04b3ecdde83a7c250b923167dc0c01", 
    "mode": "0644", 
    "owner": "root", 
    "secontext": "system_u:object_r:etc_t:s0", 
    "size": 1828, 
    "src": "/root/.ansible/tmp/ansible-tmp-1547122009.37-153584791304684/source", 
    "state": "file", 
    "uid": 0
}
node102.yinzhengjie.org.cn | CHANGED => {
    "changed": true, 
    "checksum": "5b3a42ee16d78583b834dc0dea67d282b5fa5323", 
    "dest": "/etc/profile", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "cd04b3ecdde83a7c250b923167dc0c01", 
    "mode": "0644", 
    "owner": "root", 
    "secontext": "system_u:object_r:etc_t:s0", 
    "size": 1828, 
    "src": "/root/.ansible/tmp/ansible-tmp-1547122009.34-130374655280496/source", 
    "state": "file", 
    "uid": 0
}
node101.yinzhengjie.org.cn | SUCCESS => {
    "changed": false, 
    "checksum": "5b3a42ee16d78583b834dc0dea67d282b5fa5323", 
    "dest": "/etc/profile", 
    "gid": 0, 
    "group": "root", 
    "mode": "0644", 
    "owner": "root", 
    "path": "/etc/profile", 
    "secontext": "system_u:object_r:etc_t:s0", 
    "size": 1828, 
    "state": "file", 
    "uid": 0
}
[root@node101 ~]#

[root@node101 ~]# ansible datanode -m copy -a 'src=/etc/profile dest=/etc/profile'

[root@node101 ~]# ansible datanode -m shell -a 'chmod -R 777 /yinzhengjie/softwares/jdk/bin'
 [WARNING]: Consider using the file module with mode rather than running chmod.  If you need to use command because file is insufficient you can add warn=False to this
command task or set command_warnings=False in ansible.cfg to get rid of this message.

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 ~]# 

取保拷贝到其他节点到java脚本拥有相应的执行权限（[root@node101 ~]# ansible datanode -m shell -a 'chmod -R 777 /yinzhengjie/softwares/jdk/bin'）

[root@node101 ~]# ansible datanode -m shell -a 'source /etc/profile;java -version'          
node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

[root@node101 ~]# 

查看每个节点的环境是否可用（[root@node101 ~]# ansible datanode -m shell -a 'source /etc/profile;java -version' ） 

[root@node101 ~]# ansible datanode -m shell -a 'mkdir /usr/java'                
 [WARNING]: Consider using the file module with state=directory rather than running mkdir.  If you need to use command because file is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 ~]#

[root@node101 ~]# ansible datanode -m shell -a 'mkdir /usr/java' 

[root@node101 ~]# ansible datanode -m shell -a 'ln -s /yinzhengjie/softwares/jdk1.8.0_131/ /usr/java/jdk1.8'
 [WARNING]: Consider using the file module with state=link rather than running ln.  If you need to use command because file is insufficient you can add warn=False to
this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 ~]# 

[root@node101 ~]# ansible datanode -m shell -a 'ln -s /yinzhengjie/softwares/jdk1.8.0_131/ /usr/java/jdk1.8'

[root@node101 ~]# ansible datanode -m shell -a 'ln -s /yinzhengjie/softwares/jdk1.8.0_131/ /usr/java/defalut'
 [WARNING]: Consider using the file module with state=link rather than running ln.  If you need to use command because file is insufficient you can add warn=False to
this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 ~]# 

[root@node101 ~]# ansible datanode -m shell -a 'ln -s /yinzhengjie/softwares/jdk1.8.0_131/ /usr/java/defalut'

[root@node101 ~]# ansible datanode -m shell -a 'ls -l /usr/java'
node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
total 0
lrwxrwxrwx. 1 root root 36 Jan 10 20:55 defalut -> /yinzhengjie/softwares/jdk1.8.0_131/
lrwxrwxrwx. 1 root root 36 Jan 10 20:55 jdk1.8 -> /yinzhengjie/softwares/jdk1.8.0_131/

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
total 0
lrwxrwxrwx. 1 root root 36 Jan 10 20:55 defalut -> /yinzhengjie/softwares/jdk1.8.0_131/
lrwxrwxrwx. 1 root root 36 Jan 10 20:55 jdk1.8 -> /yinzhengjie/softwares/jdk1.8.0_131/

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
total 0
lrwxrwxrwx. 1 root root 36 Jan 10 20:55 defalut -> /yinzhengjie/softwares/jdk1.8.0_131/
lrwxrwxrwx. 1 root root 36 Jan 10 20:55 jdk1.8 -> /yinzhengjie/softwares/jdk1.8.0_131/

[root@node101 ~]# 

[root@node101 ~]# ansible datanode -m shell -a 'ls -l /usr/java'

 

三.安装mysql 

1>.下载mysql的yum源

[root@node101 ~]# wget http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
--2019-01-10 21:08:45--  http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
Resolving dev.mysql.com (dev.mysql.com)... 137.254.60.11
Connecting to dev.mysql.com (dev.mysql.com)|137.254.60.11|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm [following]
--2019-01-10 21:08:45--  https://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
Connecting to dev.mysql.com (dev.mysql.com)|137.254.60.11|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://repo.mysql.com//mysql-community-release-el7-5.noarch.rpm [following]
--2019-01-10 21:08:48--  https://repo.mysql.com//mysql-community-release-el7-5.noarch.rpm
Resolving repo.mysql.com (repo.mysql.com)... 104.76.8.253
Connecting to repo.mysql.com (repo.mysql.com)|104.76.8.253|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6140 (6.0K) [application/x-redhat-package-manager]
Saving to: ‘mysql-community-release-el7-5.noarch.rpm’

100%[================================================================================================================================>] 6,140       --.-K/s   in 0s      

2019-01-10 21:08:50 (529 MB/s) - ‘mysql-community-release-el7-5.noarch.rpm’ saved [6140/6140]

[root@node101 ~]#
[root@node101 ~]# ll
total 8
-rw-r--r--. 1 root root 6140 Nov 12  2015 mysql-community-release-el7-5.noarch.rpm
[root@node101 ~]# 

[root@node101 ~]# wget http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm

2>.安装mysql的yum源

[root@node101 ~]# ll
total 8
-rw-r--r--. 1 root root 6140 Nov 12  2015 mysql-community-release-el7-5.noarch.rpm
[root@node101 ~]# 
[root@node101 ~]# rpm -ivh mysql-community-release-el7-5.noarch.rpm 
Preparing...                          ################################# [100%]
Updating / installing...
   1:mysql-community-release-el7-5    ################################# [100%]
[root@node101 ~]# 
[root@node101 ~]# ll /etc/yum.repos.d/ | grep mysql
-rw-r--r--. 1 root root 1209 Jan 29  2014 mysql-community.repo
-rw-r--r--. 1 root root 1060 Jan 29  2014 mysql-community-source.repo
[root@node101 ~]# 

[root@node101 ~]# rpm -ivh mysql-community-release-el7-5.noarch.rpm

3>.安装mysql服务

[root@node101 ~]# yum -y install mysql-community-server
Loaded plugins: fastestmirror, langpacks
mysql-connectors-community                                                                                                                         | 2.5 kB  00:00:00     
mysql-tools-community                                                                                                                              | 2.5 kB  00:00:01     
mysql56-community                                                                                                                                  | 2.5 kB  00:00:02     
(1/3): mysql-tools-community/x86_64/primary_db                                                                                                     |  48 kB  00:00:00     
(2/3): mysql-connectors-community/x86_64/primary_db                                                                                                |  29 kB  00:00:00     
(3/3): mysql56-community/x86_64/primary_db                                                                                                         | 209 kB  00:00:03     
Loading mirror speeds from cached hostfile
 * base: mirror.jdcloud.com
 * epel: mirrors.yun-idc.com
 * extras: mirror.jdcloud.com
 * updates: mirrors.tuna.tsinghua.edu.cn
Resolving Dependencies
--> Running transaction check
---> Package mysql-community-server.x86_64 0:5.6.42-2.el7 will be installed
--> Processing Dependency: mysql-community-common(x86-64) = 5.6.42-2.el7 for package: mysql-community-server-5.6.42-2.el7.x86_64
--> Processing Dependency: mysql-community-client(x86-64) >= 5.6.10 for package: mysql-community-server-5.6.42-2.el7.x86_64
--> Running transaction check
---> Package mysql-community-client.x86_64 0:5.6.42-2.el7 will be installed
--> Processing Dependency: mysql-community-libs(x86-64) >= 5.6.10 for package: mysql-community-client-5.6.42-2.el7.x86_64
---> Package mysql-community-common.x86_64 0:5.6.42-2.el7 will be installed
--> Running transaction check
---> Package mariadb-libs.x86_64 1:5.5.44-2.el7.centos will be obsoleted
---> Package mysql-community-libs.x86_64 0:5.6.42-2.el7 will be obsoleting
--> Finished Dependency Resolution

Dependencies Resolved

==========================================================================================================================================================================
 Package                                         Arch                            Version                                 Repository                                  Size
==========================================================================================================================================================================
Installing:
 mysql-community-libs                            x86_64                          5.6.42-2.el7                            mysql56-community                          2.0 M
     replacing  mariadb-libs.x86_64 1:5.5.44-2.el7.centos
 mysql-community-server                          x86_64                          5.6.42-2.el7                            mysql56-community                           59 M
Installing for dependencies:
 mysql-community-client                          x86_64                          5.6.42-2.el7                            mysql56-community                           20 M
 mysql-community-common                          x86_64                          5.6.42-2.el7                            mysql56-community                          257 k

Transaction Summary
==========================================================================================================================================================================
Install  2 Packages (+2 Dependent packages)

Total download size: 81 M
Downloading packages:
warning: /var/cache/yum/x86_64/7/mysql56-community/packages/mysql-community-common-5.6.42-2.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY02:24 ETA 
Public key for mysql-community-common-5.6.42-2.el7.x86_64.rpm is not installed
(1/4): mysql-community-common-5.6.42-2.el7.x86_64.rpm                                                                                              | 257 kB  00:00:01     
(2/4): mysql-community-libs-5.6.42-2.el7.x86_64.rpm                                                                                                | 2.0 MB  00:00:10     
(3/4): mysql-community-client-5.6.42-2.el7.x86_64.rpm                                                                                              |  20 MB  00:03:29     
(4/4): mysql-community-server-5.6.42-2.el7.x86_64.rpm                                                                                              |  59 MB  00:05:01     
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                     264 kB/s |  81 MB  00:05:14     
Retrieving key from file:/etc/pki/rpm-gpg/RPM-GPG-KEY-mysql
Importing GPG key 0x5072E1F5:
 Userid     : "MySQL Release Engineering "
 Fingerprint: a4a9 4068 76fc bd3c 4567 70c8 8c71 8d3b 5072 e1f5
 Package    : mysql-community-release-el7-5.noarch (installed)
 From       : file:/etc/pki/rpm-gpg/RPM-GPG-KEY-mysql
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
** Found 2 pre-existing rpmdb problem(s), 'yum check' output follows:
1:libreoffice-core-4.3.7.2-5.el7.x86_64 has missing requires of java-headless >= ('1', '1.6', None)
1:libreoffice-ure-4.3.7.2-5.el7.x86_64 has missing requires of libjvm.so()(64bit)
  Installing : mysql-community-common-5.6.42-2.el7.x86_64                                                                                                             1/5 
  Installing : mysql-community-libs-5.6.42-2.el7.x86_64                                                                                                               2/5 
  Installing : mysql-community-client-5.6.42-2.el7.x86_64                                                                                                             3/5 
  Installing : mysql-community-server-5.6.42-2.el7.x86_64                                                                                                             4/5 
  Erasing    : 1:mariadb-libs-5.5.44-2.el7.centos.x86_64                                                                                                              5/5 
  Verifying  : mysql-community-libs-5.6.42-2.el7.x86_64                                                                                                               1/5 
  Verifying  : mysql-community-common-5.6.42-2.el7.x86_64                                                                                                             2/5 
  Verifying  : mysql-community-client-5.6.42-2.el7.x86_64                                                                                                             3/5 
  Verifying  : mysql-community-server-5.6.42-2.el7.x86_64                                                                                                             4/5 
  Verifying  : 1:mariadb-libs-5.5.44-2.el7.centos.x86_64                                                                                                              5/5 

Installed:
  mysql-community-libs.x86_64 0:5.6.42-2.el7                                         mysql-community-server.x86_64 0:5.6.42-2.el7                                        

Dependency Installed:
  mysql-community-client.x86_64 0:5.6.42-2.el7                                        mysql-community-common.x86_64 0:5.6.42-2.el7                                       

Replaced:
  mariadb-libs.x86_64 1:5.5.44-2.el7.centos                                                                                                                               

Complete!
[root@node101 ~]# 

[root@node101 ~]# yum -y install mysql-community-server

4>.启动mysql服务

[root@node101 ~]# systemctl enable mysqld
[root@node101 ~]# 
[root@node101 ~]# systemctl start mysqld 
[root@node101 ~]# 
[root@node101 ~]# 
[root@node101 ~]# systemctl status mysqld
● mysqld.service - MySQL Community Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2019-01-10 21:40:29 HKT; 8s ago
  Process: 10810 ExecStartPost=/usr/bin/mysql-systemd-start post (code=exited, status=0/SUCCESS)
  Process: 10730 ExecStartPre=/usr/bin/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
 Main PID: 10809 (mysqld_safe)
   CGroup: /system.slice/mysqld.service
           ├─10809 /bin/sh /usr/bin/mysqld_safe --basedir=/usr
           └─10974 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --log-error=/var/log/mysqld.log --pid-file=/var/run/my...

Jan 10 21:40:28 node101.yinzhengjie.org.cn mysql-systemd-start[10730]: PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
Jan 10 21:40:28 node101.yinzhengjie.org.cn mysql-systemd-start[10730]: To do so, start the server, then issue the following commands:
Jan 10 21:40:28 node101.yinzhengjie.org.cn mysql-systemd-start[10730]: /usr/bin/mysqladmin -u root password 'new-password'
Jan 10 21:40:28 node101.yinzhengjie.org.cn mysql-systemd-start[10730]: /usr/bin/mysqladmin -u root -h node101.yinzhengjie.org.cn password 'new-password'
Jan 10 21:40:28 node101.yinzhengjie.org.cn mysql-systemd-start[10730]: Alternatively you can run:
Jan 10 21:40:28 node101.yinzhengjie.org.cn mysql-systemd-start[10730]: /usr/bin/mysql_secure_installation
Jan 10 21:40:28 node101.yinzhengjie.org.cn mysql-systemd-start[10730]: which will also give you the option of removing the test
Jan 10 21:40:28 node101.yinzhengjie.org.cn mysqld_safe[10809]: 190110 21:40:28 mysqld_safe Logging to '/var/log/mysqld.log'.
Jan 10 21:40:28 node101.yinzhengjie.org.cn mysqld_safe[10809]: 190110 21:40:28 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
Jan 10 21:40:29 node101.yinzhengjie.org.cn systemd[1]: Started MySQL Community Server.
[root@node101 ~]# 
[root@node101 ~]# 

[root@node101 ~]# systemctl start mysqld

5>.为mysql配置管理用户

[root@node101 ~]# mysql -uroot　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　#安装后的操作系统，默认是密钥root密码的，我们直接可以用root用户登陆即可。
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.42 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> set password for 'root'@'localhost' = password('yinzhengjie');　　　　　　　　　　　　　　　　　　#修改root本地用户的登陆密码为“yinzhengjie”
Query OK, 0 rows affected (0.00 sec)

mysql> grant all privileges on *.* to 'root'@'172.30.1.10%' identified by 'yinzhengjie';　　　　　　 #我这里值允许172.30.1.100～172.30.1.109这10个服务器IP使用root用户登陆我这台mysql。其他的机器不允许。
Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　#别忘记刷新权限哟！
Query OK, 0 rows affected (0.00 sec)

mysql> quit
Bye
[root@node101 ~]#

6>.配置一个cdh超级用户权限（这个步骤如果你调过了，在初始化数据库时可能与报错哟！）

[root@node101 mysql]# systemctl start mysqld
[root@node101 mysql]# 
[root@node101 mysql]# systemctl enable mysqld
Created symlink from /etc/systemd/system/mysql.service to /usr/lib/systemd/system/mysqld.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/mysqld.service to /usr/lib/systemd/system/mysqld.service.
[root@node101 mysql]# 
[root@node101 mysql]# mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.38 MySQL Community Server (GPL)

Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> GRANT ALL PRIVILEGES ON *.* TO 'cdh'@'172.30.1.10%' IDENTIFIED BY 'yinzhengjie' WITH GRANT OPTION; 
Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON *.* TO 'cdh'@'node101.yinzhengjie.org.cn' IDENTIFIED BY 'yinzhengjie' WITH GRANT OPTION;
Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

mysql> quit
Bye
[root@node101 mysql]#

 

四.部署cloudrea manager软件

1>.安装相应的依赖包（每个节点都需要安装哟～）

[root@node101 ~]# ansible datanode -m shell -a 'yum -y install chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb'

2>.安装Cloudera Manager Server，Agent（在所有机器）

　　2.1>.CDH的选择

　　　　下载地址：http://archive.cloudera.com/cdh5/parcels/5.15.1/

 

　　2.2>.CM的选择

　　　　下载地址：http://archive.cloudera.com/cm5/cm/5/

 

　　2.3>.温馨提示：

　　　　如果网速较慢的小伙伴可以用不建议使用wget下载，推荐使用迅雷下载再上传到服务器，如果有百度云超级会员的话也可以使用我给大家体现下载好的软件。百度云链接：https://pan.baidu.com/s/148t70GenClGQRtJYX8WjvQ 密码：i2yd

[root@node101 CDH-5.15.1]# ll
total 3071760
-rw-r--r--. 1 root root 2120090032 Jan 10 18:55 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel
-rw-r--r--. 1 root root         41 Jan 10 18:54 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1
-rw-r--r--. 1 root root  838894986 Jan 10 18:55 cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz
-rw-r--r--. 1 root root  185540433 Jan 10 18:53 jdk-8u131-linux-x64.tar.gz
-rw-r--r--. 1 root root      73767 Jan 10 18:55 manifest.json
-rw-r--r--. 1 root root     855946 Jan 10 18:55 mysql-connector-java-5.1.26.jar
[root@node101 CDH-5.15.1]# 

[root@node101 CDH-5.15.1]# ansible all -m shell -a 'ls /yinzhengjie/softwares'
node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
jdk
jdk1.8.0_131

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
CDH-5.15.1
jdk
jdk1.8.0_131

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
jdk
jdk1.8.0_131

[root@node101 CDH-5.15.1]# ansible all -m copy -a 'src=/yinzhengjie/softwares/CDH-5.15.1/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz dest=/yinzhengjie/softwares/'    node101.yinzhengjie.org.cn | CHANGED => {
    "changed": true, 
    "checksum": "4227988280e0f09164bbfc6c59f291c6a814b8be", 
    "dest": "/yinzhengjie/softwares/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "ca0be616f62668915d9ceeb22bdcdd4b", 
    "mode": "0644", 
    "owner": "root", 
    "secontext": "system_u:object_r:default_t:s0", 
    "size": 838894986, 
    "src": "/root/.ansible/tmp/ansible-tmp-1547172382.51-139484190159118/source", 
    "state": "file", 
    "uid": 0
}
node103.yinzhengjie.org.cn | CHANGED => {
    "changed": true, 
    "checksum": "4227988280e0f09164bbfc6c59f291c6a814b8be", 
    "dest": "/yinzhengjie/softwares/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "ca0be616f62668915d9ceeb22bdcdd4b", 
    "mode": "0644", 
    "owner": "root", 
    "secontext": "system_u:object_r:default_t:s0", 
    "size": 838894986, 
    "src": "/root/.ansible/tmp/ansible-tmp-1547172382.57-61195539797705/source", 
    "state": "file", 
    "uid": 0
}
node102.yinzhengjie.org.cn | CHANGED => {
    "changed": true, 
    "checksum": "4227988280e0f09164bbfc6c59f291c6a814b8be", 
    "dest": "/yinzhengjie/softwares/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz", 
    "gid": 0, 
    "group": "root", 
    "md5sum": "ca0be616f62668915d9ceeb22bdcdd4b", 
    "mode": "0644", 
    "owner": "root", 
    "secontext": "system_u:object_r:default_t:s0", 
    "size": 838894986, 
    "src": "/root/.ansible/tmp/ansible-tmp-1547172382.54-248038987426177/source", 
    "state": "file", 
    "uid": 0
}
[root@node101 CDH-5.15.1]# 
[root@node101 CDH-5.15.1]# 
[root@node101 CDH-5.15.1]# 
[root@node101 CDH-5.15.1]# ansible all -m shell -a 'ls /yinzhengjie/softwares'
node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
CDH-5.15.1
cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz
jdk
jdk1.8.0_131

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz
jdk
jdk1.8.0_131

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz
jdk
jdk1.8.0_131

[root@node101 CDH-5.15.1]# 

分发Cloudera Manager到各个节点（[root@node101 CDH-5.15.1]# ansible all -m copy -a 'src=/yinzhengjie/softwares/CDH-5.15.1/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz dest=/yinzhengjie/softwares/'）

[root@node101 CDH-5.15.1]# ansible all -m shell -a 'mkdir /opt/cloudera-manager'
 [WARNING]: Consider using the file module with state=directory rather than running mkdir.  If you need to use command because file is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 CDH-5.15.1]# 

创建解压CM默认解压的目录，如果你想要使用别的解压目录别忘了修改配置文件哟（[root@node101 CDH-5.15.1]# ansible all -m shell -a 'mkdir /opt/cloudera-manager'）

[root@node101 softwares]# ansible all -m shell -a 'tar -zxf /yinzhengjie/softwares/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz -C  /opt/cloudera-manager'
 [WARNING]: Consider using the unarchive module rather than running tar.  If you need to use command because unarchive is insufficient you can add warn=False to this
command task or set command_warnings=False in ansible.cfg to get rid of this message.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 softwares]# 
[root@node101 softwares]# ansible all -m shell -a 'ls -l /opt/cloudera-manager'
node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
total 8
drwxr-xr-x. 4 1106 4001 4096 Jul 31 18:28 cloudera
drwxr-xr-x. 9 1106 4001 4096 Jul 31 18:28 cm-5.15.1

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
total 8
drwxr-xr-x. 4 1106 4001 4096 Jul 31 18:28 cloudera
drwxr-xr-x. 9 1106 4001 4096 Jul 31 18:28 cm-5.15.1

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
total 8
drwxr-xr-x. 4 1106 4001 4096 Jul 31 18:28 cloudera
drwxr-xr-x. 9 1106 4001 4096 Jul 31 18:28 cm-5.15.1

[root@node101 softwares]# 

将CM解压到我们刚刚创建的目录（[root@node101 softwares]# ansible all -m shell -a 'tar -zxf /yinzhengjie/softwares/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz -C /opt/cloudera-manager'）

[root@node101 softwares]# ansible all -m shell -a 'useradd --system --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm'
node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 softwares]# 
[root@node101 softwares]# 
[root@node101 softwares]# ansible all -m shell -a 'id cloudera-scm'                                                                                
node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
uid=986(cloudera-scm) gid=981(cloudera-scm) groups=981(cloudera-scm)

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
uid=986(cloudera-scm) gid=981(cloudera-scm) groups=981(cloudera-scm)

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
uid=986(cloudera-scm) gid=981(cloudera-scm) groups=981(cloudera-scm)

[root@node101 softwares]# 

在所有的节点上创建clodera-scm用户（[root@node101 softwares]# ansible all -m shell -a 'useradd --system --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm'）

3>.配置CM的Agent端 

[root@node101 softwares]# ansible all -m shell -a 'grep server_host /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini'
node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
server_host=localhost

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
server_host=localhost

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
server_host=localhost

[root@node101 softwares]# 
[root@node101 softwares]# ansible all -m shell -a 'sed -i 's/server_host=localhost/server_host=node101.yinzhengjie.org.cn/g' /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini'
 [WARNING]: Consider using the replace, lineinfile or template module rather than running sed.  If you need to use command because replace, lineinfile or template is
insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 softwares]# 
[root@node101 softwares]# ansible all -m shell -a 'grep server_host /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini'                                    node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
server_host=node101.yinzhengjie.org.cn

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
server_host=node101.yinzhengjie.org.cn

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
server_host=node101.yinzhengjie.org.cn

[root@node101 softwares]# 

我们指定CM的Server端([root@node101 softwares]# ansible all -m shell -a 'sed -i 's/server_host=localhost/server_host=node101.yinzhengjie.org.cn/g' /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini')

4>.初始化CM Server的数据库

[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'mkdir /usr/share/java'
 [WARNING]: Consider using the file module with state=directory rather than running mkdir.  If you need to use command because file is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 CDH-5.15.1]# 

创建java环境目录（[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'mkdir /usr/share/java'）

[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'ls -l /usr/share/java'
node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
total 0

[root@node101 CDH-5.15.1]# 
[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'mv /yinzhengjie/softwares/CDH-5.15.1/mysql-connector-java-5.1.26.jar /usr/share/java/mysql-connector-java.jar'
node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'ls -l /usr/share/java'
node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
total 836
-rw-r--r--. 1 root root 855946 Jan 10 18:55 mysql-connector-java.jar

[root@node101 CDH-5.15.1]# 

将mysql的驱动拷贝到上一步创建的java环境目录，CM数据库初始化脚本默认会去该目录查找相应的驱动（[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'mv /yinzhengjie/softwares/CDH-5.15.1/mysql-connector-java-5.1.26.jar /usr/share/java/mysql-connector-java.jar'）

[root@node101 CDH-5.15.1]# ansible namenode -m shell -a '/opt/cloudera-manager/cm-5.15.1/share/cmf/schema/scm_prepare_database.sh mysql cdh -h node101.yinzhengjie.org.cn -ucdh -pyinzhengjie --scm-host node101.yinzhengjie.org.cn scm scm scm'
node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
JAVA_HOME=/usr/java/jdk1.8
Verifying that we can write to /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-server
Creating SCM configuration file in /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-server
Executing:  /usr/java/jdk1.8/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera-manager/cm-5.15.1/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
[                          main] DbCommandExecutor              INFO  Successfully connected to database.
All done, your SCM database is configured correctly!

[root@node101 CDH-5.15.1]# 

执行数据库初始化脚本（[root@node101 CDH-5.15.1]# ansible namenode -m shell -a '/opt/cloudera-manager/cm-5.15.1/share/cmf/schema/scm_prepare_database.sh mysql cdh -h node101.yinzhengjie.org.cn -ucdh -pyinzhengjie --scm-host node101.yinzhengjie.org.cn scm scm scm' 　　　　　　　　　　　　　　#其格式为：数据库类型，数据库，数据库服务器，用户名，密码，CM服务器，后面的三个scm请不要改动！）

5>.制作CDH本地源

[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'mkdir -p /opt/cloudera/parcel-repo'
 [WARNING]: Consider using the file module with state=directory rather than running mkdir.  If you need to use command because file is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 CDH-5.15.1]# 

Server端创建Parcel目录（[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'mkdir -p /opt/cloudera/parcel-repo'）  

[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo/'
 [WARNING]: Consider using the file module with owner rather than running chown.  If you need to use command because file is insufficient you can add warn=False to this
command task or set command_warnings=False in ansible.cfg to get rid of this message.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 CDH-5.15.1]# 
[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'ls -ld  /opt/cloudera/parcel-repo/'
node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
drwxr-xr-x. 2 cloudera-scm cloudera-scm 4096 Jan 11 10:42 /opt/cloudera/parcel-repo/

[root@node101 CDH-5.15.1]# 

将上一步创建的目录所有者和所属组改为cloudera-scm用户（[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo/'）

[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'mkdir -p /opt/cloudera/parcels'              
 [WARNING]: Consider using the file module with state=directory rather than running mkdir.  If you need to use command because file is insufficient you can add
warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 CDH-5.15.1]# 
[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'ls -ld /opt/cloudera/parcels'  
node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
drwxr-xr-x. 2 root root 4096 Jan 11 10:48 /opt/cloudera/parcels

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
drwxr-xr-x. 2 root root 4096 Jan 11 10:48 /opt/cloudera/parcels

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
drwxr-xr-x. 2 root root 4096 Jan 11 10:48 /opt/cloudera/parcels

[root@node101 CDH-5.15.1]# 

agent端创建parcels目录（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'mkdir -p /opt/cloudera/parcels' ）

[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'chown cloudera-scm:cloudera-scm /opt/cloudera/parcels'
 [WARNING]: Consider using the file module with owner rather than running chown.  If you need to use command because file is insufficient you can add warn=False to this
command task or set command_warnings=False in ansible.cfg to get rid of this message.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 CDH-5.15.1]# 
[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'ls -ld /opt/cloudera/parcels'                         
node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
drwxr-xr-x. 2 cloudera-scm cloudera-scm 4096 Jan 11 10:48 /opt/cloudera/parcels

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
drwxr-xr-x. 2 cloudera-scm cloudera-scm 4096 Jan 11 10:48 /opt/cloudera/parcels

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
drwxr-xr-x. 2 cloudera-scm cloudera-scm 4096 Jan 11 10:48 /opt/cloudera/parcels

[root@node101 CDH-5.15.1]# 

别忘了把parcels目录权限授权给我们之前创建的cloudera-scm用户！（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'chown cloudera-scm:cloudera-scm /opt/cloudera/parcels'）

[root@node101 CDH-5.15.1]# 
[root@node101 CDH-5.15.1]# cp CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel /opt/cloudera/parcel-repo/
[root@node101 CDH-5.15.1]# 
[root@node101 CDH-5.15.1]# cp manifest.json /opt/cloudera/parcel-repo/　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　#别小瞧这个文件，尽管它不是很大，但是它却记录着CDH和Hadoop生态圈组件的版本依赖关系！
[root@node101 CDH-5.15.1]# 
[root@node101 CDH-5.15.1]# cp CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1 /opt/cloudera/parcel-repo/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha                    #注意，我在拷贝到时候重命名该文件了！
[root@node101 CDH-5.15.1]# 
[root@node101 CDH-5.15.1]# ll /opt/cloudera/parcel-repo/
total 2070484
-rw-r--r-- 1 root root 2120090032 Sep 13 07:40 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel
-rw-r--r-- 1 root root         41 Sep 13 07:41 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha
-rw-r--r-- 1 root root      73767 Sep 13 07:41 manifest.json
[root@node101 CDH-5.15.1]# 



温馨提示：
        如果你没有下载到“CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha”文件是，可以找到“manifest.json”文件中"parcelName": "CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel"对应的"hash": "deff00898e410a34cf0a1e66c5dbe87546608f0c"复制到该文件即可。这个方法也适用于其他的版本！

  

五.启动集群

1>.启动cloudera manager的Server 端

[root@node101 CDH-5.15.1]# ansible namenode -m shell -a '/opt/cloudera-manager/cm-5.15.1/etc/init.d/cloudera-scm-server start'             
node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
Starting cloudera-scm-server: [  OK  ]

[root@node101 CDH-5.15.1]# 
[root@node101 CDH-5.15.1]# netstat  -untalp | grep 7180
tcp        0      0 0.0.0.0:7180            0.0.0.0:*               LISTEN      30064/java          
[root@node101 CDH-5.15.1]# 

............（信息会有很多，你会看到日志中启动了很多端口，如果7180默认端口启动成功，那么你就可以访问WebUI界面了，当然这个端口咱们是可以修改的！）
2019-01-11 10:57:52,253 INFO WebServerImpl:com.cloudera.server.web.cmon.JobDetailGatekeeper: ActivityMonitor configured to allow job details for all jobs.
2019-01-11 10:57:53,956 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Discovered parcel on CM server: CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel
2019-01-11 10:57:53,957 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Created torrent file: /opt/cloudera/parcel-repo/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.torrent
2019-01-11 10:57:53,963 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Creating single-file torrent for CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel...
2019-01-11 10:57:53,965 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Hashing data from CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel with 2 threads (4044 pieces)...
2019-01-11 10:57:54,522 INFO WebServerImpl:com.cloudera.server.web.cmf.AggregatorController: AggregateSummaryScheduler started.
2019-01-11 10:57:55,568 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   ... 10% complete
2019-01-11 10:57:55,957 ERROR SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: The server storage directory [/var/lib/cloudera-scm-server] doesn't exist.
2019-01-11 10:57:55,965 ERROR SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: No read permission to the server storage directory [/var/lib/cloudera-scm-server]
2019-01-11 10:57:55,965 ERROR SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: No write permission to the server storage directory [/var/lib/cloudera-scm-server]
2019-01-11 10:57:57,025 INFO WebServerImpl:org.mortbay.log: jetty-6.1.26.cloudera.4
2019-01-11 10:57:57,030 INFO WebServerImpl:org.mortbay.log: Started SelectChannelConnector@0.0.0.0:7180
2019-01-11 10:57:57,030 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
2019-01-11 10:57:57,297 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   ... 20% complete
2019-01-11 10:57:58,330 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   ... 30% complete
2019-01-11 10:57:59,348 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   ... 40% complete
2019-01-11 10:58:00,082 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent:   ... 50% complete

服务端启动时，我们应该查看相应的日志信息（[root@node101 CDH-5.15.1]# tail -f /opt/cloudera-manager/cm-5.15.1/log/cloudera-scm-server/cloudera-scm-server.log）

2>.启动cloudera manager的Agent 端

[root@node101 CDH-5.15.1]# ansible datanode -m shell -a '/opt/cloudera-manager/cm-5.15.1/etc/init.d/cloudera-scm-agent start'     
node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
Starting cloudera-scm-agent: [  OK  ]

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
Starting cloudera-scm-agent: [  OK  ]

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
Starting cloudera-scm-agent: [  OK  ]

[root@node101 CDH-5.15.1]# 
[root@node101 CDH-5.15.1]# ansible datanode -m shell -a '/opt/cloudera-manager/cm-5.15.1/etc/init.d/cloudera-scm-agent status'
node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
cloudera-scm-agent (pid  25342) is running...

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
cloudera-scm-agent (pid  25506) is running...

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
cloudera-scm-agent (pid  31929) is running...

[root@node101 CDH-5.15.1]# 

3>.关闭防火墙和selinux

　　温馨提示：

　　　　在实际生产环境中，如果是内网环境咱们可以关闭防火墙和selinux，风向毕竟很小。而且很多公司都是有硬件防火墙的，如果咱们在开启防火墙会影响服务器性能。由于我是测试机环境，为了实验方便就直接禁用这2个服务了，实际生产中，根据你的需求选择是否开启防火墙，到时候你只需要定义一些列端口开放规则即可。

[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'systemctl status firewalld'                                          
node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2019-01-10 17:58:21 HKT; 17h ago
 Main PID: 937 (firewalld)
   CGroup: /system.slice/firewalld.service
           └─937 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid

Jan 10 17:58:19 node101.yinzhengjie.org.cn systemd[1]: Starting firewalld - dynamic firewall daemon...
Jan 10 17:58:21 node101.yinzhengjie.org.cn systemd[1]: Started firewalld - dynamic firewall daemon.

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2019-01-10 17:59:02 HKT; 17h ago
 Main PID: 952 (firewalld)
   CGroup: /system.slice/firewalld.service
           └─952 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid

Jan 10 17:59:01 node103.yinzhengjie.org.cn systemd[1]: Starting firewalld - dynamic firewall daemon...
Jan 10 17:59:02 node103.yinzhengjie.org.cn systemd[1]: Started firewalld - dynamic firewall daemon.

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2019-01-10 17:58:22 HKT; 17h ago
 Main PID: 913 (firewalld)
   CGroup: /system.slice/firewalld.service
           └─913 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid

Jan 10 17:58:21 node102.yinzhengjie.org.cn systemd[1]: Starting firewalld - dynamic firewall daemon...
Jan 10 17:58:22 node102.yinzhengjie.org.cn systemd[1]: Started firewalld - dynamic firewall daemon.

[root@node101 CDH-5.15.1]# 

检查集群的防火墙状态（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'systemctl status firewalld' ） 

[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'systemctl stop firewalld'  
node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 CDH-5.15.1]# 

停用防火墙（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'systemctl stop firewalld' ）

[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'systemctl disable firewalld'
node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.

[root@node101 CDH-5.15.1]# 

禁用防火墙开机自启动（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'systemctl disable firewalld'）

[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'setenforce 0'               
node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 CDH-5.15.1]# 

临时禁用selinux（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'setenforce 0' ）

[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'getenforce'  
node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
Permissive

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
Permissive

node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
Permissive

[root@node101 CDH-5.15.1]# 

查看selinux的状态（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'getenforce' ）

[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'sed -i 's#SELINUX=enforcing#SELINUX=disabled#' /etc/selinux/config'
 [WARNING]: Consider using the replace, lineinfile or template module rather than running sed.  If you need to use command because replace, lineinfile or template is
insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message.

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>


node103.yinzhengjie.org.cn | CHANGED | rc=0 >>


node102.yinzhengjie.org.cn | CHANGED | rc=0 >>


[root@node101 CDH-5.15.1]# 
[root@node101 CDH-5.15.1]# 
[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'cat /etc/selinux/config | grep SELINUX= | grep -v ^#'
node103.yinzhengjie.org.cn | CHANGED | rc=0 >>
SELINUX=disabled

node101.yinzhengjie.org.cn | CHANGED | rc=0 >>
SELINUX=disabled

node102.yinzhengjie.org.cn | CHANGED | rc=0 >>
SELINUX=disabled

[root@node101 CDH-5.15.1]# 

永久禁用selinux，重启操作系统后生效（[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'sed -i 's#SELINUX=enforcing#SELINUX=disabled#' /etc/selinux/config'）

4>.如下图所示，访问webUI界面（http://node101.yinzhengjie.org.cn:7180/）

 　　我们安装CM的过程通过WebUI的安装向导来进行安装，推荐使用谷歌浏览器，不要使用容易崩溃的浏览器，这样会影响你安装进度的！如果你也出现了以上界面，恭喜你CM部署成功，关于CDH的部署详情请参考：https://www.cnblogs.com/yinzhengjie/p/9638360.html。

转载于:https://www.cnblogs.com/yinzhengjie/p/10250816.html