ambari本地源自动化安装

ambari本地源自动化安装hortonworks hadoop

环境配置

Hostname	IP	OS
Ambari	192.168.174.174	Centos X64 base install
Maste.hadoop	192.168.174.175	Centos X64 base install
Datanode1.hadoop	192.168.174.176	Centos X64 base install

操作系统建议以software server 方式安装，base 方式安装后，需要安装Development tools，本文以base举例。

一：制作ambari，hadoop本地源

首先需要有一台能够联网的linux机器

1：网络源制作ambari（1.4.1.25）本地源

wget http://public-repo-1.hortonworks.com/ambari/centos6/1.x/updates/1.4.1.25/ambari.repo

cp ambari.repo /etc/yum.repos.d/

将网络源中的包同步到本地：

reposync  -r HDP-UTILS-1.1.0.16

reposync  -r Updates-ambari-1.4.1.25

reposync  -r ambari-1.x

会得到相应的三个目录，每个目录中有相应的rpm包

准备一台http服务器，本文以172.16.235.164作为http服务器发布安装源

mkdir /var/www/html/ambari

将 HDP-UTILS-1.1.0.16 Updates-ambari-1.4.1.25 ambari-1.x复制到 /var/www/html/ambari下

cd /var/www/html/ambari

createrepo ./    #没有createrepo命令自己去安装

这样ambari安装源就做好了，下载好jdk-6u31-linux-x64.bin备用

  

2：准备hortonworks hadoop （HDP）2.0.6本地源

准备HDP网络源

vim /etc/yum.repos.d/HDP.repo，添加以下内容

[HDP-2.0.6]

name=HDP

baseurl=http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.0.6.0

enabled=1

gpgcheck=0

同步网络源到本地：

reposync  -r HDP-2.0.6  #将近1G

讲 HDP-2.0.6 复制到http server的/var/www/html/hdp下

cd /var/www/html/hdp

createrepo ./  

这样hadoop的本地源就做好了

  

3：制作系统本地源（安装ambari和hadoop时，可能会依赖些系统包，这时需要有系统的本地源来安装）

http server 挂载系统光盘

mount /dev/cdrom  /var/www/html/centos/

如果已经本地已经挂载无需操作

4：JDK安装

tar -zxvf jdk-7u55-linux-x64.gz 安装jdk包

export JAVA_HOME=/jdk/jdk1.7.0_55 环境变量添加

export PATH=$JAVA_HOME/bin:$PATH 执行变量

source /etc/profile 配置立即生效 

[root@wxl02 jdk]# java –version 查看java版本

java version "1.7.0_55"

Java(TM) SE Runtime Environment (build 1.7.0_55-b13)

Java HotSpot(TM) 64-Bit Server VM (build 24.55-b03, mixed mode)

 

 

二：安装ambari

系统准备：关闭iptables、selinux,启动ntpd服务

[root@ambari ]vim /etc/hosts

192.168.174.174  ambari

192.168.174.175  master.hadoop

192.168.174.176  datanode1.hadoop

配置与所有主机的无密码登入

[root@wxl02 .ssh]# ssh-keygen -t rsa #一路回车（所有主机都要执行）

Generating public/private rsa key pair.

Enter file in which to save the key (/root/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /root/.ssh/id_rsa.

Your public key has been saved in /root/.ssh/id_rsa.pub.

The key fingerprint is:

7a:9c:b2:af:27:ad:10:56:b3:d8:1d:f3:37:77:96:0b root@wxl02

The key's randomart image is:

+--[ RSA 2048]----+

| |

| |

| o o |

| + + + .|

| + o S . oE..o|

| . . o . . o.o.|

| . o.+ . |

| ..+o |

| +*. |

+-----------------+

[root@wxl02 .ssh]# ssh-copy-id -i /root/.ssh/id_rsa.pub root@wxl02

root@wxl02's password:

Now try logging into the machine, with "ssh 'root@wxl02'", and check in:

.ssh/authorized keys

to make sure we haven't added extra keys that you weren't expecting.

[root@wxl02 .ssh]# ssh wxl02

Last login: Mon Aug 1 01:57:44 2016 from wxl01

[root@wxl02 ~]# exit

logout

Connection to wxl02 closed.

 

本地安装与配置ambari服务：

[root@ambari ]vim /etc/yum.repos.d/ambari.repo  添加以下内容

[ambari-1.x]

name=Ambari 1.x

baseurl=file:///var/www/html/ambari/

gpgcheck=0

enabled=1

 

[HDP-UTILS-1.1.0.16]

name=Hortonworks Data Platform Utils Version - HDP-UTILS-1.1.0.16

baseurl=file:///var/www/html/ambari/

gpgcheck=0

enabled=1

 

[Updates-ambari-1.4.1.25]

name=ambari-1.4.1.25 - Updates

gpgcheck=0

baseurl=file:///var/www/html/ambari/

enabled=1

[root@ambari ]vim /etc/yum.repos.d/centos.repo  添加以下内容

[centos]

name = centos

baseurl=file:///var/www/html/centos/

enabled=1

gpgcheck=0

[root@ambari ]vim /etc/yum.repos.d/HDP.repo 添加以下内容

[HDP-2.0.6]

name=HDP

baseurl=file:///var/www/html/hdp

path=/

enabled=1

gpgcheck=0

4.执行安装ambari：

[root@ambari yum.repos.d]# yum -y install ambari-server

Loaded plugins: fastestmirror, security

Repository 'Updates-ambari-1.4.1.25': Error parsing config: Error parsing "gpgcheck = ''": invalid boolean value

Repository 'centos' is missing name in configuration, using id

Loading mirror speeds from cached hostfile

Setting up Install Process

Resolving Dependencies

--> Running transaction check

---> Package ambari-server.noarch 0:1.4.1.25-1 will be installed

--> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server-1.4.1.25-1.noarch

--> Running transaction check

---> Package postgresql-server.x86_64 0:8.4.13-1.el6_3 will be installed

--> Processing Dependency: postgresql-libs(x86-64) = 8.4.13-1.el6_3 for package: postgresql-server-8.4.13-1.el6_3.x86_64

--> Processing Dependency: postgresql(x86-64) = 8.4.13-1.el6_3 for package: postgresql-server-8.4.13-1.el6_3.x86_64

--> Processing Dependency: libpq.so.5()(64bit) for package: postgresql-server-8.4.13-1.el6_3.x86_64

--> Running transaction check

---> Package postgresql.x86_64 0:8.4.13-1.el6_3 will be installed

---> Package postgresql-libs.x86_64 0:8.4.13-1.el6_3 will be installed

--> Finished Dependency Resolution

  

Dependencies Resolved

  

====================================================================================================================================

 Package                           Arch                   Version                          Repository                          Size

====================================================================================================================================

Installing:

 ambari-server                     noarch                 1.4.1.25-1                       HDP-UTILS-1.1.0.16                  37 M

Installing for dependencies:

 postgresql                        x86_64                 8.4.13-1.el6_3                   centos                             2.8 M

 postgresql-libs                   x86_64                 8.4.13-1.el6_3                   centos                             200 k

 postgresql-server                 x86_64                 8.4.13-1.el6_3                   centos                             3.4 M

  

Transaction Summary

====================================================================================================================================

Install       4 Package(s)

  

Total download size: 43 M

Installed size: 88 M

Downloading Packages:

(1/4): ambari-server-1.4.1.25-1.noarch.rpm                                                                   |  37 MB     00:00    

(2/4): postgresql-server-8.4.13-1.el6_3.x86_64.rpm                                                           | 3.4 MB     00:00    

(3/4): postgresql-libs-8.4.13-1.el6_3.x86_64.rpm                                                             | 200 kB     00:00    

(4/4): postgresql-8.4.13-1.el6_3.x86_64.rpm                                                                  | 2.8 MB     00:00    

------------------------------------------------------------------------------------------------------------------------------------

Total                                                                                                43 MB/s |  43 MB     00:00    

Running rpm_check_debug

Running Transaction Test

Transaction Test Succeeded

Running Transaction

  Installing : postgresql-libs-8.4.13-1.el6_3.x86_64                                                                            1/4

  Installing : postgresql-8.4.13-1.el6_3.x86_64                                                                                 2/4

  Installing : postgresql-server-8.4.13-1.el6_3.x86_64                                                                          3/4

  Installing : ambari-server-1.4.1.25-1.noarch                                                                                  4/4

  Verifying  : postgresql-server-8.4.13-1.el6_3.x86_64                                                                          1/4

  Verifying  : postgresql-libs-8.4.13-1.el6_3.x86_64                                                                            2/4

  Verifying  : ambari-server-1.4.1.25-1.noarch                                                                                  3/4

  Verifying  : postgresql-8.4.13-1.el6_3.x86_64                                                                                 4/4

  

Installed:

  ambari-server.noarch 0:1.4.1.25-1                                                                                                 

  

Dependency Installed:

  postgresql.x86_64 0:8.4.13-1.el6_3     postgresql-libs.x86_64 0:8.4.13-1.el6_3     postgresql-server.x86_64 0:8.4.13-1.el6_3   

  

Complete!

  

5.开始配置ambari 服务：

首先上传jdk文件

[root@ambari ]cd /var/lib/ambari-server/resources  将jdk-6u31-linux-x64.bin上传到该目录

[root@ambari resources]# chmod +x jdk-6u31-linux-x64.bin

[root@ambari yum.repos.d]# ambari-server setup

Using python  /usr/bin/python2.6

Initializing...

Setup ambari-server

Checking SELinux...

SELinux status is 'enabled'

SELinux mode is 'permissive'

WARNING: SELinux is set to 'permissive' mode and temporarily disabled.

OK to continue [y/n] (y)?

Customize user account for ambari-server daemon [y/n] (n)?

Adjusting ambari-server permissions and ownership...

Checking iptables...

WARNING: iptables is running. Confirm the necessary Ambari ports are accessible. Refer to the Ambari documentation for more details on ports.

OK to continue [y/n] (y)?

Checking JDK...

JDK already exists, using /var/lib/ambari-server/resources/jdk-6u31-linux-x64.bin

Installing JDK to /usr/jdk64

Successfully installed JDK to /usr/jdk64/jdk1.6.0_31

Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-6.zip to /var/lib/ambari-server/resources/jce_policy-6.zip

To download the JCE Policy files you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.htmlNot accepting will result in errors when configuring Kerberos security.

Do you accept the Oracle Binary Code License Agreement [y/n] (y)?

Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/jce_policy-6.zip

Completing setup...

Configuring database...

Enter advanced database configuration [y/n] (n)?

Default properties detected. Using built-in database.

Checking PostgreSQL...

Running initdb: This may take upto a minute.

About to start PostgreSQL

Configuring local database...

Connecting to the database. Attempt 1...

Configuring PostgreSQL...

Restarting PostgreSQL

Ambari Server 'setup' completed successfully.

[root@ambari ~]# ambari-server start

Using python  /usr/bin/python2.6

Starting ambari-server

Ambari Server running with 'root' privileges.

Server PID at: /var/run/ambari-server/ambari-server.pid

Server out at: /var/log/ambari-server/ambari-server.out

Server log at: /var/log/ambari-server/ambari-server.log

Ambari Server 'start' completed successfully.

到此ambari安装好

  

三：本地安装hadoop

给ambari配置本地hadoo源地址：

[root@ambari ~]# cd /var/lib/ambari-server/resources/stacks/HDPLocal/2.0.6/repos/

[root@ambari repos]# vim repoinfo.xml  修改红色部分为本地源地址

 注 意：

  网络源地址：

      http:// 192.168.174.174/hdp/

      HDP-2.0.6

      HDP

  本地源地址：

  

这样在ambari执行hadoop安装时，会将本地源地址配置到所有主机上，在为所有主机安装ambari-agent时，会将ambari server下的/etc/yum.repo/ambari.repo文件复制到所有主机

  

主机准备（master.hadoop、datanode1.hadoop）配置相同，只举例master.hadoop

关闭iptables、selinux、启动ntpd服务（略）

[root@master ~]# vim /etc/hosts 添加以下

192.168.174.174  ambari

192.168.174.175  master.hadoop

192.168.174.176  datanode1.hadoop

[root@master ~]# vim /etc/yum.repos.d/centos.repo 添加以下内容

[centos]

baseurl=http://172.16.235.164/centos/

enabled=1

gpgcheck=0

@master ~]# yum -y groupinstall "Development tools"

  

登入ambari管理页面

http://192.168.174.174:8080/ 账户：admin 密码：admin

配置集群名称

选择HDP版本

选择目标主机，一行一个主机。要让ambar server自动在所以主机安装ambari-agent，需要提供ambari server root账户的私钥，即/root/.ssh/ id_rsa文件，选中使用本地源安装hadoop，然后下一步

确认并且选择主机

确认OK

8.下一步选择要安装的组件，选好后下一步

 

9.选择组件分配在哪些主机，都是master 的服务

10.选择安装client，主机有限，master同时安装client

11.配置服务，其中nagios中要填写密码及报警接受邮箱必填，其他选择性配置

12.显示配置信息

13.开始安装，过程稍微有点长

14.安装过程

 

15.Google浏览器反应比较慢，用另一某浏览器已经显示succes了

16.Ganglia：

17.nagios：