Ambari 2.7.3 源码构建与安装问题

目录

简介：

安装环境：

构建工具安装：

源码构建与安装： 

构建问题：

参考：

---

简介：

Apache Ambari 项目主要通过开发软件用于提供、管理和监控Hadoop集群，来使得管理Hadoop集群更简单。Ambari通过使用RESTful API 提供了一个直观的、方便使用的Hadoop管理web UI。（官网链接）

安装环境：

• Centos：7.6
• Mysql：5.7 （注意不建议使用Mysql8.0  参考链接）

构建工具安装：

由于下载的Ambari是源码（下载链接），因此需要构建后安装使用，构建过程需要安装一些工具（参考链接）。

•  JDK8(Ambari 2.6 and below can be compiled with JDK 7, from Ambari 2.7, it can be compiled with at least JDK 8)

   #安装完后查看版本信息

# java -version   

•  Apache Maven 3.3.9 or later

# mvn -v

• Python 2.7 (Ambari 2.7 or later require Python 2.7 as minimum supported version)

#  python -V

• Python setuptools:

    for Python 2.6: Download setuptools and run:

# sh setuptools-0.6c11-py2.6.egg

   for Python 2.7:  Download setuptools and run:

# sh setuptools-0.6c11-py2.7.egg

• rpmbuild (rpm-build package)

# yum install -y rpm-build

• g++ (gcc-c++ package)

# yum install -y gcc-c++

• Node.js 

    这里需要官网（链接）下载（Linux Binaries (x64)）安装，然后手动配置环境变量和路径；

# vim /etc/profile

 添加下面两行：
 export NODEJS=/***/***/node-v10.16.0-linux-x64      #解压NodeJS的路径
 export PATH=:$NODEJS/bin:$PATH

# source /etc/profile

   配置完后分别查看node 版本和npm 版本

# node -v

# npm -v

• Brunch

   使用 npm 安装 Brunch

# npm install -g brunch

   安装完后检查版本信息：

# brunch -v

• 修改 /etc/profile 文件， 设置JVM的参数，不然在编译时候很容易出现 OutOfMemoryError。 

   打开 /etc/profile （# vim  /etc/profile） 在最后一行添加如下代码：

export _JAVA_OPTIONS="-Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true"

源码构建与安装： 

参考连接

1.下载源码

选择下载镜像（链接）

# wget https://mirrors.tuna.tsinghua.edu.cn/apache/ambari/ambari-2.7.3/apache-ambari-2.7.3-src.tar.gz
# tar xfvz apache-ambari-2.7.3-src.tar.gz
# cd apache-ambari-2.7.3-src
# mvn versions:set -DnewVersion=2.7.3.0.0
 
# pushd ambari-metrics
# mvn versions:set -DnewVersion=2.7.3.0.0
# popd

2.构建源码

# mvn -B clean install rpm:rpm -DnewVersion=2.7.3.0.0 -DbuildNumber=4295bb16c439cbc8fb0e7362f19768dde1477868 -DskipTests -Dpython.ver="python >= 2.6"

注意：这个过程一般花费时间比较长，由于下载依赖包比较费时间，可能需要花费一两天，建议分模块构建，其中也会遇到很多问题，遇到具体问题建议参考链接1和链接2，这里在后面构建问题列出几个遇到的问题。

3.Ambari Server 安装

当构建成功后在 ambari-server/target/rpm/ambari-server/RPMS/noarch/ 目录或者 ambari-server/target/rpm/ambari-server/RPMS/x86_64/下会生成  ambari-server-2.7.3.0-0.x86_64.rpm 

# yum install ambari-server*.rpm

4.配置并启动Ambari Server

运行 ambari-server setup 命令 并且根据屏幕的命令提示一步步执行，配置Ambari Server, Database, JDK, LDAP和其他选项：

ambari-server setup

配置过程如下：

# ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? n
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (3): 3
Hostname (localhost): 
Port (3306): 
Database name (ambari): 
Username (ambari): 
Enter Database Password (***): 
Configuring ambari database...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
.
Ambari repo file contains latest json url http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it...
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

注意:这里需要根据提示的目录/home/my_Ambari/apache-ambari-2.7.3-src/ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql ，将此文件导入mysql（进入mysql 创建数据库后， source /home/my_Ambari/apache-ambari-2.7.3-src/ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql ），完成mysql的表的配置。

配置成功后启动 Ambari Server：

# ambari-server start

5.在所有主机安装并启动Ambari Agent 

在构建成功后，在/home/***/apache-ambari-2.7.3-src/ambari-agent/target/rpm/ambari-agent/RPMS/x86_64/  目录下会生成ambari-agent-2.7.3.0-0.x86_64.rpm 文件，将此文件拷贝到需要安装的主机，安装Ambari Agent ，

# yum install ambari-agent*.rpm

安装完成后配置主机地址，打开/etc/ambari-agent/conf/ambari-agent.ini  （vim /etc/ambari-agent/conf/ambari-agent.ini ）文件：

[server]
#配置 Ambari-Server 安装的主机
hostname=172.16.18.103    
url_port=8440
secured_url_port=8441
connect_retry_delay=10
max_reconnect_retry_delay=30

配置完后启动 ambari-agent 

# ambari-agent start

6.通过 Ambari Web UI 部署集群

在浏览器  输入 http://:8080.

输入用户名: admin 密码：admin

如果显示如下页面说明配置成功.

构建问题：

问题1.

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.12:check (default) on project ambari: Too many files with unapproved license: 1 See RAT report in: /home/***/Apache_Ambari/apache-ambari-2.7.3-src/target/rat.txt -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

解决：

mvn 命令中增加 ：-Drat.skip=true

 

问题2.

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (psutils-compile) on project ambari-metrics-host-monitoring: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...... @ 4:325 in /home/***/Apache_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-host-monitoring/target/antrun/build-psutils-compile.xml
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn -rf :ambari-metrics-host-monitoring
解决：

这个问题出现一般是因为之前下载某个软件超时，所以需要手动下载前面超时下载的jar包 ，放在下载时候指定的路径。

 

问题3.

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project ambari-server: Compilation failure
[ERROR] An unknown compilation problem occurred
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn -rf :ambari-server

解决：

这是因为 pom.xml 文件指定的编译器版本和Jvm运行版本，与安装环境安装的JAVA和JVM版本不一致，本机安装了Java版本为1.8 ，所以需要修改pom.xml 文件中相应版本一致，修改如下部分:

        org.apache.maven.plugins
        maven-compiler-plugin
        3.2
        
          1.8
          1.8
        

 

 问题4.

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:install-node-and-yarn (install node and yarn) on project ambari-web: The plugin com.github.eirslett:frontend-maven-plugin:1.4 requires Maven version 3.1.0 -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginIncompatibleException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn -rf :ambari-web

解决:

maven 版本太低，需要手动下载新版本的Maven，并配置环境变量。

 

问题5.

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (phoenix_download) on project ambari-metrics-timelineservice: An Ant BuildException has occured: java.net.SocketException: Connection reset
[ERROR] around Ant part ...... @ 5:301 in /home/***/Apache_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-timelineservice/target/antrun/build-Download Phoenix.xml
[ERROR] -> [Help 1]
解决：

这个是由于在构建ambari-metrics-timelineservice 这个模块的时候，下载phoenix-5.0.0.3.0.0.0-1634.tar.gz  超时，

一方面通过指定链接手动下载该文件 ，http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/phoenix/phoenix-5.0.0.3.0.0.0-1634.tar.gz，并按照要求修改名字后（phoenix.tar.gz）放在 /home/***/Apache_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/  路径下。

另一方面修改mvn库的镜像（参考5）：

编辑mvn的 settings.xml文件，一般在 ~/.m2/settings.xml 目录或者，mvn 安装目录下 home/Maven/apache-maven-3.6.1/conf/settings.xml ，添加镜像http://repo.hortonworks.com/content/groups/public/  和 http://nexus-private.hortonworks.com/nexus/content/groups/public 。

         
            
               public
               *
               http://nexus-private.hortonworks.com/nexus/content/groups/public
            
         

 

问题6.

[INFO] Downloading http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/hbase/hbase-2.0.0.3.0.0.0-1634-bin.tar.gz  to /home/my_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/hbase.tar.gz

 

[get] Getting: http://dev.hortonworks.com.s3.amazonaws.com/HDP/centos7/3.x/BUILDS/3.0.0.0-1634/tars/hbase/hbase-2.0.0.3.0.0.0-1634-bin.tar.gz 
[get] To: /home/my_Ambari/apache-ambari-2.7.3-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/hbase.tar.gz

解决：

这和问题五一样，是由于下载文件比慢，卡在这里，解决方法是手动去指定连接下载文件，然后修改名字后放在指定目录下。

 

参考：

1. https://www.cnblogs.com/barneywill/p/10264135.html

2. https://blog.csdn.net/ZhouyuanLinli/article/details/88963289

3. https://blog.csdn.net/slx_2011/article/details/45038089

4.https://blog.csdn.net/guoyachuan01/article/details/39500575

5.https://community.hortonworks.com/questions/157149/apache-ambari-error-during-maven-build-missing-ins.html