Ambari2.5.2在CentOS7上的源码编译

Ambari编译准备

• 详细Ambari编译参考文档参考：https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Development

maven环境搭建

• 参考博客：https://blog.csdn.net/Happy_Sunshine_Boy/article/details/91375219

nodejs的安装

[root@gaia tarpackage]# wget https://nodejs.org/dist/v8.11.3/node-v8.11.3-linux-x64.tar.xz
[root@gaia tarpackage]# tar -xvf node-v8.11.3-linux-x64.tar.xz -C /opt/
[root@gaia tarpackage]# cd /opt/node-v8.11.3-linux-x64/
[root@gaia node-v8.11.3-linux-x64]# vim /etc/profile
export NODE_HOME=/opt/node-v8.11.3-linux-x64
export PATH=$PATH:$NODE_HOME/bin
[root@gaia node-v8.11.3-linux-x64]# source /etc/profile
[root@gaia node-v8.11.3-linux-x64]# node -v
v8.11.3
[root@gaia node-v8.11.3-linux-x64]# npm -v
5.6.0

Ambari源码下载

• 下载ambari2.5.2版本：
• https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.2.0/bk_ambari-installation/content/ambari_repositories.html
• http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.5.2.0/ambari-2.5.2.0-centos7.tar.gz

Ambari源码编译

• 使用hortonworks官网下载的ambari进行编译，解压ambari源码

[root@gaia tarpackage]# tar zxvf ambari-2.5.2.0-centos7.tar.gz
[root@gaia tarpackage]# mv ambari/centos7/tars/ambari/ambari-2.5.0.1.2.5.2.0-298-source.tar.gz ./
[root@gaia tarpackage]# tar -zxvf ambari-2.5.0.1.2.5.2.0-298-source.tar.gz
[root@gaia tarpackage]# cd ambari-2.5.0.1.2.5.2.0-298

替换Ambari-metrics较大的包

• 启动httpd服务
• 上传tar包
  
• 替换pom.xml路径

[root@gaia ambari-2.5.0.1.2.5.2.0-298]# cd ambari-metrics/
[root@gaia ambari-metrics]# vim pom.xml

打版本号

mvn versions:set -DnewVersion=2.5.2.0.0

• 第一次对ambari进行编译，需要给ambari确定版本号后进行编译。
• ambari版本号为5位（四位数会因正则表达式检测不匹配报错），如：2.5.2.0.0

[root@gaia ambari-2.5.0.1.2.5.2.0-298]# mvn versions:set -DnewVersion=2.5.2.0.0
……
[INFO] Searching for local aggregator root...
[INFO] Local aggregation root: /root/tarpackage/ambari-2.5.0.1.2.5.2.0-298
[INFO] Processing change of org.apache.ambari:ambari:2.5.2.0.0 -> 2.5.2.0.0
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Ambari Main ........................................ SUCCESS [01:09 min]
[INFO] Apache Ambari Project POM .......................... SKIPPED
[INFO] Ambari Web ......................................... SKIPPED
[INFO] Ambari Views ....................................... SKIPPED
[INFO] Ambari Admin View .................................. SKIPPED
[INFO] utility ............................................ SKIPPED
[INFO] ambari-metrics ..................................... SKIPPED
[INFO] Ambari Metrics Common .............................. SKIPPED
[INFO] Ambari Metrics Hadoop Sink ......................... SKIPPED
[INFO] Ambari Metrics Flume Sink .......................... SKIPPED
[INFO] Ambari Metrics Kafka Sink .......................... SKIPPED
[INFO] Ambari Metrics Storm Sink .......................... SKIPPED
[INFO] Ambari Metrics Storm Sink (Legacy) ................. SKIPPED
[INFO] Ambari Metrics Collector ........................... SKIPPED
[INFO] Ambari Metrics Monitor ............................. SKIPPED
[INFO] Ambari Metrics Grafana ............................. SKIPPED
[INFO] Ambari Metrics Assembly ............................ SKIPPED
[INFO] Ambari Server ...................................... SKIPPED
[INFO] Ambari Functional Tests ............................ SKIPPED
[INFO] Ambari Agent ....................................... SKIPPED
[INFO] Ambari Client ...................................... SKIPPED
[INFO] Ambari Python Client ............................... SKIPPED
[INFO] Ambari Groovy Client ............................... SKIPPED
[INFO] Ambari Shell ....................................... SKIPPED
[INFO] Ambari Python Shell ................................ SKIPPED
[INFO] Ambari Groovy Shell ................................ SKIPPED
[INFO] ambari-logsearch ................................... SKIPPED
[INFO] Ambari Logsearch Appender .......................... SKIPPED
[INFO] Ambari Logsearch Portal ............................ SKIPPED
[INFO] Ambari Logsearch Log Feeder ........................ SKIPPED
[INFO] Ambari Logsearch Solr Client ....................... SKIPPED
[INFO] Ambari Infra Solr Plugin ........................... SKIPPED
[INFO] Ambari Logsearch Assembly .......................... SKIPPED
[INFO] Ambari Logsearch Integration Test .................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:27 min
[INFO] Finished at: 2019-06-17T15:10:51+08:00
[INFO] Final Memory: 27M/280M
[INFO] ------------------------------------------------------------------------

编译

mvn -B install package rpm:rpm -DnewVersion=2.5.2.0.0 -DskipTests -Dbuild-rpm -Dpython.ver="python >= 2.6" -Drat.skip=true
[root@gaia ambari-2.5.0.1.2.5.2.0-298]# mvn -B install package rpm:rpm -DnewVersion=2.5.2.0.0 -DskipTests -Dbuild-rpm -Dpython.ver=“python >= 2.6” -Drat.skip=true

编译问题解决

• 问题1
  [ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:install-node-and-yarn (install node and yarn) on project ambari-web: Could not extract the Yarn archive: Could not extract archive: ‘/root/.m2/repository/com/github/eirslett/yarn/0.23.2/yarn-0.23.2./yarn-v0.23.2.tar.gz’: EOFException -> [Help 1]
  [ERROR]
  [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
  [ERROR] Re-run Maven using the -X switch to enable full debug logging.
  [ERROR]
  [ERROR] For more information about the errors and possible solutions, please read the following articles:
  [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
  [ERROR]
  [ERROR] After correcting the problems, you can resume the build with the command
  [ERROR] mvn -rf :ambari-web
• 解决1

[root@gaia tarpackage]# wget https://github.com/yarnpkg/yarn/releases/download/v0.23.2/yarn-v0.23.2.tar.gz
[root@gaia tarpackage]# mkdir -p /root/.m2/repository/com/github/eirslett/yarn/0.23.2/yarn-0.23.2./
[root@gaia tarpackage]# cp ./yarn-v0.23.2.tar.gz /root/.m2/repository/com/github/eirslett/yarn/0.23.2/yarn-0.23.2./

• 问题2
  [ERROR] warning [email protected]: The platform “linux” is incompatible with this module.
  [INFO] info “[email protected]” is an optional dependency and failed compatibility check. Excluding it from installation.
  [ERROR] warning [email protected]: The platform “linux” is incompatible with this module.
  [INFO] info “[email protected]” is an optional dependency and failed compatibility check. Excluding it from installation.
• 解决2

[root@gaia ambari-2.5.0.1.2.5.2.0-298]# npm install pm2@latest -g
[root@gaia ambari-2.5.0.1.2.5.2.0-298]# pm2 install pm2-slack

• 问题3
  [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (psutils-compile) on project ambari-metrics-host-monitoring: An Ant BuildException has occured: exec returned: 1
  [ERROR] around Ant part …… @ 4:303 in /root/tarpackage/ambari-2.5.0.1.2.5.2.0-298/ambari-metrics/ambari-metrics-host-monitoring/target/antrun/build-psutils-compile.xml
  [ERROR] -> [Help 1]
  [ERROR]
  [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
  [ERROR] Re-run Maven using the -X switch to enable full debug logging.
  [ERROR]
  [ERROR] For more information about the errors and possible solutions, please read the following articles:
  [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
  [ERROR]
  [ERROR] After correcting the problems, you can resume the build with the command
  [ERROR] mvn -rf :ambari-metrics-host-monitoring
• 解决3

yum install python-devel

• 问题4
  
• 解决4
  • 重新执行编译命令
• 问题5
  
• 解决5

wget --no-check-certificate https://pypi.python.org/packages/source/s/setuptools/setuptools-12.0.3.tar.gz
tar -zxvf setuptools-12.0.3.tar.gz
cd setuptools-12.0.3
python setup.py install

• 问题6
  
• 解决6
  • 版本号不对，重新建版本号
• 问题7
  
• 解决7
  • 编辑vim ambari-metrics/ambari-metrics-storm-sink/pom.xml
    
• 问题8
  [ERROR] Failed to execute goal org.codehaus.mojo:xml-maven-plugin:1.0:transform (default) on project ambari-server: Failed to transform input file /root/tarpackage/ambari-2.5.0.1.2.5.2.0-298/ambari-server/target/findbugs/findbugsXml.html: The entity name must immediately follow the ‘&’ in the entity reference. -> [Help 1]
  [ERROR]
  [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
  [ERROR] Re-run Maven using the -X switch to enable full debug logging.
  [ERROR]
  [ERROR] For more information about the errors and possible solutions, please read the following articles:
  [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
  [ERROR]
  [ERROR] After correcting the problems, you can resume the build with the command
  [ERROR] mvn -rf :ambari-server
• 解决8
  • 方法1：删除历史文件：rm -rf /mnt/apache-ambari-2.5.2-src/ambari-server/target/findbugs/findbugsXml.*
  • 方法2：重新执行
  • 方法3：如果上述两种方法都没有办法解决，如果网络不好的话，很难通过，那只能修改pom.xml文件，将文件中的关于findbugs的部分注释掉

[root@gaia ambari-2.5.0.1.2.5.2.0-298]# vim ambari-server/pom.xml   # 找到所有findbugs地方注释掉

• 问题9
  • 无法安装hdp-select或者其他存在于hdp(util)的rpm包
  • 针对RPM包的repo文件过期了
• 解决9
  • 重新运行create repo文件，更新hdp(util)的链接
• 问题10
  [ERROR] Failed to execute goal org.codehaus.mojo:rpm-maven-plugin:2.1.4:rpm (default-cli) on project ambari-server: RPM build execution returned: ‘1’ executing ‘/bin/sh -c cd ‘/root/tarpackage/ambari-2.5.0.1.2.5.2.0-298/ambari-server/target/rpm/ambari-server/SPECS’ && ‘rpmbuild’ ‘-bb’ ‘–target’ ‘x86_64-redhat-linux’ ‘–buildroot’ ‘/root/tarpackage/ambari-2.5.0.1.2.5.2.0-298/ambari-server/target/rpm/ambari-server/buildroot’ ‘–define’ ‘_topdir /root/tarpackage/ambari-2.5.0.1.2.5.2.0-298/ambari-server/target/rpm/ambari-server’ ‘–define’ ‘_build_name_fmt %%{ARCH}/%%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm’ ‘–define’ ‘_builddir %{_topdir}/BUILD’ ‘–define’ ‘_rpmdir %{_topdir}/RPMS’ ‘–define’ ‘_sourcedir %{_topdir}/SOURCES’ ‘–define’ ‘_specdir %{_topdir}/SPECS’ ‘–define’ ‘_srcrpmdir %{_topdir}/SRPMS’ ‘ambari-server.spec’’ -> [Help 1]
  [ERROR]
  [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
  [ERROR] Re-run Maven using the -X switch to enable full debug logging.
  [ERROR]
  [ERROR] For more information about the errors and possible solutions, please read the following articles:
  [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
  [ERROR]
  [ERROR] After correcting the problems, you can resume the build with the command
  [ERROR] mvn -rf :ambari-server
• 解决10

# 查看python
[root@gaia bin]# ll | grep python
lrwxrwxrwx.   1 root root           7 Jun 17 19:14 python -> python2
lrwxrwxrwx.   1 root root           9 Jun 17 19:14 python2 -> python2.7
-rwxr-xr-x.   1 root root        7216 Apr  9 22:31 python2.7
-rwxr-xr-x.   1 root root        1835 Apr  9 22:31 python2.7-config
lrwxrwxrwx.   1 root root          16 Jun 17 19:14 python2-config -> python2.7-config
lrwxrwxrwx.   1 root root          14 Jun 17 19:14 python-config -> python2-config

# 修改 /usr/lib/rpm/brp-python-bytecompile 指定python路径
[root@gaia bin]# vim  /usr/lib/rpm/brp-python-bytecompile			# 修改第39行
python_binary=/usr/bin/$(basename $python_libdir)
修改为
python_binary=/usr/bin/python