【Hadoop】hortonworks/hadoop-release-HDP-3.1.5.1-2-tag build log

源码包下载地址

源码地址：https://github.com/hortonworks/hadoop-release/releases/tag/HDP-3.1.5.1-2-tag

buid 命令

tar zxvf hadoop-release-HDP-3.1.5.1-2-tag.tar.gz
cd hadoop-release-HDP-3.1.5.1-2-tag
mvn -B package -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true

buid 过程问题及解决办法

1. 添加 hdp repo

问题描述：
一般遇到Could not resolve dependencies

1. 首先考虑是网络原因，就根据日志中依赖地址去打开看这个，依赖是不是存在，如果存在，考虑是网络原因，重新执行命令或者手动下载放到指定位置
2. 如果发现依赖仓库中确实没有这个包，就只是表是目前项目中配置的仓库地址里都没有，所以需要我们添加仓库
   比如，build hadoop-auth的时候报org.apache.zookeeper:zookeeper:jar:3.4.6.3.1.5.1-2找不到，仓库中也确实没有
   所以需要我们添加hdp的仓库才行

   [ERROR] Failed to execute goal on project hadoop-auth: Could not resolve dependencies for project org.apache.hadoop:hadoop-auth:jar:3.1.1.3.1.5.1-2: Could not find artifact org.apache.zookeeper:zookeeper:jar:3.4.6.3.1.5.1-2 in apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots) -> [Help 1]
   org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal on project hadoop-auth: Could not resolve dependencies for project org.apache.hadoop:hadoop-auth:jar:3.1.1.3.1.5.1-2: Could not find artifact org.apache.zookeeper:zookeeper:jar:3.4.6.3.1.5.1-2 in apache.snapshots.https (https://repository.apache.org/content/repositories/snapshots)
   

解决办法：
在pom中添加

<repository>
  <id>jettyid>
  <name>jetty Repositoryname>
  <url>http://repo.hortonworks.com/content/groups/public/url>
  <releases>
      <enabled>trueenabled>
      <updatePolicy>dailyupdatePolicy>
  releases>
  <snapshots>
      <enabled>falseenabled>
      <checksumPolicy>warnchecksumPolicy>
  snapshots>
    <layout>defaultlayout>
repository>

2. 清除本地maven repo中的lastUpdated

删除命令: find ~/.m2/repository/ -name "*.lastUpdated" -exec rm -rf {} \;

3. SSL peer shut down incorrectly

可以在mvn 命令后追加参数-Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2试一下

4. 安装protocbuf

报错信息：
根据 执行protoc --version没有返回版本信息，所以需要我们安装protocbuf

[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.1.1.3.1.5.1-2:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1]

## brew install protobuf 默认安装最新版本的3.7.1 版本不对还是会报错的
##所以需要重新安装brew reinstall [email protected],brew 无法安装2.5.0
## https://github.com/protocolbuffers/protobuf/releases/tag/v2.5.0下载安装吧
## mac安装protobuf: https://blog.csdn.net/jiajane/article/details/104307998
[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.1.1.3.1.5.1-2:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: protoc version is 'libprotoc 3.7.1', expected version is '2.5.0' -> [Help 1]