hadoop-2.9.2 window 源码编译记录

依赖项

linux安装的请转 https://blog.csdn.net/miracle_8/article/details/105111982
参考hadoop-2.9.2-src/BUILDING.txt

* Windows System
* JDK 1.8  
* Maven 3.0 or later 
* ProtocolBuffer 2.5.0  
* CMake 3.1 or newer   path
* Visual Studio 2010 Professional or Higher  (2010已经足够了)
* Windows SDK 8.1 (if building CPU rate control for the container executor) OK
* zlib headers (if building native code bindings for zlib) zlib_home 
* Internet connection for first build (to fetch all Maven and Hadoop dependencies)
* Unix command-line tools from GnuWin32: sh, mkdir, rm, cp, tar, gzip. These
  tools must be present on your PATH.  git替代 path
* Python ( for generation of docs using 'mvn site')

Unix command-line tools are also included with the Windows Git package which
can be downloaded from http://git-scm.com/downloads

If using Visual Studio, it must be Professional level or higher.
Do not use Visual Studio Express.  It does not support compiling for 64-bit,
which is problematic if running a 64-bit system.

The Windows SDK 8.1 is available to download at:

http://msdn.microsoft.com/en-us/windows/bg162891.aspx

Cygwin is not required.

----------------------------------------------------------------------------------
Building:

Keep the source code tree in a short path to avoid running into problems related
to Windows maximum path length limitation (for example, C:\hdc).

There is one support command file located in dev-support called win-paths-eg.cmd.
It should be copied somewhere convenient and modified to fit your needs.

win-paths-eg.cmd sets up the environment for use. You will need to modify this
file. It will put all of the required components in the command path,
configure the bit-ness of the build, and set several optional components.

Several tests require that the user must have the Create Symbolic Links
privilege.

All Maven goals are the same as described above with the exception that
native code is built by enabling the 'native-win' Maven profile. -Pnative-win
is enabled by default when building on Windows since the native components
are required (not optional) on Windows.

If native code bindings for zlib are required, then the zlib headers must be
deployed on the build machine. Set the ZLIB_HOME environment variable to the
directory containing the headers.

set ZLIB_HOME=C:\zlib-1.2.7

At runtime, zlib1.dll must be accessible on the PATH. Hadoop has been tested
with zlib 1.2.7, built using Visual Studio 2010 out of contrib\vstudio\vc10 in
the zlib 1.2.7 source tree.

http://www.zlib.net/

----------------------------------------------------------------------------------
Building distributions:

 * Build distribution with native code    : mvn package [-Pdist][-Pdocs][-Psrc][-Dtar][-Dmaven.javadoc.skip=true]

依赖项说明：

1. jdk ： 必须1.8（其他版本没有测试过），安装后需要将bin目录加入到PATH中
2. Maven ： 必须3.0（其他版本没有测试过），安装后需要将bin目录加入到PATH中 https://maven.apache.org/download.cgi
3. ProtocolBuffer ： 必须2.5（其他版本没有测试过）。需要下载的分别是protobuf-2.5.0和protoc-2.5.0-win32，需要将protoc-2.5.0-win32中的protoc.exe加入到protobuf-2.5.0/src目录。至于protobuf-2.5.0/java 目录下的内容是否需要执行mvn install 没测试过，反正本人是执行过的
4. CMake ： 必须是3.1（其他版本没有测试过），安装后需要将bin目录加入到PATH中 https://cmake.org/download/
5. Visual Studio ： 2010足够了，不需要安装高版本 https://www.malavida.com/en/soft/visual-studio-2010/
6. Windows SDK ：8.1足够了，不需要安装高版本
7. zlib-header： 编译 native.sln的时候需要ZLIB_HOME变量，需要设置变量ZLIB_HOME=zlib-header所在路径。在运行的时候，需要将 zlib1.dll加入到path中（At runtime, zlib1.dll must be accessible on the PATH）
8. Unix command-line tools：可以用git bash 替代，安装后需要将bin目录加入到PATH中

安装命令

必须通过 visual studio 2010 command prompt 运行如下命令， 因为编译过程中需要执行msbuild命令

mvn clean compile package -Pdist,native-win -DskipTests -Dmaven.javadoc.skip=true
mvn package -Pdist -DskipTests -Dmaven.javadoc.skip=true
# 单个模块编译的话  可以直接执行如下命令
mvn clean compile -Pdist -DskipTests -Dmaven.javadoc.skip=true
# 如果遇到编译失败并且提示msbuild命令不存在时，需要通过 visual studio 2010 command prompt  运行上面命令
# 如果在编译的过程中出现任何异常，可以通过mvn -X *** > 1.log来打印mvn命令的详细信息来定位问题所在 !!!

安装过程中出现的问题

1. （模块 : Project Dist POM ）提示bash dist-copynativelibs ****
   原因： 可能是因为安装的有问题，bash命令无效，需要将bash替换成sh
   解决：将超父类的pom.xml配置文件中的shell-executable由bash改为sh 即可 。hadoop-2.9.2-src/pom.xml
   

2. （模块： Common Project）提示native.h中找不到zlib.h文件（如果遵从上文的依赖，应该是不会出现的）
   原因: zlib.h头文件信息读取不到，环境变量ZLIB_HOME未配置
   解决: 配置ZLIB_HOME=zlib的头文件

3. Could not find artifact com.amazonaws:DynamoDBLocal:jar:1.11.86 ，jar包找不到
   原因：配置的maven 镜像中找不到此包 本人配置的镜像是阿里云 (https://maven.aliyun.com/nexus/content/groups/public/)
   解决：可以手动导入，或者采用其他可用的镜像

<repository>
     <id>dynamodb-local-oregon</id>
     <name>DynamoDB Local Release Repository</name>
     <url>https://s3-us-west-2.amazonaws.com/dynamodb-local/release</url>
</repository>
// 配置过此repository之后，会去下载jar包，但是可能还会提示错误，需要将本地仓库中对应包目录下的_remote.repositories 文件删除，之后才会去读取本地的jar包文件，否则还是会去镜像中查找的

注：

1. 本人3.1.2也编译过，但是最后的时候因为3.1.2中的bash版本过高，导致最后会提示dist-tools-hooks-maker这个文件 语法错误，虽然能解决，但是懒得改，于是就没有调整了
   2