【Hadoop三】Centos7_x86_64部署Hadoop集群之编译Hadoop源代码

 

编译必需的软件

• Firebugs3.0.0
• Maven3.2.3
• Ant
• JDK1.7.0_67
• protobuf-2.5.0
• Hadoop 2.5.2源码包

 

 

 

• Firebugs3.0.0

 

http://sourceforge.jp/projects/sfnet_findbugs/downloads/findbugs/3.0.0/findbugs-3.0.0.tar.gz/
tar xzvf findbugs-3.0.0
export FINDBUGS_HOME=/home/hadoop/software/findbugs-3.0.0
export PATH=$FINDBUGS_HOME/bin:$PATH

 

第一次构件时没有安装FireBugs，结果报如下错误

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (site) on project hadoop-common: An Ant BuildException has occured: stylesheet /home/grid/hadoop-2.4.1/release-2.4.1/hadoop-common-project/hadoop-common/${env.FINDBUGS_HOME}/src/xsl/default.xsl doesn't exist.
[ERROR] around Ant part ...<xslt style="${env.FINDBUGS_HOME}/src/xsl/default.xsl" in="/home/grid/hadoop-2.4.1/release-2.4.1/hadoop-common-project/hadoop-common/target/findbugsXml.xml" out="/home/grid/hadoop-2.4.1/release-2.4.1/hadoop-common-project/hadoop-common/target/site/findbugs.html"/>... @ 44:267 in /home/grid/hadoop-2.4.1/release-2.4.1/hadoop-common-project/hadoop-common/target/antrun/build-main.xml
[ERROR] -> [Help 1]

 

 

• Maven3.2.3

 

export M2_HOME=/home/hadoop/software/apache-maven-3.2.3
export PATH=$M2_HOME/bin:$PATH

 

• Ant

export ANT_HOME=/home/hadoop/software/apache-ant-1.9.4
export PATH=$ANT_HOME/bin:$PATH

 

• JDK1.7.0_67

 

export JAVA_HOME=/home/hadoop/software/jdk1.7.0_67
export PATH=$JAVA_HOME/bin:$PATH

 

• protobuf-2.5.0

 

tar zxvf protobuf-2.5.0.tar.gz  
cd protobuf-2.5.0  
./configure --prefix=/usr/local  
make 
make install 

   

• Hadoop 2.5.2源码包

tar zxvf hadoop-2.5.2-src.tar.gz  
cd hadoop-2.5.2-src  
mvn package -Pdist,native,docs -DskipTests -Dtar

• 编译结果

Maven构建大概持续1个小时，构建的结果应该显示所有的构建都正确，如下所示：

 

[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop Main ................................. SUCCESS [  3.460 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [  2.440 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [  8.027 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [  0.502 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [  6.744 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [  7.694 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [ 10.779 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [01:08 min]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [  3.241 s]
[INFO] Apache Hadoop Common ............................... SUCCESS [07:05 min]
[INFO] Apache Hadoop NFS .................................. SUCCESS [ 11.771 s]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [  0.039 s]
[INFO] Apache Hadoop HDFS ................................. SUCCESS [14:51 min]
[INFO] Apache Hadoop HttpFS ............................... SUCCESS [05:01 min]
[INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [01:45 min]
[INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [  5.854 s]
[INFO] Apache Hadoop HDFS Project ......................... SUCCESS [  0.049 s]
[INFO] hadoop-yarn ........................................ SUCCESS [  0.066 s]
[INFO] hadoop-yarn-api .................................... SUCCESS [01:24 min]
[INFO] hadoop-yarn-common ................................. SUCCESS [01:32 min]
[INFO] hadoop-yarn-server ................................. SUCCESS [  0.096 s]
[INFO] hadoop-yarn-server-common .......................... SUCCESS [ 45.766 s]
[INFO] hadoop-yarn-server-nodemanager ..................... SUCCESS [01:45 min]
[INFO] hadoop-yarn-server-web-proxy ....................... SUCCESS [  4.768 s]
[INFO] hadoop-yarn-server-applicationhistoryservice ....... SUCCESS [  9.864 s]
[INFO] hadoop-yarn-server-resourcemanager ................. SUCCESS [ 23.242 s]
[INFO] hadoop-yarn-server-tests ........................... SUCCESS [  0.779 s]
[INFO] hadoop-yarn-client ................................. SUCCESS [  8.364 s]
[INFO] hadoop-yarn-applications ........................... SUCCESS [  0.068 s]
[INFO] hadoop-yarn-applications-distributedshell .......... SUCCESS [  4.872 s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SUCCESS [  2.523 s]
[INFO] hadoop-yarn-site ................................... SUCCESS [  0.051 s]
[INFO] hadoop-yarn-project ................................ SUCCESS [  9.001 s]
[INFO] hadoop-mapreduce-client ............................ SUCCESS [  0.124 s]
[INFO] hadoop-mapreduce-client-core ....................... SUCCESS [ 35.116 s]
[INFO] hadoop-mapreduce-client-common ..................... SUCCESS [ 27.584 s]
[INFO] hadoop-mapreduce-client-shuffle .................... SUCCESS [  7.655 s]
[INFO] hadoop-mapreduce-client-app ........................ SUCCESS [ 17.608 s]
[INFO] hadoop-mapreduce-client-hs ......................... SUCCESS [ 15.425 s]
[INFO] hadoop-mapreduce-client-jobclient .................. SUCCESS [ 39.538 s]
[INFO] hadoop-mapreduce-client-hs-plugins ................. SUCCESS [  2.738 s]
[INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [  9.863 s]
[INFO] hadoop-mapreduce ................................... SUCCESS [  4.919 s]
[INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [ 19.115 s]
[INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [ 12.291 s]
[INFO] Apache Hadoop Archives ............................. SUCCESS [  3.078 s]
[INFO] Apache Hadoop Rumen ................................ SUCCESS [ 11.125 s]
[INFO] Apache Hadoop Gridmix .............................. SUCCESS [  7.794 s]
[INFO] Apache Hadoop Data Join ............................ SUCCESS [  6.021 s]
[INFO] Apache Hadoop Extras ............................... SUCCESS [  5.012 s]
[INFO] Apache Hadoop Pipes ................................ SUCCESS [ 10.999 s]
[INFO] Apache Hadoop OpenStack support .................... SUCCESS [  8.789 s]
[INFO] Apache Hadoop Client ............................... SUCCESS [ 10.152 s]
[INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [  0.165 s]
[INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [ 18.100 s]
[INFO] Apache Hadoop Tools Dist ........................... SUCCESS [  6.974 s]
[INFO] Apache Hadoop Tools ................................ SUCCESS [  0.028 s]
[INFO] Apache Hadoop Distribution ......................... SUCCESS [02:37 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 44:48 min
[INFO] Finished at: 2014-12-03T08:12:48-05:00
[INFO] Final Memory: 144M/483M
[INFO] ------------------------------------------------------------------------

 

 

编译得到的安装包信息是

/home/hadoop/software/hadoop-2.5.2-src/hadoop-dist/target/hadoop-2.5.2.tar.gz