hadoop-2.5.1 源码编译

hadoop-2.5.1编译步骤(只需一个节点即可) 环境： 系统   CentOS 6.4 64 位   m1      172.103.55.12 软件：  jdk1.7、hadoop- 2.5.1- src.tar.gz、maven3.2.3、protobuf2.5.0 注：需要连接互联网   1.       安装配置JDK（略） 2.       安装基本应用程序 (需要连接互联网网)  yum -y install  svn   ncurses-devel   gcc*                                              yum -y install lzo-devel zlib-devel autoconf    automake    libtool    cmake     openssl –devel 3.      安装 maven 下载 apache-maven-3.2.3-bin.tar.gz    并解压 tar   zxvf   apache- maven- 3.2.3- bin.tar.gz 配置环境变量 export MAVEN_HOME=/usr/local/program/maven/ apache- maven- 3.2.3 export PATH=$PATH:$MAVEN_HOME/bin        添加之后执行sourc /etc/profile 4.     安装 protobuf（不安装，编译将无法完成） hadoop使用protocol buffer进行通信，需要下载和安装protobhf-2.5.0.tar.gz；由于官网已经无法下载了， 可以到百度云盘下载http://pan.baidu.com/s/1pJlZubT 编译安装 protobuf ①  cd   protobuf - 2.5.0    ②  ./configure  ③  make ④  make install 检测  protoc –version   5.     编译 hadoop-2.5.1.src a)     下载  hadoop-2.5.1-src.tat.gz  源码包下载地址  http://apache.claz.org/hadoop/common/hadoop-2.5.1/  红色标志  hadoop-2.5.1- src.tar.gz 为源码包   b)    解压hadoop-2.5.1-src.tar.gz   c)     开始编译 进入hadoop-2.5.1.src 目录： 执行命令   mvn clean package –Pdist,native –DskipTests –Dtar  （或者 mvn package -Pdist,native -DskipTests -Dtar）       此步骤要求逐级能访问公网，并且此步骤比较漫长，请耐心等待。   经过漫长的等待后，终于看到了Building SUCCESS: [INFO] Reactor Summary: [INFO]  [INFO] Apache Hadoop Main ................................. SUCCESS [  4.389 s] [INFO] Apache Hadoop Project POM .......................... SUCCESS [  3.163 s] [INFO] Apache Hadoop Annotations .......................... SUCCESS [  6.377 s] [INFO] Apache Hadoop Assemblies ........................... SUCCESS [  0.562 s] [INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [  3.136 s] [INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [  6.757 s] [INFO] Apache Hadoop MiniKDC .............................. SUCCESS [  5.506 s] [INFO] Apache Hadoop Auth ................................. SUCCESS [  6.269 s] [INFO] Apache Hadoop Auth Examples ........................ SUCCESS [  4.341 s] [INFO] Apache Hadoop Common ............................... SUCCESS [03:25 min] [INFO] Apache Hadoop NFS .................................. SUCCESS [ 23.998 s] [INFO] Apache Hadoop Common Project ....................... SUCCESS [  0.160 s] [INFO] Apache Hadoop HDFS ................................. SUCCESS [07:52 min] [INFO] Apache Hadoop HttpFS ............................... SUCCESS [11:03 min] [INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [03:23 min] [INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [ 14.324 s] [INFO] Apache Hadoop HDFS Project ......................... SUCCESS [  0.343 s] [INFO] hadoop-yarn ........................................ SUCCESS [  0.092 s] [INFO] hadoop-yarn-api .................................... SUCCESS [02:46 min] [INFO] hadoop-yarn-common ................................. SUCCESS [02:37 min] [INFO] hadoop-yarn-server ................................. SUCCESS [  1.142 s] [INFO] hadoop-yarn-server-common .......................... SUCCESS [01:02 min] [INFO] hadoop-yarn-server-nodemanager ..................... SUCCESS [02:26 min] [INFO] hadoop-yarn-server-web-proxy ....................... SUCCESS [ 22.258 s] [INFO] hadoop-yarn-server-applicationhistoryservice ....... SUCCESS [ 16.444 s] [INFO] hadoop-yarn-server-resourcemanager ................. SUCCESS [ 51.768 s] [INFO] hadoop-yarn-server-tests ........................... SUCCESS [  9.864 s] [INFO] hadoop-yarn-client ................................. SUCCESS [ 16.057 s] [INFO] hadoop-yarn-applications ........................... SUCCESS [  0.162 s] [INFO] hadoop-yarn-applications-distributedshell .......... SUCCESS [  7.307 s] [INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SUCCESS [  3.243 s] [INFO] hadoop-yarn-site ................................... SUCCESS [  0.099 s] [INFO] hadoop-yarn-project ................................ SUCCESS [ 18.997 s] [INFO] hadoop-mapreduce-client ............................ SUCCESS [  2.306 s] [INFO] hadoop-mapreduce-client-core ....................... SUCCESS [01:34 min] [INFO] hadoop-mapreduce-client-common ..................... SUCCESS [01:05 min] [INFO] hadoop-mapreduce-client-shuffle .................... SUCCESS [ 20.109 s] [INFO] hadoop-mapreduce-client-app ........................ SUCCESS [ 22.691 s] [INFO] hadoop-mapreduce-client-hs ......................... SUCCESS [ 30.368 s] [INFO] hadoop-mapreduce-client-jobclient .................. SUCCESS [01:30 min] [INFO] hadoop-mapreduce-client-hs-plugins ................. SUCCESS [ 19.958 s] [INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [ 24.097 s] [INFO] hadoop-mapreduce ................................... SUCCESS [ 17.757 s] [INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [01:59 min] [INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [ 48.464 s] [INFO] Apache Hadoop Archives ............................. SUCCESS [ 14.187 s] [INFO] Apache Hadoop Rumen ................................ SUCCESS [ 15.098 s] [INFO] Apache Hadoop Gridmix .............................. SUCCESS [ 13.630 s] [INFO] Apache Hadoop Data Join ............................ SUCCESS [  8.080 s] [INFO] Apache Hadoop Extras ............................... SUCCESS [  6.496 s] [INFO] Apache Hadoop Pipes ................................ SUCCESS [ 23.855 s] [INFO] Apache Hadoop OpenStack support .................... SUCCESS [ 17.886 s] [INFO] Apache Hadoop Client ............................... SUCCESS [ 18.728 s] [INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [  1.060 s] [INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [ 35.296 s] [INFO] Apache Hadoop Tools Dist ........................... SUCCESS [ 13.476 s] [INFO] Apache Hadoop Tools ................................ SUCCESS [  0.134 s] [INFO] Apache Hadoop Distribution ......................... SUCCESS [01:20 min] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 51:55 min [INFO] Finished at: 2014-12-09T19:16:49-08:00 [INFO] Final Memory: 142M/350M [INFO] ------------------------------------------------------------------------ 编译成功后，  /hadoop-dist/target/hadoop-2.5.1.tar.gz   就是我们需要的文件了；

 可能遇到的问题：

报错[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) on project hadoop-hdfs: Fatal error compiling: Error while executing the compiler. InvocationTargetException: Java heap space

分配内存不足,参考如下为maven配置JVM参数: export MAVEN_OPTS=”-Xms256m -Xmx512m -Djava.awt.headless=true”

我直接在profile文件中：export MAVEN_OPTS=-Xmx1024