win7系统进行hadoop 2.3 源代码编译

阅读更多

1、从官网上下载源代码，并解压；
2、下载maven2.3.1版本（2.0.3版本也执行过没问题）
3、到%hadoop-2.3.0-src%/hadoop-maven-plugins/目录下， 执行
mvn clean install
4、在有些公司或网络下，可能mvn install命令会执行失败，报maven的资源库无法访问，我碰到的是会自动连到report1那个上面，这个可以修改%MAVEN_HOME%/conf/settings.xml，修改中的内容，如下：

 
   
      mirrorId
      repositoryId
      Human Readable Name for this Mirror.
      http://repo.maven.apache.org/maven2///这个url如果不可用，可以使用其它能用的替换
   
 
5、如果有的公司上网是通过proxy，则需要继续修改同一个文件的,例如：
 
   
      optional
      true
      http
      proxyuser
      proxypass
      proxy.host.net
      80
      local.net|some.host.com
   
 
6、到这里，基本上mvn clean install这个命令可以执行完了；然后回退到上一个目录，也就是cd %hadoop-2.3.0-src%/,执行：
mvn eclipse:eclipse -DskipTests
也即是跳过测试了
7、上面命令执行过程中，一般会出现一个问题：
[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.3.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR][ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn -rf :hadoop-common

8、上面这个错误，可以看出是由于没有找到proto引起的：org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1]
9、win7下，可以直接从网上下载protoc-2.5.0-win32.zip，win7 64位也可以执行，我用的就是64位操作系统。然后设置环境变量，进行加载即可，也可以用一个偷懒的方法：
把protoc.exe编译器，放入program files/protoc/自定的文件夹下，将这个目录追加入path环境变量。这样mvn就能调用protoc.exe，并且你自己也可以在cmd中随意执行。
10、重新执行mvn eclipse:eclipse -DskipTests
11、等到的最终执行完的结果：
[INFO] Apache Hadoop Main ................................ SUCCESS [  1.222 s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [  1.376 s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [  0.263 s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [  0.189 s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [  0.214 s]
[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [  2.718 s]
[INFO] Apache Hadoop MiniKDC ............................. SUCCESS [  2.666 s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [  1.948 s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [  0.191 s]
[INFO] Apache Hadoop Common .............................. SUCCESS [  5.259 s]
[INFO] Apache Hadoop NFS ................................. SUCCESS [01:15 min]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [  0.132 s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [01:55 min]
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [ 25.091 s]
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [ 15.946 s]
[INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [  4.452 s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [  0.497 s]
[INFO] hadoop-yarn ....................................... SUCCESS [  0.150 s]
[INFO] hadoop-yarn-api ................................... SUCCESS [  2.251 s]
[INFO] hadoop-yarn-common ................................ SUCCESS [ 26.932 s]
[INFO] hadoop-yarn-server ................................ SUCCESS [  0.143 s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [  8.565 s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [ 51.856 s]
[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [  2.283 s]
[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [  4.427 s]
[INFO] hadoop-yarn-server-tests .......................... SUCCESS [  9.880 s]
[INFO] hadoop-yarn-client ................................ SUCCESS [  2.852 s]
[INFO] hadoop-yarn-applications .......................... SUCCESS [  0.112 s]
[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [  3.241 s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [  1.021 s]
[INFO] hadoop-yarn-site .................................. SUCCESS [  0.099 s]
[INFO] hadoop-yarn-project ............................... SUCCESS [  0.151 s]
[INFO] hadoop-mapreduce-client ........................... SUCCESS [  0.263 s]
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [ 25.972 s]
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [  8.810 s]
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [  1.110 s]
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [  6.729 s]
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [  8.809 s]
[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [ 10.474 s]
[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [  0.708 s]
[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [ 18.896 s]
[INFO] hadoop-mapreduce .................................. SUCCESS [  0.199 s]
[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [  8.281 s]
[INFO] Apache Hadoop Distributed Copy .................... SUCCESS [03:25 min]
[INFO] Apache Hadoop Archives ............................ SUCCESS [  2.256 s]
[INFO] Apache Hadoop Rumen ............................... SUCCESS [  1.381 s]
[INFO] Apache Hadoop Gridmix ............................. SUCCESS [  4.527 s]
[INFO] Apache Hadoop Data Join ........................... SUCCESS [  0.865 s]
[INFO] Apache Hadoop Extras .............................. SUCCESS [  0.690 s]
[INFO] Apache Hadoop Pipes ............................... SUCCESS [  0.139 s]
[INFO] Apache Hadoop OpenStack support ................... SUCCESS [  1.876 s]
[INFO] Apache Hadoop Client .............................. SUCCESS [  2.828 s]
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [  1.942 s]
[INFO] Apache Hadoop Scheduler Load Simulator ............ SUCCESS [ 14.240 s]
[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [  7.795 s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [  0.123 s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [  0.520 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11:45 min
[INFO] Finished at: 2014-04-05T12:52:52+08:00
[INFO] Final Memory: 79M/307M
[INFO] ------------------------------------------------------------------------
D:\Download\bigdata\hadoop-2.3.0-src>

12、最后在Eclipse中按照以下流程导入源代码：“File” →  “Import”→  “Existing Projects into Workspace”。（我用的是eclipse-java-kepler）
13，代码导入后，可能会出现一些java类不存在，例如：EchoResponseProto.java/EmptyResponseProto,主要是测试类，可以通过google解决。