[大数据-hadoop基础]window下编译hadoop2.7.4-eclipse mar2插件

编译hadoop2.7.4-eclipse mar2插件
准备条件:
1.将自编译的hadoop-2.7.4.tar.gz解压到D:\BigData\hadoop-2.7.4(管理员运行winRAR，否则会解压失败）。
2.下载eclipse安装包

eclipse-jee-oxygen-1a-win32-x86_64.zip
eclipse-jee-mars-2-linux-gtk-x86_64.tar.gz
3.下载hadoop2x-eclipse-plugin源代码(以zip格式下载,然后解压到一个合适的路径)
hadoop2x-eclipse-plugin
4.下载及配置Ant（网上百度大把教程）.
5.解压后的路径.

D:\BigData\hadoop-2.7.4
E:\Workspaces\Ant\hadoop2x-eclipse-plugin-master

6.修改依赖包版本文件libraries.properties

E:\Workspaces\Ant\hadoop2x-eclipse-plugin-master\src\ivy\libraries.properties

# This is the version of hadoop we are generating
hadoop.version=2.7.4
hadoop-gpl-compression.version=0.1.0

#These are the versions of our dependencies (in alphabetical order)
apacheant.version=1.7.0
ant-task.version=2.0.10

asm.version=3.2
aspectj.version=1.6.5
aspectj.version=1.6.11

checkstyle.version=4.2

commons-cli.version=1.2
commons-codec.version=1.4
commons-collections.version=3.2
commons-configuration.version=1.6
commons-daemon.version=1.0.13
commons-httpclient.version=3.0.1
commons-lang.version=2.6
commons-logging.version=1.0.4
commons-logging-api.version=1.0.4
commons-math.version=2.1
commons-el.version=1.0
commons-fileupload.version=1.2
commons-io.version=2.4
commons-net.version=3.1
core.version=3.1.1
coreplugin.version=1.3.2

# hsqldb.version=1.8.0.10
hsqldb.version=2.0.0

ivy.version=2.1.0

jasper.version=5.5.12
jackson.version=1.8.8
#not able to figureout the version of jsp & jsp-api version to get it resolved throught ivy
# but still declared here as we are going to have a local copy from the lib folder
jsp.version=2.1
jsp-api.version=5.5.12
jsp-api-2.1.version=6.1.14
jsp-2.1.version=6.1.14
jets3t.version=0.6.1
jetty.version=6.1.26
jetty-util.version=6.1.26
# jersey-core.version=1.8
# jersey-json.version=1.8
# jersey-server.version=1.8

jersey-core.version=1.9
jersey-json.version=1.9
jersey-server.version=1.9

# junit.version=4.5
junit.version=4.11

jdeb.version=0.8
jdiff.version=1.0.9

# json.version=1.0
json.version=1.1

kfs.version=0.1

# log4j.version=1.2.15
log4j.version=1.2.17

lucene-core.version=2.3.1

mockito-all.version=1.8.5
# jsch.version=0.1.42
jsch.version=0.1.54

oro.version=2.0.8

rats-lib.version=0.5.1

servlet.version=4.0.6
servlet-api.version=2.5
# slf4j-api.version=1.4.3
# slf4j-log4j12.version=1.4.3

slf4j-api.version=1.7.10
slf4j-log4j12.version=1.7.10

wagon-http.version=1.0-beta-2
xmlenc.version=0.52
# xerces.version=1.4.4
xerces.version=2.9.1

htrace.version=3.1.0-incubating

7.修改依赖包版本文件libraries.properties

E:\Workspaces\Ant\hadoop2x-eclipse-plugin-master\ivy\libraries.properties
注意:添加了一项:htrace.version=3.1.0-incubating

# This is the version of hadoop we are generating
hadoop.version=2.7.4
hadoop-gpl-compression.version=0.1.0

#These are the versions of our dependencies (in alphabetical order)
apacheant.version=1.7.0
ant-task.version=2.0.10

asm.version=3.2
aspectj.version=1.6.5
aspectj.version=1.6.11

checkstyle.version=4.2

commons-cli.version=1.2
commons-codec.version=1.4
commons-collections.version=3.2.2
commons-configuration.version=1.6
commons-daemon.version=1.0.13
commons-httpclient.version=3.0.1
commons-lang.version=2.6
commons-logging.version=1.0.4
commons-logging-api.version=1.0.4
commons-math.version=2.1
commons-el.version=1.0
commons-fileupload.version=1.2
commons-io.version=2.4
commons-net.version=3.1
core.version=3.1.1
coreplugin.version=1.3.2

# hsqldb.version=1.8.0.10
hsqldb.version=2.0.0

ivy.version=2.1.0

jasper.version=5.5.12
jackson.version=1.9.13
#not able to figureout the version of jsp & jsp-api version to get it resolved throught ivy
# but still declared here as we are going to have a local copy from the lib folder
jsp.version=2.1
jsp-api.version=5.5.12
jsp-api-2.1.version=6.1.14
jsp-2.1.version=6.1.14
jets3t.version=0.6.1
jetty.version=6.1.26
jetty-util.version=6.1.26

# jersey-core.version=1.8
# jersey-json.version=1.8
# jersey-server.version=1.8

jersey-core.version=1.9
jersey-json.version=1.9
jersey-server.version=1.9

# junit.version=4.5
junit.version=4.11

jdeb.version=0.8
jdiff.version=1.0.9

# json.version=1.0
json.version=1.1

kfs.version=0.1

# log4j.version=1.2.15
log4j.version=1.2.17

lucene-core.version=2.3.1

mockito-all.version=1.8.5
# jsch.version=0.1.42
jsch.version=0.1.54


oro.version=2.0.8

rats-lib.version=0.5.1

servlet.version=4.0.6
servlet-api.version=2.5

# slf4j-api.version=1.4.3
# slf4j-log4j12.version=1.4.3

slf4j-api.version=1.7.10
slf4j-log4j12.version=1.7.10

wagon-http.version=1.0-beta-2
xmlenc.version=0.52
# xerces.version=1.4.4
xerces.version=2.9.1

protobuf.version=2.5.0
guava.version=11.0.2
netty.version=3.6.2.Final

htrace.version=3.1.0-incubating

以上版本号：请自行与D:\BigData\hadoop-2.7.4\share\hadoop包含的版本进行核对并进行修改。

8.修改build.xml

E:\Workspaces\Ant\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin\build.xml

"${hadoop.home}/share/hadoop/common/lib/commons-httpclient-${commons-httpclient.version}.jar"  todir="${build.dir}/lib" verbose="true"/>

"${hadoop.home}/share/hadoop/common/lib/servlet-api-${servlet-api.version}.jar"  todir="${build.dir}/lib" verbose="true"/>

"${hadoop.home}/share/hadoop/common/lib/commons-io-${commons-io.version}.jar"  todir="${build.dir}/lib" verbose="true"/>

 lib/servlet-api-${servlet-api.version}.jar,
 lib/commons-io-${commons-io.version}.jar,

10.编译过程:

E:>cd E:\Workspaces\Ant\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin
 E:\Workspaces\Ant\hadoop2x-eclipse-plugin-master\src\contrib\eclipse-plugin>ant jar -Dversion=2.7.4 -Dhadoop.version=2.7.4 -Declipse.home=D:\Android\eclipse -Dhadoop.home=D:\BigData\hadoop-2.7.4 -debug

最后生成：hadoop-eclipse-plugin-2.7.4.jar

E:\Workspaces\Ant\hadoop2x-eclipse-plugin-master\build\contrib\eclipse-plugin/hadoop-eclipse-plugin-2.7.4.jar

11.可能出现的问题:（如下适用linux系统，window系统只要下载对应的jar，然后拷贝到错误提示中路径即可）。
11.1 问题1:缺少包commons-collections-3.2.1.jar

BUILD FAILED
/cloud/src/hadoop2.7/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin/build.xml:118: Warning: Could not find file /cloud/hadoop-2.7.4/share/hadoop/common/lib/commons-collections-3.2.1.jar to copy.

解决方法:
将commons-collections-3.2.1.jar下载后放入对应的目录。
commons-collections-3.2.1.jarx下载地址

11.2 问题2:缺少包slf4j-log4j12-1.7.5.jar

BUILD FAILED
/cloud/src/hadoop2.7/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin/build.xml:121: Warning: Could not find file /cloud/hadoop-2.7.4/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar to copy.

解决方法:

#cd /cloud/hadoop-2.7.4/share/hadoop/common/lib/
#http://central.maven.org/maven2/org/slf4j/slf4j-log4j12/1.7.5/slf4j-log4j12-1.7.5.jar

11.3问题3:slf4j-api-1.7.5.jar

BUILD FAILED
/cloud/src/hadoop2.7/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin/build.xml:122: Warning: Could not find file /cloud/hadoop-2.7.4/share/hadoop/common/lib/slf4j-api-1.7.5.jar to copy.

解决方法:

wget http://central.maven.org/maven2/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar

11.4 问题4:缺少htrace-core-3.0.4.jar

BUILD FAILED
/cloud/src/hadoop2.7/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin/build.xml:127: Warning: Could not find file /cloud/hadoop-2.7.4/share/hadoop/common/lib/htrace-core-3.0.4.jar to copy.

解决方法:

wget http://central.maven.org/maven2/org/htrace/htrace-core/3.0.4/htrace-core-3.0.4.jar