SQOOP2 源代码build

Downloading source code

Sqoop project is using git as a revision control system hosted at Apache Software Foundation. You can clone entire repository using following command:

git clone https://git-wip-us.apache.org/repos/asf/sqoop.git sqoop2
Sqoop2 is currently developed in special branch sqoop2 that you need to check out after clone:


cd sqoop2
git checkout sqoop2
Building project


You can use usual maven targets like compile or package to build the project. Sqoop supports two major Hadoop revisions at the moment - 1.x and 2.x. As compiled code for one Hadoop major version can’t be used on another, you must compile Sqoop against appropriate Hadoop version. You can change the target Hadoop version by specifying -Dhadoop.profile=$hadoopVersion on the maven command line. Possible values of $hadoopVersions are 100 and 200 for Hadoop version 1.x and 2.x respectively. Sqoop will compile against Hadoop 2 by default. Following example will compile Sqoop against Hadoop 1.x:


mvn compile -Dhadoop.profile=100
Maven target package can be used to create Sqoop packages similar to the ones that are officially available for download. Sqoop will build only source tarball by default. You need to specify -Pbinary to build binary distribution. You might need to explicitly specify Hadoop version if the default is not accurate.


mvn package -Pbinary
Running tests


Sqoop supports two different sets of tests. First smaller and much faster set is called unit tests and will be executed on maven target test. Second larger set of integration tests will be executed on maven target integration-test. Please note that integration tests might require manual steps for installing various JDBC drivers into your local maven cache.


Example for running unit tests:


mvn test
Example for running integration tests:


mvn integration-test

app/prog/hadoop/sqoop-1.99.3-bin-hadoop200/dist/target

 cp sqoop-1.99.3.tar.gz /app/prog/hadoop/

sqoop-1.99.3.tar.gz

tar -xvf sqoop-<version>-bin-hadoop<hadoop-version>.tar.gz


Tests run: 25, Failures: 0, Errors: 25, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Sqoop ............................................. SUCCESS [0.724s]
[INFO] Sqoop Common ...................................... SUCCESS [11.495s]
[INFO] Sqoop SPI ......................................... SUCCESS [0.152s]
[INFO] Sqoop Core ........................................ SUCCESS [2.035s]
[INFO] Sqoop Repository .................................. SUCCESS [0.020s]
[INFO] Sqoop Derby Repository ............................ SUCCESS [46.436s]
[INFO] Sqoop Execution Engines ........................... SUCCESS [0.019s]
[INFO] Sqoop Mapreduce Execution Engine .................. SUCCESS [27.791s]
[INFO] Sqoop Submission Engines .......................... SUCCESS [0.017s]
[INFO] Sqoop Mapreduce Submission Engine ................. SUCCESS [0.228s]
[INFO] Sqoop Connectors .................................. SUCCESS [0.015s]
[INFO] Sqoop Connector SDK ............................... SUCCESS [0.075s]
[INFO] Sqoop Generic JDBC Connector ...................... SUCCESS [18.117s]
[INFO] Sqoop Server ...................................... SUCCESS [0.805s]
[INFO] Sqoop Client ...................................... SUCCESS [0.605s]
[INFO] Sqoop Shell ....................................... SUCCESS [0.684s]
[INFO] Sqoop Distribution ................................ SUCCESS [2.102s]
[INFO] Sqoop Integration Tests ........................... FAILURE [3:13.571s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5:05.473s
[INFO] Finished at: Thu Jul 31 10:16:30 CST 2014
[INFO] Final Memory: 28M/952M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12:test (integration-test) on project test: There are test failures.
[ERROR]
[ERROR] Please refer to /app/prog/hadoop-2.4.1/sqoop-1.99.3-bin-hadoop200/test/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :test
[root@n1 sqoop-1.99.3-bin-hadoop200]# cd ..

如果出现以上红色错误，只需要把pom.xml文件里面的<doc> 项目注释掉既可以。