【成功】maven编译 apache falcon0.9源码_RedHat6.4

背景

最近一直在调查数据分析以及集群的软件,接下来,又开始了BI的调查了,今天就收到邮件,让调查apache falcon。各种环境折腾又开始了。

先理解一下Falcon的架构:

【成功】maven编译 apache falcon0.9源码_RedHat6.4_第1张图片


事前准备

  1. apache falcon源码  (本次使用的是最新的apache-falcon-0.9-sources.tar.gz)
  2. jdk(1.7/1.8)
  3. maven(3.X)  apache-maven-3.0.3-bin.tar.gz 【后续提示出错,换成maven3.1.0】


安装

JDK

  1. JDK的安装,此处就不再说明了,很简单,装完后记得配置/etc/profile里JAVA_HOME,完后source一下就OK了,确认如下:

[root@sv004 project]# java -version
java version "1.7.0_67"
Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)


maven

  1. maven的安装,本人第一次,也很简单,解压,配置MAVEN_BIN,然后使用mvn -v确认是否安装。效果如下所示:

[root@sv004 apache-maven-3.0.3]# ./bin/mvn -v
Apache Maven 3.0.3 (r1075438; 2011-03-01 02:31:09+0900)
Maven home: /home/project/apache-maven-3.0.3
Java version: 1.7.0_67, vendor: Oracle Corporation
Java home: /usr/java/jdk1.7.0_67/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-431.el6.x86_64", arch: "amd64", family: "unix"

本次的重点来了

使用maven编译falcon源码

1.解压falcon源码

tar -xzvf apache-falcon-${project.version}-source.tar.gz
cd falcon-sources-${project.version}

2.编译falcon

export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=256m -noverify" && mvn clean install

但此时报错了

[INFO] Scanning for projects...
Downloading: http://repo1.maven.org/maven2/org/apache/apache/17/apache-17.pom
Downloading: https://raw.github.com/InMobi/mvn-repo/master/releases/org/apache/apache/17/apache-17.pom
Downloading: http://repo.hortonworks.com/content/repositories/releases/org/apache/apache/17/apache-17.pom
Downloading: https://repository.apache.org/content/groups/snapshots/org/apache/apache/17/apache-17.pom
Downloading: https://repository.apache.org/content/groups/public/org/apache/apache/17/apache-17.pom
Downloading: https://maven.java.net/content/groups/public/org/apache/apache/17/apache-17.pom
Downloading: https://repository.jboss.org/nexus/content/groups/public/org/apache/apache/17/apache-17.pom
Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/apache/17/apache-17.pom
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project org.apache.falcon:falcon-main:0.9 (/home/project/apache-maven-3.0.3/falcon-sources-0.9/pom.xml) has 1 error
[ERROR]     Non-resolvable parent POM: Could not transfer artifact org.apache:apache:pom:17 from/to central (http://repo1.maven.org/maven2): Error transferring file: repo1.maven.org and 'parent.relativePath' points at wrong local POM @ line 21, column 13: Unknown host repo1.maven.org -> [Help 2]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException


原因

根据Unkown host信息调查了一下,应该是代理设置的有问题,我的maven使用的是默认的,没有设置,坑就被我踩到了。

参考URL:http://stackoverflow.com/questions/15334394/could-not-transfer-artifact-org-apache-maven-pluginsmaven-surefire-pluginpom2/15334627#15334627


解决策

修改我的maven/conf/setting.xml,如下所示:

  
     
      true
      http
      ***.***.***.***  ⇒代理的IP地址
      8080
      maven
    
  

完后重新执行

export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=256m -noverify" && mvn clean install
第一次时间比较长,取决你的网速,因为要下载好多依赖包文件。


好事多磨呀.....

Exception while executing SCM command.
[INFO] Storing buildScmBranch: UNKNOWN_BRANCH
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ falcon-ui ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Falcon ..................................... SUCCESS [18:56.122s]
[INFO] Apache Falcon UI .................................. FAILURE [3.810s]
[INFO] Checkstyle ........................................ SKIPPED
[INFO] Build Tools ....................................... SKIPPED
[INFO] Apache Falcon Java client ......................... SKIPPED
[INFO] Apache Falcon CLI client .......................... SKIPPED
[INFO] Apache Falcon Metrics ............................. SKIPPED
[INFO] Apache Falcon Hadoop Dependencies ................. SKIPPED
[INFO] Apache Falcon Test Utility ........................ SKIPPED
[INFO] Apache Falcon Commons ............................. SKIPPED
[INFO] Apache Falcon Oozie EL Extension .................. SKIPPED
[INFO] Apache Falcon Embedded Hadoop - Test Cluster ...... SKIPPED
[INFO] Apache Falcon Sharelib Hive - Test Cluster ........ SKIPPED
[INFO] Apache Falcon Sharelib Pig - Test Cluster ......... SKIPPED
[INFO] Apache Falcon Sharelib Hcatalog - Test Cluster .... SKIPPED
[INFO] Apache Falcon Sharelib Oozie - Test Cluster ....... SKIPPED
[INFO] Apache Falcon Test Tools - Test Cluster ........... SKIPPED
[INFO] Apache Falcon Messaging ........................... SKIPPED
[INFO] Apache Falcon LIfecycle Module .................... SKIPPED
[INFO] Apache Falcon Oozie Adaptor ....................... SKIPPED
[INFO] Apache Falcon Scheduler ........................... SKIPPED
[INFO] Apache Falcon Acquisition ......................... SKIPPED
[INFO] Apache Falcon Distcp Replication .................. SKIPPED
[INFO] Apache Falcon Retention ........................... SKIPPED
[INFO] Apache Falcon Archival ............................ SKIPPED
[INFO] Apache Falcon Rerun ............................... SKIPPED
[INFO] Apache Falcon Prism ............................... SKIPPED
[INFO] falcon-unit ....................................... SKIPPED
[INFO] Apache Falcon Web Application ..................... SKIPPED
[INFO] Apache Falcon Documentation ....................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 19:04.642s
[INFO] Finished at: Thu Apr 14 02:38:36 JST 2016
[INFO] Final Memory: 18M/237M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:0.0.23:install-node-and-npm (install node and npm) on project falcon-ui: The plugin com.github.eirslett:frontend-maven-plugin:0.0.23 requires Maven version 3.1.0 -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginIncompatibleException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :falcon-ui

根据报错信息来看,maven的版本与下载的插件的版本不一致导致的,根据提示信息,遂决定使用maven3.1.0。

构建好Maven3.1.0的环境后,重新再次编译falcon源码,结果悲剧了,install-node-and-npm无法下载.......

[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Falcon ..................................... SUCCESS [2.380s]
[INFO] Apache Falcon UI .................................. FAILURE [1:02.489s]
[INFO] Checkstyle ........................................ SKIPPED
[INFO] Build Tools ....................................... SKIPPED
[INFO] Apache Falcon Java client ......................... SKIPPED
[INFO] Apache Falcon CLI client .......................... SKIPPED
[INFO] Apache Falcon Metrics ............................. SKIPPED
[INFO] Apache Falcon Hadoop Dependencies ................. SKIPPED
[INFO] Apache Falcon Test Utility ........................ SKIPPED
[INFO] Apache Falcon Commons ............................. SKIPPED
[INFO] Apache Falcon Oozie EL Extension .................. SKIPPED
[INFO] Apache Falcon Embedded Hadoop - Test Cluster ...... SKIPPED
[INFO] Apache Falcon Sharelib Hive - Test Cluster ........ SKIPPED
[INFO] Apache Falcon Sharelib Pig - Test Cluster ......... SKIPPED
[INFO] Apache Falcon Sharelib Hcatalog - Test Cluster .... SKIPPED
[INFO] Apache Falcon Sharelib Oozie - Test Cluster ....... SKIPPED
[INFO] Apache Falcon Test Tools - Test Cluster ........... SKIPPED
[INFO] Apache Falcon Messaging ........................... SKIPPED
[INFO] Apache Falcon LIfecycle Module .................... SKIPPED
[INFO] Apache Falcon Oozie Adaptor ....................... SKIPPED
[INFO] Apache Falcon Scheduler ........................... SKIPPED
[INFO] Apache Falcon Acquisition ......................... SKIPPED
[INFO] Apache Falcon Distcp Replication .................. SKIPPED
[INFO] Apache Falcon Retention ........................... SKIPPED
[INFO] Apache Falcon Archival ............................ SKIPPED
[INFO] Apache Falcon Rerun ............................... SKIPPED
[INFO] Apache Falcon Prism ............................... SKIPPED
[INFO] falcon-unit ....................................... SKIPPED
[INFO] Apache Falcon Web Application ..................... SKIPPED
[INFO] Apache Falcon Documentation ....................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:05.883s
[INFO] Finished at: Thu Apr 14 20:57:47 JST 2016
[INFO] Final Memory: 30M/303M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:0.0.23:install-node-and-npm (install node and npm) on project falcon-ui: Could not extract the Node archive: Could not extract archive: '/home/project/apache-maven-3.1.0/falcon-sources-0.9/falcon-ui/node_tmp/node.tar.gz': EOFException -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :falcon-ui

调查了好久,最终来发现,原来是磁盘没有空间了,刚开始还以为是公司的网速的问题【PS:我们公司的网太烂了,刷个页面要好久】


清空一部分空间出来后,又出幺蛾子了.....

[ERROR] npm WARN package.json [email protected] No repository field.
[ERROR] npm http GET https://registry.npmjs.org/phantomjs
[ERROR] npm http GET https://registry.npmjs.org/protractor
[ERROR] npm http 304 https://registry.npmjs.org/protractor
[ERROR] npm http 304 https://registry.npmjs.org/phantomjs
[ERROR] npm http GET https://registry.npmjs.org/hasha
[ERROR] npm http GET https://registry.npmjs.org/kew
[ERROR] npm http GET https://registry.npmjs.org/progress
[ERROR] npm http GET https://registry.npmjs.org/request
[ERROR] npm http GET https://registry.npmjs.org/request-progress
[ERROR] npm http GET https://registry.npmjs.org/which
[ERROR] npm http GET https://registry.npmjs.org/extract-zip
[ERROR] npm http GET https://registry.npmjs.org/fs-extra
[ERROR] npm http GET https://registry.npmjs.org/request
[ERROR] npm http GET https://registry.npmjs.org/selenium-webdriver/2.44.0
[ERROR] npm http GET https://registry.npmjs.org/minijasminenode/1.1.1
[ERROR] npm http GET https://registry.npmjs.org/jasminewd/1.1.0
[ERROR] npm http GET https://registry.npmjs.org/jasminewd2/0.0.2
[ERROR] npm http GET https://registry.npmjs.org/jasmine/2.1.1
[ERROR] npm http GET https://registry.npmjs.org/saucelabs
[ERROR] npm http GET https://registry.npmjs.org/glob
[ERROR] npm http GET https://registry.npmjs.org/adm-zip/0.4.4
[ERROR] npm http GET https://registry.npmjs.org/optimist
[ERROR] npm http GET https://registry.npmjs.org/q/1.0.0
[ERROR] npm http GET https://registry.npmjs.org/lodash
[ERROR] npm http GET https://registry.npmjs.org/source-map-support
[ERROR] npm http GET https://registry.npmjs.org/html-entities
[ERROR] npm http GET https://registry.npmjs.org/accessibility-developer-tools
[ERROR] npm http 304 https://registry.npmjs.org/request-progress
[ERROR] npm http 304 https://registry.npmjs.org/request
[ERROR] npm http 304 https://registry.npmjs.org/hasha
[ERROR] npm http 304 https://registry.npmjs.org/fs-extra
[ERROR] npm http 304 https://registry.npmjs.org/html-entities
[ERROR] npm http 304 https://registry.npmjs.org/adm-zip/0.4.4
[ERROR] npm http 304 https://registry.npmjs.org/q/1.0.0
[ERROR] npm http 304 https://registry.npmjs.org/selenium-webdriver/2.44.0
[ERROR] npm http GET https://registry.npmjs.org/selenium-webdriver/-/selenium-webdriver-2.44.0.tgz
[ERROR] npm http 304 https://registry.npmjs.org/accessibility-developer-tools
[ERROR] npm http 304 https://registry.npmjs.org/progress
[ERROR] npm http 304 https://registry.npmjs.org/jasminewd/1.1.0
[ERROR] npm http 304 https://registry.npmjs.org/request
[ERROR] npm http 304 https://registry.npmjs.org/jasmine/2.1.1
[ERROR] npm http 304 https://registry.npmjs.org/extract-zip
[ERROR] npm http 304 https://registry.npmjs.org/which
[ERROR] npm http 304 https://registry.npmjs.org/source-map-support
[ERROR] npm http 304 https://registry.npmjs.org/jasminewd2/0.0.2
[ERROR] npm http 200 https://registry.npmjs.org/selenium-webdriver/-/selenium-webdriver-2.44.0.tgz
[ERROR] npm http 304 https://registry.npmjs.org/kew
[ERROR] npm http GET https://registry.npmjs.org/is-stream
[ERROR] npm http GET https://registry.npmjs.org/pinkie-promise
[ERROR] npm http GET https://registry.npmjs.org/mkdirp/0.5.0
[ERROR] npm http GET https://registry.npmjs.org/yauzl/2.4.1
[ERROR] npm http GET https://registry.npmjs.org/concat-stream/1.5.0
[ERROR] npm http GET https://registry.npmjs.org/debug/0.7.4
[ERROR] npm http GET https://registry.npmjs.org/is-absolute
[ERROR] npm http GET https://registry.npmjs.org/isexe
[ERROR] npm http GET https://registry.npmjs.org/throttleit
[ERROR] npm http GET https://registry.npmjs.org/graceful-fs
[ERROR] npm http GET https://registry.npmjs.org/jsonfile
[ERROR] npm http GET https://registry.npmjs.org/klaw
[ERROR] npm http GET https://registry.npmjs.org/path-is-absolute
[ERROR] npm http GET https://registry.npmjs.org/rimraf
[ERROR] npm http GET https://registry.npmjs.org/bl
[ERROR] npm http GET https://registry.npmjs.org/caseless
[ERROR] npm http GET https://registry.npmjs.org/extend
[ERROR] npm http GET https://registry.npmjs.org/forever-agent
[ERROR] npm http GET https://registry.npmjs.org/form-data
[ERROR] npm http GET https://registry.npmjs.org/json-stringify-safe
[ERROR] npm http GET https://registry.npmjs.org/mime-types
[ERROR] npm http GET https://registry.npmjs.org/node-uuid
[ERROR] npm http GET https://registry.npmjs.org/qs
[ERROR] npm http GET https://registry.npmjs.org/tunnel-agent
[ERROR] npm http GET https://registry.npmjs.org/tough-cookie
[ERROR] npm http GET https://registry.npmjs.org/http-signature
[ERROR] npm http GET https://registry.npmjs.org/oauth-sign
[ERROR] npm http GET https://registry.npmjs.org/aws-sign2
[ERROR] npm http GET https://registry.npmjs.org/hawk
[ERROR] npm http GET https://registry.npmjs.org/stringstream
[ERROR] npm http GET https://registry.npmjs.org/combined-stream
[ERROR] npm http GET https://registry.npmjs.org/isstream
[ERROR] npm http GET https://registry.npmjs.org/is-typedarray
[ERROR] npm http GET https://registry.npmjs.org/har-validator
[ERROR] npm http 304 https://registry.npmjs.org/saucelabs
[ERROR] npm http 304 https://registry.npmjs.org/mkdirp/0.5.0
[ERROR] npm http 304 https://registry.npmjs.org/pinkie-promise
[ERROR] npm http 304 https://registry.npmjs.org/is-stream
[ERROR] npm http 304 https://registry.npmjs.org/path-is-absolute
[ERROR] npm http GET https://registry.npmjs.org/pinkie
[ERROR] npm http 304 https://registry.npmjs.org/rimraf
[ERROR] npm http 304 https://registry.npmjs.org/concat-stream/1.5.0
[ERROR] npm http 304 https://registry.npmjs.org/isexe
[ERROR] npm http 304 https://registry.npmjs.org/graceful-fs
[ERROR] npm http 304 https://registry.npmjs.org/klaw
[ERROR] npm http 304 https://registry.npmjs.org/json-stringify-safe
[ERROR] npm http 304 https://registry.npmjs.org/qs
[ERROR] npm http 304 https://registry.npmjs.org/tough-cookie
[ERROR] npm http 304 https://registry.npmjs.org/http-signature
[ERROR] npm http 304 https://registry.npmjs.org/stringstream
[ERROR] npm http 304 https://registry.npmjs.org/isstream
[ERROR] npm http 304 https://registry.npmjs.org/har-validator
[ERROR] npm http 304 https://registry.npmjs.org/yauzl/2.4.1
[ERROR] npm http 304 https://registry.npmjs.org/tunnel-agent
[ERROR] npm http 304 https://registry.npmjs.org/form-data
[ERROR] npm http 304 https://registry.npmjs.org/combined-stream
[ERROR] npm http 304 https://registry.npmjs.org/mime-types
[ERROR] npm http 304 https://registry.npmjs.org/optimist
[ERROR] npm http 304 https://registry.npmjs.org/lodash
[ERROR] npm http 304 https://registry.npmjs.org/is-absolute
[ERROR] npm http GET https://registry.npmjs.org/is-relative
[ERROR] npm http 304 https://registry.npmjs.org/extend
[ERROR] npm http 304 https://registry.npmjs.org/caseless
[ERROR] npm http 304 https://registry.npmjs.org/is-typedarray
[ERROR] npm http 304 https://registry.npmjs.org/node-uuid
[ERROR] npm http 304 https://registry.npmjs.org/debug/0.7.4
[ERROR] npm http GET https://registry.npmjs.org/inherits
[ERROR] npm http GET https://registry.npmjs.org/typedarray
[ERROR] npm http GET https://registry.npmjs.org/readable-stream
[ERROR] npm http GET https://registry.npmjs.org/fd-slicer
[ERROR] npm http GET https://registry.npmjs.org/minimist/0.0.8
[ERROR] npm http 304 https://registry.npmjs.org/forever-agent
[ERROR] npm http 304 https://registry.npmjs.org/oauth-sign
[ERROR] npm http 304 https://registry.npmjs.org/is-relative
[ERROR] npm http 304 https://registry.npmjs.org/inherits
[ERROR] npm http 304 https://registry.npmjs.org/readable-stream
[ERROR] npm http 304 https://registry.npmjs.org/pinkie
[ERROR] npm http 304 https://registry.npmjs.org/glob
[ERROR] npm http 304 https://registry.npmjs.org/bl
[ERROR] npm http 304 https://registry.npmjs.org/hawk
[ERROR] npm http 304 https://registry.npmjs.org/minimist/0.0.8
[ERROR] npm http 304 https://registry.npmjs.org/throttleit
[ERROR] npm http 304 https://registry.npmjs.org/fd-slicer
[ERROR] npm http GET https://registry.npmjs.org/pend
[ERROR] npm http 304 https://registry.npmjs.org/pend
[ERROR] npm http 304 https://registry.npmjs.org/aws-sign2
[ERROR] npm WARN engine [email protected]: wanted: {"node":">=0.10.32"} (current: {"node":"v0.10.30","npm":"1.4.3"})
[ERROR] npm http GET https://registry.npmjs.org/delayed-stream
[ERROR] npm http GET https://registry.npmjs.org/mime-db
[ERROR] npm http GET https://registry.npmjs.org/async
[ERROR] npm http GET https://registry.npmjs.org/assert-plus
[ERROR] npm http GET https://registry.npmjs.org/jsprim
[ERROR] npm http GET https://registry.npmjs.org/sshpk
[ERROR] npm http GET https://registry.npmjs.org/chalk
[ERROR] npm http GET https://registry.npmjs.org/commander
[ERROR] npm http GET https://registry.npmjs.org/is-my-json-valid
[ERROR] npm http GET https://registry.npmjs.org/core-util-is
[ERROR] npm http GET https://registry.npmjs.org/isarray
[ERROR] npm http GET https://registry.npmjs.org/process-nextick-args
[ERROR] npm http GET https://registry.npmjs.org/string_decoder
[ERROR] npm http GET https://registry.npmjs.org/util-deprecate
[ERROR] npm http GET https://registry.npmjs.org/hoek
[ERROR] npm http GET https://registry.npmjs.org/boom
[ERROR] npm http GET https://registry.npmjs.org/cryptiles
[ERROR] npm http GET https://registry.npmjs.org/sntp
[ERROR] npm http 304 https://registry.npmjs.org/mime-db
[ERROR] npm http 304 https://registry.npmjs.org/delayed-stream
[ERROR] npm http 304 https://registry.npmjs.org/process-nextick-args
[ERROR] npm http 304 https://registry.npmjs.org/util-deprecate
[ERROR] npm http 304 https://registry.npmjs.org/assert-plus
[ERROR] npm http 304 https://registry.npmjs.org/hoek
[ERROR] npm http 304 https://registry.npmjs.org/is-my-json-valid
[ERROR] npm http 304 https://registry.npmjs.org/sshpk
[ERROR] npm http 304 https://registry.npmjs.org/core-util-is
[ERROR] npm http 200 https://registry.npmjs.org/isarray
[ERROR] npm http 304 https://registry.npmjs.org/chalk
[ERROR] npm http 304 https://registry.npmjs.org/cryptiles
[ERROR] npm http 304 https://registry.npmjs.org/boom
[ERROR] npm http 304 https://registry.npmjs.org/typedarray
[ERROR] npm http 304 https://registry.npmjs.org/sntp
[ERROR] npm WARN engine [email protected]: wanted: {"node":">=0.10.40"} (current: {"node":"v0.10.30","npm":"1.4.3"})
[ERROR] npm WARN engine [email protected]: wanted: {"node":">=0.10.40"} (current: {"node":"v0.10.30","npm":"1.4.3"})
[ERROR] npm WARN engine [email protected]: wanted: {"node":">=0.10.40"} (current: {"node":"v0.10.30","npm":"1.4.3"})
[ERROR] npm http 304 https://registry.npmjs.org/jsprim
[ERROR] npm http GET https://registry.npmjs.org/extsprintf/1.0.2
[ERROR] npm http GET https://registry.npmjs.org/json-schema/0.2.2
[ERROR] npm http GET https://registry.npmjs.org/verror/1.3.6
[ERROR] npm http 200 https://registry.npmjs.org/commander
[ERROR] npm http 200 https://registry.npmjs.org/async
[ERROR] npm http GET https://registry.npmjs.org/ansi-styles
[ERROR] npm http GET https://registry.npmjs.org/escape-string-regexp
[ERROR] npm http GET https://registry.npmjs.org/has-ansi
[ERROR] npm http GET https://registry.npmjs.org/strip-ansi
[ERROR] npm http GET https://registry.npmjs.org/supports-color
[ERROR] npm http GET https://registry.npmjs.org/graceful-readlink
[ERROR] npm http GET https://registry.npmjs.org/asn1
[ERROR] npm http GET https://registry.npmjs.org/dashdash
[ERROR] npm http GET https://registry.npmjs.org/jsbn
[ERROR] npm http GET https://registry.npmjs.org/tweetnacl
[ERROR] npm http GET https://registry.npmjs.org/jodid25519
[ERROR] npm http GET https://registry.npmjs.org/ecc-jsbn
[ERROR] npm http GET https://registry.npmjs.org/generate-function
[ERROR] npm http GET https://registry.npmjs.org/generate-object-property
[ERROR] npm http GET https://registry.npmjs.org/jsonpointer/2.0.0
[ERROR] npm http GET https://registry.npmjs.org/xtend
[ERROR] npm http 200 https://registry.npmjs.org/string_decoder
[ERROR] npm http 304 https://registry.npmjs.org/jsonfile
[ERROR] npm http GET https://registry.npmjs.org/glob
[ERROR] npm http 304 https://registry.npmjs.org/json-schema/0.2.2
[ERROR] npm http 304 https://registry.npmjs.org/supports-color
[ERROR] npm http 304 https://registry.npmjs.org/jodid25519
[ERROR] npm http 304 https://registry.npmjs.org/tweetnacl
[ERROR] npm http 304 https://registry.npmjs.org/jsbn
[ERROR] npm http 304 https://registry.npmjs.org/ansi-styles
[ERROR] npm http 304 https://registry.npmjs.org/asn1
[ERROR] npm http 304 https://registry.npmjs.org/dashdash
[ERROR] npm http 304 https://registry.npmjs.org/generate-object-property
[ERROR] npm http 304 https://registry.npmjs.org/strip-ansi
[ERROR] npm http 304 https://registry.npmjs.org/verror/1.3.6
[ERROR] npm http 304 https://registry.npmjs.org/ecc-jsbn
[ERROR] npm http 304 https://registry.npmjs.org/jsonpointer/2.0.0
[ERROR] npm http 304 https://registry.npmjs.org/graceful-readlink
[ERROR] npm http 304 https://registry.npmjs.org/generate-function
[ERROR] npm http 304 https://registry.npmjs.org/glob
[ERROR] npm http GET https://registry.npmjs.org/inflight
[ERROR] npm http GET https://registry.npmjs.org/inherits
[ERROR] npm http GET https://registry.npmjs.org/minimatch
[ERROR] npm http GET https://registry.npmjs.org/once
[ERROR] npm http 304 https://registry.npmjs.org/once
[ERROR] npm http 304 https://registry.npmjs.org/inherits
[ERROR] npm http 304 https://registry.npmjs.org/minimatch
[ERROR] npm http 304 https://registry.npmjs.org/has-ansi
[ERROR] npm http 304 https://registry.npmjs.org/xtend
[ERROR] npm http GET https://registry.npmjs.org/is-property
[ERROR] npm http 200 https://registry.npmjs.org/inflight
[ERROR] npm http GET https://registry.npmjs.org/wrappy
[ERROR] npm http GET https://registry.npmjs.org/brace-expansion
[ERROR] npm http 304 https://registry.npmjs.org/escape-string-regexp
[ERROR] npm http GET https://registry.npmjs.org/ansi-regex
[ERROR] npm http 304 https://registry.npmjs.org/ansi-regex
[ERROR] npm http 304 https://registry.npmjs.org/brace-expansion
[ERROR] npm http GET https://registry.npmjs.org/balanced-match
[ERROR] npm http GET https://registry.npmjs.org/concat-map/0.0.1
[ERROR] npm http 304 https://registry.npmjs.org/is-property
[ERROR] npm http 200 https://registry.npmjs.org/balanced-match
[ERROR] npm http 304 https://registry.npmjs.org/concat-map/0.0.1
[ERROR] npm http 200 https://registry.npmjs.org/wrappy
[ERROR] npm http GET https://registry.npmjs.org/minijasminenode/1.1.1
[ERROR] npm http 304 https://registry.npmjs.org/minijasminenode/1.1.1
[ERROR] npm http GET https://registry.npmjs.org/extsprintf/1.0.2
[ERROR] npm http 304 https://registry.npmjs.org/extsprintf/1.0.2

还有一个额外的信息:Exception while executing SCM command.

最后在英文网站上找到了一个解决策,但原因是什么,没有明白:https://github.com/Mach5/supersonic/issues/85

Find the lines that read:

org.codehaus.mojo
buildnumber-maven-plugin
1.0

and remove them.



但,新的问题又出现了,打出的堆栈信息如下所示....

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (BUILD-OOZIE) on project build-tools: Command execution failed. Process exited with an error: 6 (Exit value: 6) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (BUILD-OOZIE) on project build-tools: Command execution failed.
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:318)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:153)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:555)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:414)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:357)
Caused by: org.apache.maven.plugin.MojoExecutionException: Command execution failed.
        at org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:362)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
        ... 19 more
Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: 6 (Exit value: 6)
        at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:377)
        at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:160)
        at org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:610)
        at org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:352)
        ... 21 more
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :build-tools

根据上面的提示信息,找到一个对应的帖子:

http://mail-archives.apache.org/mod_mbox/falcon-dev/201402.mbox/%[email protected]%3E

https://issues.apache.org/jira/browse/FALCON-291

看着是Falcon的一个降级BUG,发邮件询问Falcon社区的负责人,一天过去了,没有回信。

自己动手去琢磨,falcon0.4作入的BUG,0.9的话,应该是已经被修正了,估计是自己这边的环境有问题,再继续折腾吧,找到解决策后再继续追加。


@2016-4-15 15:32

全局使用maven编译的时候,一直报错

Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (BUILD-OOZIE)

但具体是哪有问题,即使使用mvn clean install -X打印出详细的堆栈信息,对于不清楚源码的人来说,还是没有什么实质性的帮助。遂决定单独去编译build-tool模块。

单独编译build-tool时,出错信息就详细多了,且很有价值的信息。具体如下所示:

[INFO] --- exec-maven-plugin:1.2.1:exec (BUILD-OOZIE) @ build-tools ---
oozie version 
Using maven from  /home/project/apache-maven-3.1.0/bin/mvn
+ VERSION=4.1.0
+ BUILD_VERSION=4.1.0-falcon
+ FORCE_BUILD=false
+ echo 'oozie version '
+ '[' -z /home/project/apache-maven-3.1.0 ']'
+ export MVN_CMD=/home/project/apache-maven-3.1.0/bin/mvn
+ MVN_CMD=/home/project/apache-maven-3.1.0/bin/mvn
+ echo 'Using maven from ' /home/project/apache-maven-3.1.0/bin/mvn
+ [[ false == \f\a\l\s\e ]]
++ cut '-d>' -f2
++ cut '-d<' -f1
++ grep localRepository
++ /home/project/apache-maven-3.1.0/bin/mvn help:effective-settings
+ [[ -f /root/.m2/repository/org/apache/oozie/oozie-webapp/4.1.0-falcon/oozie-webapp-4.1.0-falcon.war ]]
+ PKG_URL=http://archive.apache.org/dist/oozie/4.1.0/oozie-4.1.0.tar.gz
+ PKG=oozie-4.1.0
+ mkdir -p ../target
+ pushd ../target
/home/project/apache-maven-3.1.0/falcon-sources-0.9/target /home/project/apache-maven-3.1.0/falcon-sources-0.9/build-tools
+ rm -rf 'oozie-*'
+ curl -v http://archive.apache.org/dist/oozie/4.1.0/oozie-4.1.0.tar.gz -o oozie-4.1.0.tgz
* getaddrinfo(3) failed for archive.apache.org:80
* Couldn't resolve host 'archive.apache.org'
* Closing connection #0

curl: (6) Couldn't resolve host 'archive.apache.org'
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.362s
[INFO] Finished at: Sat Apr 16 00:51:40 JST 2016
[INFO] Final Memory: 10M/239M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (BUILD-OOZIE) on project build-tools: Command execution failed. Process exited with an error: 6 (Exit value: 6) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException


看到上面黑体加错的那三行,就知道是什么原因造成build-tool一直无法编译了。

getaddrinfo(3)出错导致的。

继续解析,发现是自己的虚拟机没有配DNS,或者说,没有在虚拟机中设置代理。可悲的是,下班后坐在车上才想通了,原来只需要在UI界面设置一下代理,就可以下载这个http://archive.apache.org/dist/oozie/4.1.0/oozie-4.1.0.tar.gz文件了。可惜,下班之前没有搞定,只能下周一接着去确认了。

@2016/04/18

今天到公司后,配好虚拟机的代理。能上网,但就是ping不通www.baidu.com。主机这边也是ping不通www.baidu.com。猜测应该是整个公司将DNS的解析给屏蔽掉了。再次执行mvn clean install,还是出问题,但在vmware下执行的时候,之前的错误就没发生,这会一直处于关联文件下载中,对公司的网速表示呵呵了。但没报错就是好事了....

新的问题又出现了,在build testcase的时候出错了。。。

Tests run: 41, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.763 sec - in org.apache.falcon.workflow.WorkflowExecutionContextTest
Running org.apache.falcon.update.UpdateHelperTest
Tests run: 5, Failures: 4, Errors: 0, Skipped: 0, Time elapsed: 3.421 sec <<< FAILURE! - in org.apache.falcon.update.UpdateHelperTest
testIsEntityACLUpdated(org.apache.falcon.update.UpdateHelperTest)  Time elapsed: 0.397 sec  <<< FAILURE!
org.apache.hadoop.fs.ParentNotDirectoryException: Parent path is not a directory: /home/project/apache-maven-3.3.9/falcon-sources-0.9/common/target/falcon/tmp-hadoop-root/jail-fs/testCluster/projects/falcon/staging/falcon/workflows/process/sample
        at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:418)
        at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:432)
        at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:281)
        at org.apache.falcon.hadoop.JailedFileSystem.mkdirs(JailedFileSystem.java:152)
        at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1815)
        at org.apache.falcon.update.UpdateHelperTest.prepare(UpdateHelperTest.java:90)
        at org.apache.falcon.update.UpdateHelperTest.testIsEntityACLUpdated(UpdateHelperTest.java:262)

testIsEntityLateProcessUpdated(org.apache.falcon.update.UpdateHelperTest)  Time elapsed: 0.068 sec  <<< FAILURE!
org.apache.hadoop.fs.ParentNotDirectoryException: Parent path is not a directory: /home/project/apache-maven-3.3.9/falcon-sources-0.9/common/target/falcon/tmp-hadoop-root/jail-fs/testCluster/projects/falcon/staging/falcon/workflows/process/sample
        at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:418)
        at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:432)
        at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:281)
        at org.apache.falcon.hadoop.JailedFileSystem.mkdirs(JailedFileSystem.java:152)
        at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1815)
        at org.apache.falcon.update.UpdateHelperTest.prepare(UpdateHelperTest.java:90)
        at org.apache.falcon.update.UpdateHelperTest.testIsEntityLateProcessUpdated(UpdateHelperTest.java:280)

testIsEntityUpdated(org.apache.falcon.update.UpdateHelperTest)  Time elapsed: 0.209 sec  <<< FAILURE!
org.apache.hadoop.fs.ParentNotDirectoryException: Parent path is not a directory: /home/project/apache-maven-3.3.9/falcon-sources-0.9/common/target/falcon/tmp-hadoop-root/jail-fs/testCluster/projects/falcon/staging/falcon/workflows/process/sample
        at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:418)
        at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:432)
        at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:281)
        at org.apache.falcon.hadoop.JailedFileSystem.mkdirs(JailedFileSystem.java:152)
        at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1815)
        at org.apache.falcon.update.UpdateHelperTest.prepare(UpdateHelperTest.java:90)
        at org.apache.falcon.update.UpdateHelperTest.testIsEntityUpdated(UpdateHelperTest.java:117)

testShouldUpdateAffectedEntities(org.apache.falcon.update.UpdateHelperTest)  Time elapsed: 0.052 sec  <<< FAILURE!
org.apache.hadoop.fs.ParentNotDirectoryException: Parent path is not a directory: /home/project/apache-maven-3.3.9/falcon-sources-0.9/common/target/falcon/tmp-hadoop-root/jail-fs/testCluster/projects/falcon/staging/falcon/workflows/process/sample
        at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:418)
        at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:432)
        at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:281)
        at org.apache.falcon.hadoop.JailedFileSystem.mkdirs(JailedFileSystem.java:152)
        at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1815)
        at org.apache.falcon.update.UpdateHelperTest.prepare(UpdateHelperTest.java:90)
        at org.apache.falcon.update.UpdateHelperTest.testShouldUpdateAffectedEntities(UpdateHelperTest.java:157)

Running org.apache.falcon.metadata.MetadataMappingServiceTest
Tests run: 17, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.512 sec - in org.apache.falcon.metadata.MetadataMappingServiceTest
Running org.apache.falcon.security.AuthenticationInitializationServiceTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.027 sec - in org.apache.falcon.security.AuthenticationInitializationServiceTest
Running org.apache.falcon.security.DefaultAuthorizationProviderTest
Tests run: 34, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.884 sec - in org.apache.falcon.security.DefaultAuthorizationProviderTest
Running org.apache.falcon.security.CurrentUserTest
Tests run: 22, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.062 sec - in org.apache.falcon.security.CurrentUserTest
Running org.apache.falcon.security.SecurityUtilTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.202 sec - in org.apache.falcon.security.SecurityUtilTest
Running org.apache.falcon.cleanup.LogCleanupServiceTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.601 sec - in org.apache.falcon.cleanup.LogCleanupServiceTest
Running org.apache.falcon.util.RadixTreeTest
Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.894 sec - in org.apache.falcon.util.RadixTreeTest
Running org.apache.falcon.util.ApplicationPropertiesTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.514 sec - in org.apache.falcon.util.ApplicationPropertiesTest
Running org.apache.falcon.util.RadixNodeTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.432 sec - in org.apache.falcon.util.RadixNodeTest
Running org.apache.falcon.util.ReflectionUtilsTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.133 sec - in org.apache.falcon.util.ReflectionUtilsTest
Running org.apache.falcon.expression.ExpressionHelperTest
Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.458 sec - in org.apache.falcon.expression.ExpressionHelperTest
Running org.apache.falcon.group.FeedGroupMapTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.25 sec - in org.apache.falcon.group.FeedGroupMapTest

Results :

Failed tests: 
  UpdateHelperTest.testIsEntityACLUpdated:262->prepare:90 » ParentNotDirectory P...
  UpdateHelperTest.testIsEntityLateProcessUpdated:280->prepare:90 » ParentNotDirectory
  UpdateHelperTest.testIsEntityUpdated:117->prepare:90 » ParentNotDirectory Pare...
  UpdateHelperTest.testShouldUpdateAffectedEntities:157->prepare:90 » ParentNotDirectory

Tests run: 575, Failures: 4, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Falcon ...................................... SUCCESS [  2.882 s]
[INFO] Apache Falcon UI ................................... SUCCESS [  7.223 s]
[INFO] Checkstyle ......................................... SUCCESS [  0.470 s]
[INFO] Build Tools ........................................ SUCCESS [  4.177 s]
[INFO] Apache Falcon Java client .......................... SUCCESS [ 12.203 s]
[INFO] Apache Falcon CLI client ........................... SUCCESS [  3.790 s]
[INFO] Apache Falcon Metrics .............................. SUCCESS [  4.277 s]
[INFO] Apache Falcon Hadoop Dependencies .................. SUCCESS [  1.939 s]
[INFO] Apache Falcon Test Utility ......................... SUCCESS [  3.077 s]
[INFO] Apache Falcon Commons .............................. FAILURE [01:44 min]
[INFO] Apache Falcon Oozie EL Extension ................... SKIPPED
[INFO] Apache Falcon Embedded Hadoop - Test Cluster ....... SKIPPED
[INFO] Apache Falcon Sharelib Hive - Test Cluster ......... SKIPPED
[INFO] Apache Falcon Sharelib Pig - Test Cluster .......... SKIPPED
[INFO] Apache Falcon Sharelib Hcatalog - Test Cluster ..... SKIPPED
[INFO] Apache Falcon Sharelib Oozie - Test Cluster ........ SKIPPED
[INFO] Apache Falcon Test Tools - Test Cluster ............ SKIPPED
[INFO] Apache Falcon Messaging ............................ SKIPPED
[INFO] Apache Falcon LIfecycle Module ..................... SKIPPED
[INFO] Apache Falcon Oozie Adaptor ........................ SKIPPED
[INFO] Apache Falcon Scheduler ............................ SKIPPED
[INFO] Apache Falcon Acquisition .......................... SKIPPED
[INFO] Apache Falcon Distcp Replication ................... SKIPPED
[INFO] Apache Falcon Retention ............................ SKIPPED
[INFO] Apache Falcon Archival ............................. SKIPPED
[INFO] Apache Falcon Rerun ................................ SKIPPED
[INFO] Apache Falcon Prism ................................ SKIPPED
[INFO] falcon-unit ........................................ SKIPPED
[INFO] Apache Falcon Web Application ...................... SKIPPED
[INFO] Apache Falcon Documentation ........................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:25 min
[INFO] Finished at: 2016-04-18T22:29:01+09:00
[INFO] Final Memory: 154M/567M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.16:test (default-test) on project falcon-common: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/project/apache-maven-3.3.9/falcon-sources-0.9/common/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :falcon-common

因为本次编译的重点是falcon源码,至于自带的测试项,对后续的使用是没有用处的,换句话说,编译测试项,只是进一步证明了而已,为了,解决方案是跳过它。

解决命令:

export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=256m -noverify" && mvn clean install -Dhadoop.version=2.5.2 -DskipTests
再次执行,编译又可以继续了,又是不停的下载关联的文件的模式了......

但,还是看到了希望

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Falcon ...................................... SUCCESS [  3.030 s]
[INFO] Apache Falcon UI ................................... SUCCESS [  7.757 s]
[INFO] Checkstyle ......................................... SUCCESS [  0.504 s]
[INFO] Build Tools ........................................ SUCCESS [  4.186 s]
[INFO] Apache Falcon Java client .......................... SUCCESS [ 10.193 s]
[INFO] Apache Falcon CLI client ........................... SUCCESS [  3.747 s]
[INFO] Apache Falcon Metrics .............................. SUCCESS [  3.663 s]
[INFO] Apache Falcon Hadoop Dependencies .................. SUCCESS [  1.945 s]
[INFO] Apache Falcon Test Utility ......................... SUCCESS [  2.465 s]
[INFO] Apache Falcon Commons .............................. SUCCESS [  9.517 s]
[INFO] Apache Falcon Oozie EL Extension ................... SUCCESS [  5.234 s]
[INFO] Apache Falcon Embedded Hadoop - Test Cluster ....... SUCCESS [07:07 min]
[INFO] Apache Falcon Sharelib Hive - Test Cluster ......... SUCCESS [  8.564 s]
[INFO] Apache Falcon Sharelib Pig - Test Cluster .......... SUCCESS [01:01 min]
[INFO] Apache Falcon Sharelib Hcatalog - Test Cluster ..... SUCCESS [  0.826 s]
[INFO] Apache Falcon Sharelib Oozie - Test Cluster ........ SUCCESS [  0.093 s]
[INFO] Apache Falcon Test Tools - Test Cluster ............ SUCCESS [  0.049 s]
[INFO] Apache Falcon Messaging ............................ SUCCESS [  2.779 s]
[INFO] Apache Falcon LIfecycle Module ..................... SUCCESS [01:25 min]
[INFO] Apache Falcon Oozie Adaptor ........................ SUCCESS [  7.485 s]
[INFO] Apache Falcon Scheduler ............................ SUCCESS [02:04 min]
[INFO] Apache Falcon Acquisition .......................... SUCCESS [  0.251 s]
[INFO] Apache Falcon Distcp Replication ................... SUCCESS [  1.719 s]
[INFO] Apache Falcon Retention ............................ SUCCESS [  1.805 s]
[INFO] Apache Falcon Archival ............................. SUCCESS [  0.243 s]
[INFO] Apache Falcon Rerun ................................ SUCCESS [  3.625 s]
[INFO] Apache Falcon Prism ................................ SUCCESS [09:49 min]
[INFO] falcon-unit ........................................ SUCCESS [  5.552 s]
[INFO] Apache Falcon Web Application ...................... SUCCESS [02:45 min]
[INFO] Apache Falcon Documentation ........................ FAILURE [ 11.370 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 25:50 min
[INFO] Finished at: 2016-04-18T22:56:18+09:00
[INFO] Final Memory: 252M/868M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-site-plugin:3.4:site (default) on project falcon-docs: SiteToolException: The site descriptor cannot be resolved from the repository: ArtifactResolutionException: Unable to locate site descriptor: Could not transfer artifact org.apache:apache:xml:site_en:17 from/to java.net-Public (https://maven.java.net/content/groups/public/): Failed to transfer file: https://maven.java.net/content/groups/public/org/apache/apache/17/apache-17-site_en.xml. Return code is: 503 , ReasonPhrase:Service Unavailable.
[ERROR] org.apache:apache:xml:17
[ERROR] 
[ERROR] from the specified remote repositories:
[ERROR] central (http://repo1.maven.org/maven2, releases=true, snapshots=false),
[ERROR] im (https://raw.github.com/InMobi/mvn-repo/master/releases, releases=true, snapshots=false),
[ERROR] hortonworks.repo (http://repo.hortonworks.com/content/repositories/releases, releases=true, snapshots=false),
[ERROR] apache.snapshots.repo (https://repository.apache.org/content/groups/snapshots, releases=true, snapshots=true),
[ERROR] default (https://repository.apache.org/content/groups/public/, releases=true, snapshots=true),
[ERROR] java.net-Public (https://maven.java.net/content/groups/public/, releases=true, snapshots=true),
[ERROR] repository.jboss.org-public (https://repository.jboss.org/nexus/content/groups/public, releases=true, snapshots=true),
[ERROR] cdh.repo (https://repository.cloudera.com/artifactory/cloudera-repos/, releases=true, snapshots=false),
[ERROR] apache.snapshots (http://repository.apache.org/snapshots, releases=false, snapshots=true)
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :falcon-docs
[INFO] Shutdown hook executing
[INFO] Shutdown hook complete
Shutting down hive metastore.

根据这报错信息,以及这么久折腾falcon的经验来看,应该还是网速的原因,在timeout时间内没有找到对应的服务器导致了,果断的重新来过一次,果然,这次编译过了,效果如下所示:

[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Falcon ...................................... SUCCESS [  2.613 s]
[INFO] Apache Falcon UI ................................... SUCCESS [  7.466 s]
[INFO] Checkstyle ......................................... SUCCESS [  0.414 s]
[INFO] Build Tools ........................................ SUCCESS [  4.255 s]
[INFO] Apache Falcon Java client .......................... SUCCESS [ 10.174 s]
[INFO] Apache Falcon CLI client ........................... SUCCESS [  3.492 s]
[INFO] Apache Falcon Metrics .............................. SUCCESS [  3.342 s]
[INFO] Apache Falcon Hadoop Dependencies .................. SUCCESS [  1.973 s]
[INFO] Apache Falcon Test Utility ......................... SUCCESS [  2.497 s]
[INFO] Apache Falcon Commons .............................. SUCCESS [  8.428 s]
[INFO] Apache Falcon Oozie EL Extension ................... SUCCESS [  2.378 s]
[INFO] Apache Falcon Embedded Hadoop - Test Cluster ....... SUCCESS [ 26.021 s]
[INFO] Apache Falcon Sharelib Hive - Test Cluster ......... SUCCESS [  3.103 s]
[INFO] Apache Falcon Sharelib Pig - Test Cluster .......... SUCCESS [  0.378 s]
[INFO] Apache Falcon Sharelib Hcatalog - Test Cluster ..... SUCCESS [  0.677 s]
[INFO] Apache Falcon Sharelib Oozie - Test Cluster ........ SUCCESS [  0.097 s]
[INFO] Apache Falcon Test Tools - Test Cluster ............ SUCCESS [  0.042 s]
[INFO] Apache Falcon Messaging ............................ SUCCESS [  2.377 s]
[INFO] Apache Falcon LIfecycle Module ..................... SUCCESS [ 11.402 s]
[INFO] Apache Falcon Oozie Adaptor ........................ SUCCESS [  6.631 s]
[INFO] Apache Falcon Scheduler ............................ SUCCESS [  6.307 s]
[INFO] Apache Falcon Acquisition .......................... SUCCESS [  0.280 s]
[INFO] Apache Falcon Distcp Replication ................... SUCCESS [  1.735 s]
[INFO] Apache Falcon Retention ............................ SUCCESS [  1.936 s]
[INFO] Apache Falcon Archival ............................. SUCCESS [  0.277 s]
[INFO] Apache Falcon Rerun ................................ SUCCESS [  4.892 s]
[INFO] Apache Falcon Prism ................................ SUCCESS [ 37.770 s]
[INFO] falcon-unit ........................................ SUCCESS [  6.706 s]
[INFO] Apache Falcon Web Application ...................... SUCCESS [01:09 min]
[INFO] Apache Falcon Documentation ........................ SUCCESS [ 25.824 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:14 min
[INFO] Finished at: 2016-04-18T23:03:24+09:00
[INFO] Final Memory: 141M/739M
[INFO] ------------------------------------------------------------------------
[INFO] Shutdown hook executing
[INFO] Shutdown hook complete
Shutting down hive metastore.

至此,apache falcon的源码终于编译成功了。


部署嵌入式模式:

命令:

mvn clean assembly:assembly -DskipTests -DskipITs

效果:

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Falcon ...................................... SUCCESS [03:40 min]
[INFO] Apache Falcon UI ................................... SUCCESS [  0.008 s]
[INFO] Checkstyle ......................................... SUCCESS [  0.002 s]
[INFO] Build Tools ........................................ SUCCESS [  0.004 s]
[INFO] Apache Falcon Java client .......................... SUCCESS [  0.033 s]
[INFO] Apache Falcon CLI client ........................... SUCCESS [  0.007 s]
[INFO] Apache Falcon Metrics .............................. SUCCESS [  0.007 s]
[INFO] Apache Falcon Hadoop Dependencies .................. SUCCESS [  0.017 s]
[INFO] Apache Falcon Test Utility ......................... SUCCESS [  0.004 s]
[INFO] Apache Falcon Commons .............................. SUCCESS [  0.024 s]
[INFO] Apache Falcon Oozie EL Extension ................... SUCCESS [  0.006 s]
[INFO] Apache Falcon Embedded Hadoop - Test Cluster ....... SUCCESS [  0.092 s]
[INFO] Apache Falcon Sharelib Hive - Test Cluster ......... SUCCESS [  0.004 s]
[INFO] Apache Falcon Sharelib Pig - Test Cluster .......... SUCCESS [  0.004 s]
[INFO] Apache Falcon Sharelib Hcatalog - Test Cluster ..... SUCCESS [  0.004 s]
[INFO] Apache Falcon Sharelib Oozie - Test Cluster ........ SUCCESS [  0.003 s]
[INFO] Apache Falcon Test Tools - Test Cluster ............ SUCCESS [  0.003 s]
[INFO] Apache Falcon Messaging ............................ SUCCESS [  0.006 s]
[INFO] Apache Falcon LIfecycle Module ..................... SUCCESS [  0.020 s]
[INFO] Apache Falcon Oozie Adaptor ........................ SUCCESS [  0.020 s]
[INFO] Apache Falcon Scheduler ............................ SUCCESS [  0.049 s]
[INFO] Apache Falcon Acquisition .......................... SUCCESS [  0.004 s]
[INFO] Apache Falcon Distcp Replication ................... SUCCESS [  0.005 s]
[INFO] Apache Falcon Retention ............................ SUCCESS [  0.004 s]
[INFO] Apache Falcon Archival ............................. SUCCESS [  0.004 s]
[INFO] Apache Falcon Rerun ................................ SUCCESS [  0.006 s]
[INFO] Apache Falcon Prism ................................ SUCCESS [  0.118 s]
[INFO] falcon-unit ........................................ SUCCESS [  0.007 s]
[INFO] Apache Falcon Web Application ...................... SUCCESS [  0.245 s]
[INFO] Apache Falcon Documentation ........................ SUCCESS [  0.008 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:43 min
[INFO] Finished at: 2016-04-18T23:13:40+09:00
[INFO] Final Memory: 603M/882M
[INFO] ------------------------------------------------------------------------

解压apache-falcon-${project.version}-bin.tar.gz

文件tree:

|- bin
   |- falcon
   |- falcon-start
   |- falcon-stop
   |- falcon-config.sh
   |- service-start.sh
   |- service-stop.sh
|- conf
   |- startup.properties
   |- runtime.properties
   |- client.properties
   |- log4j.xml
   |- falcon-env.sh
|- docs
|- client
   |- lib (client support libs)
|- server
   |- webapp
      |- falcon.war
|- hadooplibs
|- README
|- NOTICE.txt
|- LICENSE.txt
|- DISCLAIMER.txt
|- CHANGES.txt


位置:{project dir}/target/apache-falcon-${project.version}-bin.tar.gz

* tar -xzvf apache-falcon-${project.version}-bin.tar.gz
* cd falcon-${project.version}


启动falcon:

bin/falcon-start

Hadoop home is set, adding libraries from '/home/project/hadoop-2.5.2/bin/hadoop classpath' into falcon classpath
/home/project/apache-maven-3.3.9/falcon-sources-0.9/target/falcon-0.9
falcon started using hadoop version:  Hadoop 2.5.2

注意点:

falcon.url=https://localhost:15443/

但要使用web界面来访问的话,需要稍加修改一下:https://localhost:15443/public

firfox或者chorme或者IE9以上登录,追加认证后就可以显示falcon的画面了。


感触:

1.本次的搭建环境从头到尾,制约因素最大的就是网速了

2.虚拟机的DNS和代理没有配置,导致开始building时没有准确的定位到。以为设置了maven的代理,应该就没事了。

3.没有善用-x参数,在解析maven.apche.org的时候,不知所措。

------ over -------


你可能感兴趣的:(apache,falcon)