源码编译zepplin踩过的那些坑

最近项目需要从源码安装zepplin，所以就来试下源码安装。

官方教材： http://zeppelin.apache.org/docs/0.7.3/install/build.html

Step1. 确定系统环境和需要的各软件版本

Centos6.7

Orace JDK1.8

Hadoop 2.6.0-cdh5.10.2
hbase 1.2.0-cdh5.10.2
spark 2.2.0

我们用到的版本如上, 因为spark版本2.2 默认使用的scala2.11。

修改scala版本为2.11

# update all pom.xml to use scala 2.11
./dev/change_scala_version.sh 2.11

Step2. 编译打包

mvn clean package -Pbuild-distr,helium-dev,hadoop-2.6,r,spark-2.2,vendor-repo -Dhadoop.version=2.6.0-cdh5.10.2 -Dhbase.hbase.version=1.2.0-cdh5.10.2 -Dhbase.hadoop.version=2.6.0-cdh5.10.2 -DskipTests

碰到的第一个问题：

Failed to execute goal org.scala-tools:maven-scala-plugin:2.15.2:compile (default) on project zeppelin-display_2.11: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1) -> [Help 1]

开始被这里的错误迷惑了，查看上面的错误提示，

Compiling 6 source files to /tmp/zeppelin/zeppelin-display/target/classes at 1516876315586
[ERROR] /tmp/zeppelin/zeppelin-display/src/main/scala/org/apache/zeppelin/display/angular/AbstractAngularElem.scala:25: error: object xml is not a member of package scala
[INFO] import scala.xml._

scala2.10版中直接有xml包，在2.11版的时候已经移除来了。因此在编译中加入Scala-2.11.

mvn clean package -Pbuild-distr,helium-dev,hadoop-2.6,r,spark-2.2,vendor-repo -Dhadoop.version=2.6.0-cdh5.10.2 -Dhbase.hbase.version=1.2.0-cdh5.10.2 -Dhbase.hadoop.version=2.6.0-cdh5.10.2 -Pscala-2.11 -DskipTests

碰到的第二个问题：zepplin-zengine模块,zepplin-srever模块下都有同样的依赖版本冲突。

Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce (enforce) on project zeppelin-zengine: org.apache.maven.plugins.enforcer.DependencyConvergence failed with mes
sage:
[ERROR] Failed while enforcing releasability the error(s) are [
[ERROR] Dependency convergence error for com.amazonaws:aws-java-sdk-core:1.10.62 paths to dependency are:
[ERROR] +-org.apache.zeppelin:zeppelin-zengine:0.8.0-SNAPSHOT
[ERROR] +-com.amazonaws:aws-java-sdk-s3:1.10.62
[ERROR] +-com.amazonaws:aws-java-sdk-kms:1.10.62
[ERROR] +-com.amazonaws:aws-java-sdk-core:1.10.62
[ERROR] and
[ERROR] +-org.apache.zeppelin:zeppelin-zengine:0.8.0-SNAPSHOT
[ERROR] +-com.amazonaws:aws-java-sdk-s3:1.10.62
[ERROR] +-com.amazonaws:aws-java-sdk-core:1.10.62
[ERROR] and
[ERROR] +-org.apache.zeppelin:zeppelin-zengine:0.8.0-SNAPSHOT
[ERROR] +-org.apache.hadoop:hadoop-client:2.6.0-cdh5.10.2
[ERROR] +-org.apache.hadoop:hadoop-aws:2.6.0-cdh5.10.2
[ERROR] +-com.amazonaws:aws-java-sdk-sts:1.10.6
[ERROR] +-com.amazonaws:aws-java-sdk-core:1.10.6

根据提示的冲突版本号，在project的pom.xml中加入：

        org.codehaus.jackson
        jackson-mapper-asl
        1.9.13


        org.codehaus.jackson
        jackson-core-asl
        1.9.13


        org.apache.zookeeper
        zookeeper
        3.4.6


        com.amazonaws
        aws-java-sdk-s3
        1.10.62


        com.amazonaws
        aws-java-sdk-core
        1.10.62

碰到的第三个问题：zepplin-web install phantomjs的时候报错。

Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.3:npm (npm install) on project zeppelin-web: Failed to run task: 'npm install --no-lockfile' failed. (error code 1) -> [Help 1]

Running 'npm install --no-lockfile' in /tmp/zeppelin/zeppelin-web
[INFO] 
[INFO] > [email protected] install /tmp/zeppelin/zeppelin-web/node_modules/phantomjs-prebuilt
[INFO] > node install.js
[INFO] 
[INFO] PhantomJS not found on PATH
[INFO] Downloading https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2
[INFO] Saving to /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
[INFO] Receiving...
[INFO] 
[INFO] Install exited unexpectedly
[WARNING] npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
[WARNING] npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
[ERROR] 
[ERROR] npm ERR! code ELIFECYCLE
[ERROR] npm ERR! errno 1
[ERROR] npm ERR! [email protected] install: `node install.js`
[ERROR] npm ERR! Exit status 1
[ERROR] npm ERR! 
[ERROR] npm ERR! Failed at the [email protected] install script.
[ERROR] npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

这个错误没找到原因，原来是在/tmp目录下编译的，后来发现空间满了移到/opt目录下编译通过了，没有这个错误了。

碰到的第四个问题：

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (default) on project zeppelin-zrinterpreter_2.11: Command execution failed. Process exited with an error: 127 (Exit value: 127) -> [Help 1]

[INFO] --- exec-maven-plugin:1.2.1:exec (default) @ zeppelin-zrinterpreter_2.11 ---
+++ dirname R/install-dev.sh
++ cd R
++ pwd
+ FWDIR=/opt/zeppelin/r/R
+ LIB_DIR=/opt/zeppelin/r/R/../../R/lib
+ mkdir -p /opt/zeppelin/r/R/../../R/lib
+ pushd /opt/zeppelin/r/R
+ R CMD INSTALL --library=/opt/zeppelin/r/R/../../R/lib /opt/zeppelin/r/R/rzeppelin/
R/install-dev.sh: line 38: R: command not found

发现需要R命令运行环境，解决方案如下：

yum install epel-release
yum install R -y

静默安装R环境。

安装R环境后，碰到下一个问题：

ERROR: dependency ‘evaluate’ is not available for package ‘rzeppelin’
* removing ‘/opt/zeppelin/R/lib/rzeppelin’

发现依赖evaluate没有，使用R进行安装。

R
>install.packages("evaluate")

 
>q()

参考资料：

http://blog.csdn.net/zhanhong39/article/details/47749023