kylin（一）介绍 编译 与 单节点测试
文章目录
- 1. kylin的产生背景
- 1.1 传统数仓会遇到的问题
- 1.2 kylin核心设计理念的诞生的过程
- 2.kylin的发展历史
- 3.kylin技术架构
- 3.1 数据源
- 3.2 核心模块
- 4.kylin核心概念
- 4.1 维度/度量
- 4.2 cube/cuboid
- 4.3 segment
- 5.kylin目录
- 6.kylin部署架构
- 7.sample_cube体验
- 8.kylin的编译
- 8.1 nodejs安装
- 8.2 bower安装
- 8.3 phantomjs安装
- 8.4 pom.xml文件修改 cdh5.7 ==> 对应所要编译的cdh版本
- 8.5 关闭DocLint特性[可选]
- 8.6 编译报错问题排查
- 8.7 最终编译成功
- 9.运行kylin所需要环境
- 9.1 Hadoop部署
- 9.2 Hive部署
- 9.3 HBase部署
- 9.4 Kylin部署
1. kylin的产生背景
1.1 传统数仓会遇到的问题
传统的数仓架构，实际上只支持垂直扩展
Hadoop与BI平台衔接不够成熟，无法提供高效的交互式查询的
在这个背景下，eBay 2013年 BI on Hadoop
1.2 kylin核心设计理念的诞生的过程
Hive、Spark SQL等SQL on Hadoop的框架 大规模的并行处理和列式存储
1亿条记录的 耗时1分钟
10亿 ？10
随着数据的指数级的增长，查询速度是无法保证
sql：join group by sum() count() count(distinct)
对于大多数的数据表来说 维度和指标 基本上都是能够去确定的
预计算的概念 把数据计算完 存起来 ==> 查询
join group by ==> select ==> 能够节省一些计算
2.kylin的发展历史
2013年 ebay启动了BI on Hadoop
2014年9月 内部版本上线
2014年10月 正式开源
2014年11月 apache孵化器项目
2015年12月 正式成为顶级项目
2016年3月 kylingence
3.kylin技术架构
3.1 数据源
实时(kafka)/离线(Hadoop、Hive数仓)
3.2 核心模块
(1)cube构建引擎 MR/Spark
(2)元数据管理模块/工具：cube、构建作业等等 存哪里？HBase上
(3)路由模块：
kylin sql ====> hbase
问题：
1.在定义cube 选择维度、度量 很难保证线上的所有查询都会命中我们所设计的cube，对应去查hive里的数据
2.cube依赖于hive数据表，有可能存在同张hive表 ==> >=2个cube
(4)查询引擎
(5)数据存储：HBase
4.kylin核心概念
4.1 维度/度量
数据分析当中最最基本的两个概念
(1)维度：审视数据的一个角度，记录数据的一个属性：时间、地点等等
(2)度量：基于数据所计算出来的一个具体的数值
select item,sum(price) from tbl where xxx group by item
4.2 cube/cuboid
cube：n维；对于每一种维度组合，都会去做聚合运算 n种
cuboid：是用来组成cube的
4.3 segment
在kylin当中，设计完cube之后，需要去构建的
每次构建的时候，都会去选择对应的时间范围 1周的数据 segment
5.kylin目录
[hadoop@hadoop001 apache-kylin-2.6.0]$ ll
total 48
drwxr-xr-x. 2 hadoop hadoop 4096 May 17 00:13 bin
-rw-r--r--. 1 hadoop hadoop 823 Jan 13 2019 commit_SHA1
drwxr-xr-x. 2 hadoop hadoop 4096 May 16 22:51 conf
drwxrwxr-x. 2 hadoop hadoop 4096 May 4 2019 ext
drwxr-xr-x. 2 hadoop hadoop 4096 May 4 2019 lib
drwxrwxr-x. 2 hadoop hadoop 4096 May 20 04:20 logs
drwxr-xr-x. 5 hadoop hadoop 4096 Mar 22 09:12 sample_cube
-rw-rw-r--. 1 hadoop hadoop 242 May 16 23:31 SCSinkTools.json
drwxr-xr-x. 10 hadoop hadoop 4096 May 4 2019 spark
drwxrwxr-x. 7 hadoop hadoop 4096 Jan 19 09:37 system_cube
drwxr-xr-x. 9 hadoop hadoop 4096 May 4 2019 tomcat
drwxr-xr-x. 2 hadoop hadoop 4096 Jan 13 2019 tool
6.kylin部署架构
http://kylin.apache.org/docs/install/kylin_cluster.html
mode：query 用于数据查询的节点 多台查询节点做负载均衡
job 用于构建cube的节点 任务构建引擎高可用
7.sample_cube体验
[hadoop@hadoop001 bin]$ ./sample.sh
Cardinality 维度基数 cout(distinct)之后的结果 item 200
数据加载
load table 库名.表名
Load Table Metadata From Tree
Aggregation Groups
聚合组 ==> 是我们做cube优化 维度裁剪的 关键步骤 ==> 对于cube大小、构建速度 起到关键作用的
Includes 需要包含的维度
Mandatory Dimensions 强制维度/必须维度 每个查询都会带的维度 一般来说会选时间字段
Hierarchy Dimensions 层级维度 省市区关系 一/二/三/四/五...类目(电商)
Joint Dimensions 联合维度 id和name
Rowkeys
设计的好与坏 与查询HBase数据的效率 息息相关
Cube Engine
MapReduce/Spark
Advanced Dictionaries
涉及到精准去重的指标 bitmap/hll(hyperloglog) 有一定的误差，但是存储的空间比小
Advanced Snapshot Table
Advanced ColumnFamily
去重/求和
kylin部署
[kylin.properties]
kylin.env.hadoop-conf-dir 指定到具体的$HADOOP_HOME/etc/hadoop
# 动手操作
1.kylin基于cdh5.16.2编译
2.kylin部署好，sample_cube给构建起来 对比一下查Hive和查kylin的速率
8.kylin的编译
前置准备：
http://archive.cloudera.com/cdh5/cdh/5/
8.1 nodejs安装
$> wget https://nodejs.org/dist/v10.9.0/node-v10.9.0-linux-x64.tar.xz
$> tar xf node-v10.9.0-linux-x64.tar.xz
$> ln -s /usr/software/nodejs/bin/npm /usr/local/bin/
$> ln -s /usr/software/nodejs/bin/node /usr/local/bin/
$> node -v ##查看版本
8.2 bower安装
$> npm install -g bower
npm太慢， 淘宝npm镜像使用方法
https://blog.csdn.net/qq_37674858/article/details/80065701
npm --registry https://registry.npm.taobao.org install express
npm install -g bower
ln -s /usr/local/node/node-v6.17.1-linux-x64/bin/bower /usr/local/bin/bower
# 能够执行出命令
[root@hadoop003 /]# bower --allow-root -v
1.8.8
8.3 phantomjs安装
手动安装部署phantomjs即可，并配置环境变量，下载地址：
https://github.com/Medium/phantomjs/releases/download/v1.9.19/phantomjs-1.9.8-linux-x86_64.tar.bz2
yum install -y bzip2
tar -jxvf phantomjs-1.9.8-linux-x86_64.tar.bz2
安装 PhantomJS 安装过程如下
https://www.cnblogs.com/guotianqi/p/7686441.html
yum -y install wget fontconfig
# 编译报错
> [email protected] install /kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs
> node install.js
Considering PhantomJS found at /usr/local/phantomjs/phantomjs-2.1.1-linux-x86_64/bin/phantomjs
Found PhantomJS at /usr/local/phantomjs/phantomjs-2.1.1-linux-x86_64/bin/phantomjs ...verifying
PhantomJS detected, but wrong version 2.1.1 @ /usr/local/phantomjs/phantomjs-2.1.1-linux-x86_64/bin/phantomjs.
/kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/phantomjs is not writable: EACCES: permission denied, mkdir '/kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/phantomjs'
Download already available at /tmp/phantomjs/phantomjs-1.9.8-linux-x86_64.tar.bz2
Verified checksum of previously downloaded file
Extracting tar contents (via spawned process)
Removing /kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/lib/phantom
Copying extracted folder /tmp/phantomjs/phantomjs-1.9.8-linux-x86_64.tar.bz2-extract-1593327325826/phantomjs-1.9.8-linux-x86_64 -> /kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/lib/phantom
Phantom installation failed { [Error: EACCES: permission denied, link '/tmp/phantomjs/phantomjs-1.9.8-linux-x86_64.tar.bz2-extract-1593327325826/phantomjs-1.9.8-linux-x86_64' -> '/kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/lib/phantom']
errno: -13,
code: 'EACCES',
syscall: 'link',
path:
'/tmp/phantomjs/phantomjs-1.9.8-linux-x86_64.tar.bz2-extract-1593327325826/phantomjs-1.9.8-linux-x86_64',
dest:
'/kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/lib/phantom' } Error: EACCES: permission denied, link '/tmp/phantomjs/phantomjs-1.9.8-linux-x86_64.tar.bz2-extract-1593327325826/phantomjs-1.9.8-linux-x86_64' -> '/kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/lib/phantom'
npm WARN [email protected] No license field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node install.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2020-06-28T06_55_28_303Z-debug.log
# 报错解决
在Centos上安装phantomjs的过程
https://segmentfault.com/a/1190000013317848
/kylin/source/apache-kylin-2.6.0/build/script/build.sh
[root@hadoop003 script]# cat build.sh
echo 'package front-end'
cd webapp
npm install -g bower || { exit 1; }
bower --allow-root install || { exit 1; }
npm install || { exit 1; }
npm install -g grunt-cli || { exit 1; }
grunt dev --buildEnv=dev --buildNumber=`date "+%Y%m%d%H%M%S"` || { exit 1; }
# 将上面build脚本中的npm install 后面加一个--unsafe-perm
npm clean --force
npm install --unsafe-perm
npm install --unsafe-perm -g grunt-cli
8.4 pom.xml文件修改 cdh5.7 ==> 对应所要编译的cdh版本
$> sed -i "s/2.6.0-cdh5.7.0/2.6.0-cdh5.16.2/g" `grep "cdh" -rl pom.xml`
$> sed -i "s/1.1.0-cdh5.7.0/1.1.0-cdh5.16.2/g" `grep "cdh" -rl pom.xml`
$> sed -i "s/1.2.0-cdh5.7.0/1.2.0-cdh5.16.2/g" `grep "cdh" -rl pom.xml`
$> sed -i "s/3.4.5-cdh5.7.0/3.4.5-cdh5.16.2/g" `grep "cdh" -rl pom.xml`
$> sed -i "s/cdh5.7/cdh5.16/g" `grep "cdh" -rl pom.xml`
cdh5.16
2.6.0-cdh5.16.2
2.6.0-cdh5.16.2
1.1.0-cdh5.16.2
1.1.0-cdh5.16.2
1.2.0-cdh5.16.2
3.4.5-cdh5.16.2
......
CDH各个版本下载&&组件版本
https://blog.csdn.net/xiaozhaoshigedasb/article/details/89020421
5.x的CDH各个组件版本信息
https://www.cloudera.com/documentation/enterprise/release-notes/topics/rg_cdh_vd.html
6.x的CDH各个组件版本信息
https://www.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_62_packaging.html
8.5 关闭DocLint特性[可选]
Java 8 新增了DocLint特性，这个特性主要是在开发阶段生产javadoc文档之前就检查Javadoc注释的错误，并且链接到源代码；如果javadoc的注释有错误，不生产javadoc
-Xdoclint:none
8.6 编译报错问题排查
最终使用的编译命令
build/script/package.sh -Dmaven.test.skip=true -Dcheckstyle.skip -Pcdh5.16.2 -X
编译过程中的报错排查
使用
build/script/package.sh -DskipTests -Pcdh5.16.2 -Papache-release -Dcheckstyle.skip
-X
进行编译一直报错：
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-gpg-plugin:1.6:sign (sign-release-artifacts) on project kylin: Exit code: 2 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
使用build/script/package.sh -DskipTests -Pcdh5.16这个编译则没有
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (process-resource-bundles) on project kylin-engine-spark: Failed to resolve dependencies for one or more projects in the reactor. Reason: No versions are present in the repository for the artifact with a range [1.8,2.0)
[ERROR] commons-codec:commons-codec:jar:null
[ERROR]
[ERROR] from the specified remote repositories:
[ERROR] central (http://repo.maven.apache.org/maven2, releases=true, snapshots=false),
[ERROR] conjars (http://conjars.org/repo/, releases=true, snapshots=true),
[ERROR] cloudera (https://repository.cloudera.com/artifactory/cloudera-repos/, releases=true, snapshots=true),
[ERROR] shibboleth (https://build.shibboleth.net/nexus/content/repositories/releases/, releases=true, snapshots=true),
[ERROR] nexus (http://repository.kyligence.io:8081/repository/maven-public/, releases=true, snapshots=true),
[ERROR] apache.snapshots (https://repository.apache.org/snapshots, releases=false, snapshots=true),
[ERROR] sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots, releases=false, snapshots=true)
[ERROR] Path to dependency:
[ERROR] 1) org.apache.kylin:kylin-engine-spark:jar:2.6.0
[ERROR] 2) org.apache.spark:spark-core_2.11:jar:2.3.2
[ERROR] 3) net.java.dev.jets3t:jets3t:jar:0.9.4
[ERROR] -> [Help 1]
原因：无法将依赖下载下来，maven仓库中没有找到对应的文件，setting.xml中添加阿里云的maven仓库
******添加阿里云仓库即可******
nexus-aliyun
central
Nexus aliyun
http://maven.aliyun.com/nexus/content/groups/public
CN
OSChina Central
http://maven.oschina.net/content/groups/public/
central
alimaven
central
aliyun maven
https://maven.aliyun.com/nexus/content/repositories/central/
jboss-public-repository-group
central
JBoss Public Repository Group
https://repository.jboss.org/nexus/content/groups/public
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.4.0:compile (scala-compile-first) on project kylin-engine-spark: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:4.4.0:compile failed. CompileFailed -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.4.0:compile (scala-compile-first) on project kylin-engine-spark: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:4.4.0:compile failed.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:4.4.0:compile failed.
原因：在pom.xml中找到net.alchim31.maven，并取消注释(在发布时取消注释)，使用的是3.4.1版本
net.alchim31.maven
scala-maven-plugin
3.4.1
修正后继续编译，仍然报错：
Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.4.1:add-source (scala-compile-first) on project kylin-engine-spark: The plugin net.alchim31.maven:scala-maven-plugin:3.4.1 requires Maven version 3.5.3
需要将maven版本改为3.5.3
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.5.1:testCompile (default-testCompile) on project kylin-source-jdbc: Compilation failure
[ERROR] /opt/source/apache-kylin-2.6.0/source-jdbc/src/test/java/org/apache/kylin/source/jdbc/JdbcExplorerTest.java:[89,26] error: incompatible types: inferred type does not conform to upper bound(s)
[ERROR]
[ERROR] -> [Help 1]
原因：
使用错误命令`build/script/package.sh -DskipTests -Dcheckstyle.skip -Pcdh5.16`
-Dmaven.test.skip=true 不但跳过单元测试的运行，也跳过测试代码的编译
-DskipTests 跳过单元测试，但是会继续编译
应该使用`build/script/package.sh -Dmaven.test.skip=true -Dcheckstyle.skip -Pcdh5.16`
[ERROR] Failed to execute goal on project kylin-server-base: Could not resolve dependencies for project org.apache.kylin:kylin-server-base:jar:2.6.0: Failed to collect dependencies at org.springframework.security.extensions:spring-security-saml2-core:jar:1.0.2.RELEASE -> org.opensaml:opensaml:jar:2.6.6: Failed to read artifact descriptor for org.opensaml:opensaml:jar:2.6.6: Could not transfer artifact org.opensaml:opensaml:pom:2.6.6 from/to spring-snapshots (http://repo.spring.io/libs-snapshot): Access denied to: http://repo.spring.io/libs-snapshot/org/opensaml/opensaml/2.6.6/opensaml-2.6.6.pom , ReasonPhrase:Forbidden. -> [Help 1]
[ERROR] Failed to execute goal on project kylin-server-base: Could not resolve dependencies for project org.apache.kylin:kylin-server-base:jar:2.6.0: Failed to collect dependencies at org.springframework.security.extensions:spring-security-saml2-core:jar:1.0.2.RELEASE -> org.opensaml:opensaml:jar:2.6.6: Failed to read artifact descriptor for org.opensaml:opensaml:jar:2.6.6: Could not transfer artifact net.shibboleth:parent-v2:pom:4 from/to spring-snapshots (http://repo.spring.io/libs-snapshot): Access denied to: http://repo.spring.io/libs-snapshot/net/shibboleth/parent-v2/4/parent-v2-4.pom , ReasonPhrase:Forbidden. -> [Help 1]
[ERROR] Failed to execute goal on project kylin-server-base: Could not resolve dependencies for project org.apache.kylin:kylin-server-base:jar:2.6.0: Failed to collect dependencies at org.springframework.security.extensions:spring-security-saml2-core:jar:1.0.2.RELEASE -> org.opensaml:opensaml:jar:2.6.6 -> org.opensaml:openws:jar:1.5.6: Failed to read artifact descriptor for org.opensaml:openws:jar:1.5.6: Could not transfer artifact org.opensaml:openws:pom:1.5.6 from/to spring-snapshots (http://repo.spring.io/libs-snapshot): Access denied to: http://repo.spring.io/libs-snapshot/org/opensaml/openws/1.5.6/openws-1.5.6.pom , ReasonPhrase:Forbidden. -> [Help 1]
[ERROR] Failed to execute goal on project kylin-server-base: Could not resolve dependencies for project org.apache.kylin:kylin-server-base:jar:2.6.0: Failed to collect dependencies at org.springframework.security.extensions:spring-security-saml2-core:jar:1.0.2.RELEASE -> org.opensaml:opensaml:jar:2.6.6 -> org.opensaml:openws:jar:1.5.6 -> org.opensaml:xmltooling:jar:1.4.6: Failed to read artifact descriptor for org.opensaml:xmltooling:jar:1.4.6: Could not transfer artifact org.opensaml:xmltooling:pom:1.4.6 from/to spring-snapshots (http://repo.spring.io/libs-snapshot): Access denied to: http://repo.spring.io/libs-snapshot/org/opensaml/xmltooling/1.4.6/xmltooling-1.4.6.pom , ReasonPhrase:Forbidden. -> [Help 1]
解决：
手动下载https://mvnrepository.com/artifact/org.opensaml/opensaml/2.6.6 jar
手动下载https://build.shibboleth.net/nexus/content/repositories/releases/net/shibboleth/parent-v2/4/parent-v2-4.pom
手动下载https://mvnrepository.com/artifact/org.opensaml/openws/1.5.6 jar和pom
手动下载https://mvnrepository.com/artifact/org.opensaml/xmltooling/1.4.6 jar和pom
# 找不到spark依赖 添加阿里云仓库即可
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (process-resource-bundles) on project kylin-engine-spark: Failed to resolve dependencies for one or more projects in the reactor. Reason: No versions are present in the repository for the artifact with a range [1.8,2.0)
[ERROR] commons-codec:commons-codec:jar:null
[ERROR]
[ERROR] from the specified remote repositories:
[ERROR] central (http://repo.maven.apache.org/maven2, releases=true, snapshots=false),
[ERROR] conjars (http://conjars.org/repo/, releases=true, snapshots=true),
[ERROR] cloudera (https://repository.cloudera.com/artifactory/cloudera-repos/, releases=true, snapshots=true),
[ERROR] shibboleth (https://build.shibboleth.net/nexus/content/repositories/releases/, releases=true, snapshots=true),
[ERROR] nexus (http://repository.kyligence.io:8081/repository/maven-public/, releases=true, snapshots=true),
[ERROR] apache.snapshots (https://repository.apache.org/snapshots, releases=false, snapshots=true),
[ERROR] sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots, releases=false, snapshots=true)
[ERROR] Path to dependency:
[ERROR] 1) org.apache.kylin:kylin-engine-spark:jar:2.6.0
[ERROR] 2) org.apache.spark:spark-core_2.11:jar:2.3.2
[ERROR] 3) net.java.dev.jets3t:jets3t:jar:0.9.4
[ERROR]
[ERROR]
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (process-resource-bundles) on project kylin-engine-spark: Failed to resolve dependencies for one or more projects in the reactor. Reason: No versions are present in the repository for the artifact with a range [1.8,2.0)
commons-codec:commons-codec:jar:null
from the specified remote repositories:
central (http://repo.maven.apache.org/maven2, releases=true, snapshots=false),
conjars (http://conjars.org/repo/, releases=true, snapshots=true),
cloudera (https://repository.cloudera.com/artifactory/cloudera-repos/, releases=true, snapshots=true),
shibboleth (https://build.shibboleth.net/nexus/content/repositories/releases/, releases=true, snapshots=true),
nexus (http://repository.kyligence.io:8081/repository/maven-public/, releases=true, snapshots=true),
apache.snapshots (https://repository.apache.org/snapshots, releases=false, snapshots=true),
sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots, releases=false, snapshots=true)
Path to dependency:
1) org.apache.kylin:kylin-engine-spark:jar:2.6.0
2) org.apache.spark:spark-core_2.11:jar:2.3.2
3) net.java.dev.jets3t:jets3t:jar:0.9.4
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.MojoExecutionException: Failed to resolve dependencies for one or more projects in the reactor. Reason: No versions are present in the repository for the artifact with a range [1.8,2.0)
commons-codec:commons-codec:jar:null
from the specified remote repositories:
central (http://repo.maven.apache.org/maven2, releases=true, snapshots=false),
conjars (http://conjars.org/repo/, releases=true, snapshots=true),
cloudera (https://repository.cloudera.com/artifactory/cloudera-repos/, releases=true, snapshots=true),
shibboleth (https://build.shibboleth.net/nexus/content/repositories/releases/, releases=true, snapshots=true),
nexus (http://repository.kyligence.io:8081/repository/maven-public/, releases=true, snapshots=true),
apache.snapshots (https://repository.apache.org/snapshots, releases=false, snapshots=true),
sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots, releases=false, snapshots=true)
Path to dependency:
1) org.apache.kylin:kylin-engine-spark:jar:2.6.0
2) org.apache.spark:spark-core_2.11:jar:2.3.2
3) net.java.dev.jets3t:jets3t:jar:0.9.4
[WARNING] Could not transfer metadata commons-codec:commons-codec/maven-metadata.xml from/to central (http://repo.maven.apache.org/maven2): Transfer failed for http://repo.maven.apache.org/maven2/commons-codec/commons-codec/maven-metadata.xml 501 HTTPS Required
org.eclipse.aether.transfer.MetadataTransferException: Could not transfer metadata commons-codec:commons-codec/maven-metadata.xml from/to central (http://repo.maven.apache.org/maven2): Transfer failed for http://repo.maven.apache.org/maven2/commons-codec/commons-codec/maven-metadata.xml 501 HTTPS Required
at org.eclipse.aether.connector.basic.MetadataTransportListener.transferFailed (MetadataTransportListener.java:52)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run (BasicRepositoryConnector.java:369)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute (BasicRepositoryConnector.java:644)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get (BasicRepositoryConnector.java:235)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver$ResolveTask.run (DefaultMetadataResolver.java:573)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
at java.lang.Thread.run (Thread.java:748)
Caused by: org.apache.maven.wagon.TransferFailedException: Transfer failed for http://repo.maven.apache.org/maven2/commons-codec/commons-codec/maven-metadata.xml 501 HTTPS Required
at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1196)
at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1138)
at org.apache.maven.wagon.StreamWagon.getInputStream (StreamWagon.java:126)
at org.apache.maven.wagon.StreamWagon.getIfNewer (StreamWagon.java:88)
at org.apache.maven.wagon.StreamWagon.get (StreamWagon.java:61)
at org.eclipse.aether.transport.wagon.WagonTransporter$GetTaskRunner.run (WagonTransporter.java:567)
at org.eclipse.aether.transport.wagon.WagonTransporter.execute (WagonTransporter.java:435)
at org.eclipse.aether.transport.wagon.WagonTransporter.get (WagonTransporter.java:412)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask (BasicRepositoryConnector.java:457)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run (BasicRepositoryConnector.java:364)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute (BasicRepositoryConnector.java:644)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get (BasicRepositoryConnector.java:235)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver$ResolveTask.run (DefaultMetadataResolver.java:573)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
at java.lang.Thread.run (Thread.java:748)
[WARNING] Could not transfer metadata commons-codec:commons-codec/maven-metadata.xml from/to nexus (http://repository.kyligence.io:8081/repository/maven-public/): Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/maven-metadata.xml 401 Unauthorized
org.eclipse.aether.transfer.MetadataTransferException: Could not transfer metadata commons-codec:commons-codec/maven-metadata.xml from/to nexus (http://repository.kyligence.io:8081/repository/maven-public/): Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/maven-metadata.xml 401 Unauthorized
at org.eclipse.aether.connector.basic.MetadataTransportListener.transferFailed (MetadataTransportListener.java:52)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run (BasicRepositoryConnector.java:369)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute (BasicRepositoryConnector.java:644)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get (BasicRepositoryConnector.java:235)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver$ResolveTask.run (DefaultMetadataResolver.java:573)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
at java.lang.Thread.run (Thread.java:748)
Caused by: org.apache.maven.wagon.authorization.AuthorizationException: Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/maven-metadata.xml 401 Unauthorized
at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1180)
at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1138)
at org.apache.maven.wagon.StreamWagon.getInputStream (StreamWagon.java:126)
at org.apache.maven.wagon.StreamWagon.getIfNewer (StreamWagon.java:88)
at org.apache.maven.wagon.StreamWagon.get (StreamWagon.java:61)
at org.eclipse.aether.transport.wagon.WagonTransporter$GetTaskRunner.run (WagonTransporter.java:567)
at org.eclipse.aether.transport.wagon.WagonTransporter.execute (WagonTransporter.java:435)
at org.eclipse.aether.transport.wagon.WagonTransporter.get (WagonTransporter.java:412)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask (BasicRepositoryConnector.java:457)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run (BasicRepositoryConnector.java:364)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute (BasicRepositoryConnector.java:644)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get (BasicRepositoryConnector.java:235)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver$ResolveTask.run (DefaultMetadataResolver.java:573)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
at java.lang.Thread.run (Thread.java:748)
[DEBUG] Could not find metadata commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in local (/root/.m2/repository)
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in http://conjars.org/repo/ was cached in the local repository, resolution will not be reattempted until the update interval of conjars has elapsed or updates are forced
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in https://repository.cloudera.com/artifactory/cloudera-repos/ was cached in the local repository, resolution will not be reattempted until the update interval of cloudera has elapsed or updates are forced
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in https://build.shibboleth.net/nexus/content/repositories/releases/ was cached in the local repository, resolution will not be reattempted until the update interval of shibboleth has elapsed or updates are forced
[DEBUG] Skipped remote request for commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml, locally cached metadata up-to-date.
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in https://oss.sonatype.org/content/repositories/snapshots was cached in the local repository, resolution will not be reattempted until the update interval of sonatype-nexus-snapshots has elapsed or updates are forced
[DEBUG] Using transporter WagonTransporter with priority -1.0 for http://repository.kyligence.io:8081/repository/maven-public/
[DEBUG] Using connector BasicRepositoryConnector with priority 0.0 for http://repository.kyligence.io:8081/repository/maven-public/
Downloading from nexus: http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/1.15-SNAPSHOT/maven-metadata.xml
[DEBUG] Writing tracking file /root/.m2/repository/commons-codec/commons-codec/1.15-SNAPSHOT/resolver-status.properties
[WARNING] Could not transfer metadata commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml from/to nexus (http://repository.kyligence.io:8081/repository/maven-public/): Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/1.15-SNAPSHOT/maven-metadata.xml 401 Unauthorized
org.eclipse.aether.transfer.MetadataTransferException: Could not transfer metadata commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml from/to nexus (http://repository.kyligence.io:8081/repository/maven-public/): Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/1.15-SNAPSHOT/maven-metadata.xml 401 Unauthorized
at org.eclipse.aether.connector.basic.MetadataTransportListener.transferFailed (MetadataTransportListener.java:52)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run (BasicRepositoryConnector.java:369)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute (BasicRepositoryConnector.java:644)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get (BasicRepositoryConnector.java:235)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver$ResolveTask.run (DefaultMetadataResolver.java:573)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver$1.execute (DefaultMetadataResolver.java:494)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver.resolve (DefaultMetadataResolver.java:350)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver.resolveMetadata (DefaultMetadataResolver.java:181)
at org.apache.maven.repository.internal.DefaultVersionResolver.resolveVersion (DefaultVersionResolver.java:213)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom (DefaultArtifactDescriptorReader.java:204)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor (DefaultArtifactDescriptorReader.java:171)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.resolveCachedArtifactDescriptor (DefaultDependencyCollector.java:541)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.getArtifactDescriptorResult (DefaultDependencyCollector.java:524)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:412)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse (DefaultDependencyCollector.java:509)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:461)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse (DefaultDependencyCollector.java:509)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:461)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.collectDependencies (DefaultDependencyCollector.java:254)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.collectDependencies (DefaultRepositorySystem.java:284)
at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve (DefaultProjectDependenciesResolver.java:169)
at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies (LifecycleDependencyResolver.java:243)
at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies (LifecycleDependencyResolver.java:147)
at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved (MojoExecutor.java:248)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:202)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.wagon.authorization.AuthorizationException: Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/1.15-SNAPSHOT/maven-metadata.xml 401 Unauthorized
at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1180)
at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1138)
at org.apache.maven.wagon.StreamWagon.getInputStream (StreamWagon.java:126)
at org.apache.maven.wagon.StreamWagon.getIfNewer (StreamWagon.java:88)
at org.apache.maven.wagon.StreamWagon.get (StreamWagon.java:61)
at org.eclipse.aether.transport.wagon.WagonTransporter$GetTaskRunner.run (WagonTransporter.java:567)
at org.eclipse.aether.transport.wagon.WagonTransporter.execute (WagonTransporter.java:435)
at org.eclipse.aether.transport.wagon.WagonTransporter.get (WagonTransporter.java:412)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask (BasicRepositoryConnector.java:457)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run (BasicRepositoryConnector.java:364)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute (BasicRepositoryConnector.java:644)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get (BasicRepositoryConnector.java:235)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver$ResolveTask.run (DefaultMetadataResolver.java:573)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver$1.execute (DefaultMetadataResolver.java:494)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver.resolve (DefaultMetadataResolver.java:350)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver.resolveMetadata (DefaultMetadataResolver.java:181)
at org.apache.maven.repository.internal.DefaultVersionResolver.resolveVersion (DefaultVersionResolver.java:213)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom (DefaultArtifactDescriptorReader.java:204)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor (DefaultArtifactDescriptorReader.java:171)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.resolveCachedArtifactDescriptor (DefaultDependencyCollector.java:541)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.getArtifactDescriptorResult (DefaultDependencyCollector.java:524)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:412)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse (DefaultDependencyCollector.java:509)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:461)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse (DefaultDependencyCollector.java:509)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:461)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.collectDependencies (DefaultDependencyCollector.java:254)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.collectDependencies (DefaultRepositorySystem.java:284)
at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve (DefaultProjectDependenciesResolver.java:169)
at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies (LifecycleDependencyResolver.java:243)
at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies (LifecycleDependencyResolver.java:147)
at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved (MojoExecutor.java:248)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:202)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[DEBUG] Could not find metadata commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in local (/root/.m2/repository)
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in http://conjars.org/repo/ was cached in the local repository, resolution will not be reattempted until the update interval of conjars has elapsed or updates are forced
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in https://repository.cloudera.com/artifactory/cloudera-repos/ was cached in the local repository, resolution will not be reattempted until the update interval of cloudera has elapsed or updates are forced
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in https://build.shibboleth.net/nexus/content/repositories/releases/ was cached in the local repository, resolution will not be reattempted until the update interval of shibboleth has elapsed or updates are forced
[DEBUG] Skipped remote request for commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml, already updated during this session.
[WARNING] Failure to transfer commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml from http://repository.kyligence.io:8081/repository/maven-public/ was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed or updates are forced. Original error: Could not transfer metadata commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml from/to nexus (http://repository.kyligence.io:8081/repository/maven-public/): Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/1.15-SNAPSHOT/maven-metadata.xml 401 Unauthorized
org.eclipse.aether.transfer.MetadataTransferException: Failure to transfer commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml from http://repository.kyligence.io:8081/repository/maven-public/ was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed or updates are forced. Original error: Could not transfer metadata commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml from/to nexus (http://repository.kyligence.io:8081/repository/maven-public/): Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/1.15-SNAPSHOT/maven-metadata.xml 401 Unauthorized
at org.eclipse.aether.internal.impl.DefaultUpdateCheckManager.newException (DefaultUpdateCheckManager.java:336)
at org.eclipse.aether.internal.impl.DefaultUpdateCheckManager.checkMetadata (DefaultUpdateCheckManager.java:296)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver.resolve (DefaultMetadataResolver.java:302)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver.resolveMetadata (DefaultMetadataResolver.java:181)
at org.apache.maven.repository.internal.DefaultVersionResolver.resolveVersion (DefaultVersionResolver.java:213)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom (DefaultArtifactDescriptorReader.java:211)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor (DefaultArtifactDescriptorReader.java:171)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.resolveCachedArtifactDescriptor (DefaultDependencyCollector.java:541)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.getArtifactDescriptorResult (DefaultDependencyCollector.java:524)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:412)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse (DefaultDependencyCollector.java:509)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:461)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse (DefaultDependencyCollector.java:509)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:461)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.collectDependencies (DefaultDependencyCollector.java:254)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.collectDependencies (DefaultRepositorySystem.java:284)
at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve (DefaultProjectDependenciesResolver.java:169)
at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies (LifecycleDependencyResolver.java:243)
at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies (LifecycleDependencyResolver.java:147)
at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved (MojoExecutor.java:248)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:202)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[DEBUG] Skipped remote request for commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml, locally cached metadata up-to-date.
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in https://oss.sonatype.org/content/repositories/snapshots was cached in the local repository, resolution will not be reattempted until the update interval of sonatype-nexus-snapshots has elapsed or updates are forced
# bower与npm错误
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.1.2 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN [email protected] No license field.
npm ERR! Linux 3.10.0-514.26.2.el7.x86_64
npm ERR! argv "/usr/local/node/node-v6.17.1-linux-x64/bin/node" "/usr/local/bin/npm" "install"
npm ERR! node v6.17.1
npm ERR! npm v3.10.10
npm ERR! path /kylin/source/apache-kylin-2.6.0/webapp/node_modules/.staging/@types/color-name-28c4216b
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall rename
npm ERR! enoent ENOENT: no such file or directory, rename '/kylin/source/apache-kylin-2.6.0/webapp/node_modules/.staging/@types/color-name-28c4216b' -> '/kylin/source/apache-kylin-2.6.0/webapp/node_modules/tap/node_modules/@types/color-name'
npm ERR! enoent ENOENT: no such file or directory, rename '/kylin/source/apache-kylin-2.6.0/webapp/node_modules/.staging/@types/color-name-28c4216b' -> '/kylin/source/apache-kylin-2.6.0/webapp/node_modules/tap/node_modules/@types/color-name'
npm ERR! enoent This is most likely not a problem with npm itself
npm ERR! enoent and is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! Please include the following file with any support request:
npm ERR! /kylin/source/apache-kylin-2.6.0/webapp/npm-debug.log
8.7 最终编译成功
[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ kylin-tomcat-ext ---
[INFO] Installing /opt/source/apache-kylin-2.6.0/tomcat-ext/target/kylin-tomcat-ext-2.6.0.jar to /home/hadoop/.m2/repository/org/
apache/kylin/kylin-tomcat-ext/2.6.0/kylin-tomcat-ext-2.6.0.jar
[INFO] Installing /opt/source/apache-kylin-2.6.0/tomcat-ext/pom.xml to /home/hadoop/.m2/repository/org/apache/kylin/kylin-tomcatext/2.6.0/kylin-tomcat-ext-2.6.0.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Kylin 2.6.0 ................................. SUCCESS [ 6.896 s]
[INFO] Apache Kylin - Core Common ......................... SUCCESS [ 9.497 s]
[INFO] Apache Kylin - Core Metadata ....................... SUCCESS [ 11.527 s]
[INFO] Apache Kylin - Core Dictionary ..................... SUCCESS [ 7.266 s]
[INFO] Apache Kylin - Core Cube ........................... SUCCESS [ 9.569 s]
[INFO] Apache Kylin - Core Metrics ........................ SUCCESS [ 4.294 s]
[INFO] Apache Kylin - Core Job ............................ SUCCESS [ 6.162 s]
[INFO] Apache Kylin - Core Storage ........................ SUCCESS [ 4.810 s]
[INFO] Apache Kylin - MapReduce Engine .................... SUCCESS [ 11.646 s]
[INFO] Apache Kylin - Spark Engine ........................ SUCCESS [ 36.087 s]
[INFO] Apache Kylin - Hive Source ......................... SUCCESS [ 16.175 s]
[INFO] Apache Kylin - DataSource SDK ...................... SUCCESS [ 9.821 s]
[INFO] Apache Kylin - Jdbc Source ......................... SUCCESS [ 6.607 s]
[INFO] Apache Kylin - Kafka Source ........................ SUCCESS [ 7.157 s]
[INFO] Apache Kylin - Cache ............................... SUCCESS [ 5.200 s]
[INFO] Apache Kylin - HBase Storage ....................... SUCCESS [ 21.739 s]
[INFO] Apache Kylin - Query ............................... SUCCESS [ 10.917 s]
[INFO] Apache Kylin - Metrics Reporter Hive ............... SUCCESS [ 8.855 s]
[INFO] Apache Kylin - Metrics Reporter Kafka .............. SUCCESS [ 5.503 s]
[INFO] Apache Kylin - REST Server Base .................... SUCCESS [01:21 min]
[INFO] Apache Kylin - REST Server ......................... SUCCESS [01:47 min]
[INFO] Apache Kylin - JDBC Driver ......................... SUCCESS [01:29 min]
[INFO] Apache Kylin - Assembly ............................ SUCCESS [02:21 min]
[INFO] Apache Kylin - Tool ................................ SUCCESS [ 33.614 s]
[INFO] Apache Kylin - Tool Assembly ....................... SUCCESS [ 22.368 s]
[INFO] Apache Kylin - Integration Test .................... SUCCESS [01:08 min]
[INFO] Apache Kylin - Tomcat Extension 2.6.0 .............. SUCCESS [ 6.227 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12:31 min
[INFO] Finished at: 2020-05-17T04:29:36+08:00
[INFO] ------------------------------------------------------------------------
....
apache-kylin-2.6.0-bin/bin/check-port-availability.sh
apache-kylin-2.6.0-bin/bin/build-incremental-cube.sh
apache-kylin-2.6.0-bin/bin/find-spark-dependency.sh
apache-kylin-2.6.0-bin/bin/check-migration-acl.sh
apache-kylin-2.6.0-bin/bin/find-hadoop-conf-dir.sh
apache-kylin-2.6.0-bin/bin/find-hbase-dependency.sh
apache-kylin-2.6.0-bin/bin/check-env.sh
apache-kylin-2.6.0-bin/bin/set-java-home.sh
apache-kylin-2.6.0-bin/bin/sample.sh
apache-kylin-2.6.0-bin/bin/find-kafka-dependency.sh
apache-kylin-2.6.0-bin/bin/metastore.sh
apache-kylin-2.6.0-bin/bin/sample-streaming.sh
apache-kylin-2.6.0-bin/bin/diag.sh
apache-kylin-2.6.0-bin/bin/find-hive-dependency.sh
apache-kylin-2.6.0-bin/bin/kylin.sh
apache-kylin-2.6.0-bin/bin/check-hive-usability.sh
apache-kylin-2.6.0-bin/bin/system-cube.sh
Package ready: dist/apache-kylin-2.6.0-bin.tar.gz
真正意义上是编译成功了 基于cdh5.16.2
# 最终编译包在根目录 dist包下
/kylin/source/apache-kylin-2.6.0/dist/apache-kylin-2.6.0-bin.tar.gz
[root@hadoop003 dist]# pwd
/kylin/source/apache-kylin-2.6.0/dist
[root@hadoop003 dist]# ls -lah
total 337M
drwxr-xr-x 2 root root 4.0K Jun 28 15:22 .
drwxrwxrwx 36 root root 4.0K Jun 28 15:33 ..
-rw-r--r-- 1 root root 337M Jun 28 15:37 apache-kylin-2.6.0-bin.tar.gz
9.运行kylin所需要环境
9.1 Hadoop部署
目录：/opt/app/hadoop-2.6.0-cdh5.16.2/etc/hadoop
corr-site.xml
fs.defaultFS
hdfs://hadoop001:8020
hadoop.tmp.dir
/opt/app/tmp
/opt/app/tmp目录下的文件信息：
[hadoop@hadoop001 hadoop]$ cd /opt/app/tmp
[hadoop@hadoop001 tmp]$ ll
total 8
drwxrwxr-x. 5 hadoop hadoop 4096 Nov 4 2017 dfs
drwxr-xr-x. 5 hadoop hadoop 4096 May 20 08:27 nm-local-dir
hdfs-site.xml
dfs.replication
1
mapred-site.xml
mapreduce.framework.name
yarn
mapreduce.jobhistory.address
hadoop001:10020
yarn-site.xml
yarn.nodemanager.aux-services
mapreduce_shuffle
slaves
localhost
9.2 Hive部署
一般会用于大多数公司的离线数据仓库的构建，SQL on Hadoop的框架，我们可以通过写hive sql的方式去运行一个job，这个job会对应的转换为mr作业进行运行
[拓展]hive on mr/spark/tez
hive-env.sh
HADOOP_HOME=/opt/app/hadoop-2.6.0-cdh5.16.2
hive-site.xml
javax.jdo.option.ConnectionURL
jdbc:mysql://localhost:3306/ruozedata_biproject?createDatabaseIfNotExist=true
javax.jdo.option.ConnectionDriverName
com.mysql.jdbc.Driver
javax.jdo.option.ConnectionUserName
root
javax.jdo.option.ConnectionPassword
root
为什么需要配置mysql相关的内容?
hive的数据组成：data(hdfs) + meta data(mysql)
启动hive报错：
hive> show databases;
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
排查方向：去看具体的log /tmp/hadoop/hive.log
报错日志：
Caused by: org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundException: The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
at org.datanucleus.store.rdbms.connectionpool.AbstractConnectionPoolFactory.loadDriver(AbstractConnectionPoolFactory.java:58)
at org.datanucleus.store.rdbms.connectionpool.BoneCPConnectionPoolFactory.createConnectionPool(BoneCPConnectionPoolFactory.java:54)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:238)
... 81 more
原因：缺少jar包 mysql-connector-java-5.1.39-bin.jar 拷贝到$HIVE_HOME/lib下即可
修改过后启动hive仍然报错：
hive> show databases;
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
排查方向：去看具体的log /tmp/hadoop/hive.log
报错日志：
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:211)
at com.mysql.jdbc.MysqlIO.(MysqlIO.java:300)
... 89 more
原因：启动mysql
最终运行成功：
hive> show databases;
OK
default
kylin
ruozedata
test
9.3 HBase部署
hbase-env.sh
export JAVA_HOME=/opt/app/jdk1.8.0_45
如果不配置的话，启动会报错
[hadoop@hadoop001 conf]$ start-hbase.sh
192.168.26.131: +======================================================================+
192.168.26.131: | Error: JAVA_HOME is not set |
192.168.26.131: +----------------------------------------------------------------------+
192.168.26.131: | Please download the latest Sun JDK from the Sun Java web site |
192.168.26.131: | > http://www.oracle.com/technetwork/java/javase/downloads |
192.168.26.131: | |
192.168.26.131: | HBase requires Java 1.7 or later. |
192.168.26.131: +======================================================================+
starting master, logging to /opt/app/hbase-1.2.0-cdh5.16.2/logs/hbase-hadoop-master-hadoop001.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
starting regionserver, logging to /opt/app/hbase-1.2.0-cdh5.16.2/logs/hbase-hadoop-1-regionserver-hadoop001.out
hbase-site.xml
hbase.rootdir
hdfs://192.168.26.131:8020/hbase
hbase.cluster.distributed
true
hbase.zookeeper.quorum
192.168.26.131
对应的进程：HMaster、HRegionServer、HQuorumPeer
web ui：http://hadoop001:60010/master-status
9.4 Kylin部署
需要去编译源码 基于kylin的2.6.0版本 cdh5.16.2
kylin.properties
kylin.env.hadoop-conf-dir=/opt/app/hadoop-2.6.0-cdh5.16.2/etc/hadoop
启动kylin：
[hadoop@hadoop001 bin]$ ./kylin.sh start
看到如下信息输出，即启动成功：
A new Kylin instance is started by hadoop. To stop it, run 'kylin.sh stop'
Check the log at /opt/app/apache-kylin-2.6.0/logs/kylin.log
Web UI is at http://:7070/kylin
You have new mail in /var/spool/mail/hadoop
[hadoop@hadoop001 bin]$ ps -ef | grep kylin
web ui：http://hadoop001:7070/kylin/login ADMIN/KYLIN