cdh6.3.2安装apache-kylin-3.0.2-bin-cdh60

首先，由于本人是cdh6.3.2，故点击下载选择下图cdh60包

[root@cdh632-worker03 ~]# cd /opt/soft/
[root@cdh632-worker03 soft]# tar zxf apache-kylin-3.0.2-bin-cdh60.tar.gz -C /opt/module/
[root@cdh632-worker03 soft]# cd /opt/module/apache-kylin-3.0.2-bin-cdh60/
为解决报错01：
[root@cdh632-worker03 ~]# vi /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/hbase/bin/hbase
# CLASSPATH=${CLASSPATH}:$JAVA_HOME/lib/tools.jar
# 将上面的注释掉修改为如下
CLASSPATH=${CLASSPATH}:$JAVA_HOME/lib/tools.jar:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/hbase/lib/*
[root@cdh632-worker03 apache-kylin-3.0.2-bin-cdh60]# vi ~/.bashrc
# kylin3.0.2
# 为解决报错02
export HADOOP_USER_NAME=hdfs
# 为解决报错03
export SPARK_HOME=/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/spark
export KYLIN_HOME=/opt/module/apache-kylin-3.0.2-bin-cdh60
[root@cdh632-worker03 apache-kylin-3.0.2-bin-cdh60]# $KYLIN_HOME/bin/check-env.sh
Retrieving hadoop conf dir...
报错01：
Error: Could not find or load main class org.apache.hadoop.hbase.util.GetJavaProperty
KYLIN_HOME is set to /opt/module/apache-kylin-3.0.2-bin-cdh60
报错02：
mkdir: Permission denied: user=root, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
Failed to create /kylin. Please make sure the user has right to access /kylin
[root@cdh632-worker03 apache-kylin-3.0.2-bin-cdh60]# source ~/.bashrc
为启动kylin做准备，修改配置文件：
[root@cdh632-worker03 ~]# vi $KYLIN_HOME/conf/kylin.properties
# 设置为开发环境，便于查看一些新的特性
kylin.env=DEV
### 若多机器部署，可以把第二台机器设置为kylin.server.mode=query等
## Kylin server mode, valid value [all, query, job]
#kylin.server.mode=all
kylin.server.cluster-servers=cdh632-worker03:7070
kylin.web.timezone=GMT+8
#kylin.stream.receiver.use-threads-per-query=8
#每个查询使用四个线程进行扫描
kylin.stream.receiver.use-threads-per-query=4
[root@cdh632-worker03 apache-kylin-3.0.2-bin-cdh60]# $KYLIN_HOME/bin/kylin.sh start
Retrieving hadoop conf dir...
KYLIN_HOME is set to /opt/module/apache-kylin-3.0.2-bin-cdh60
Retrieving hive dependency...
Retrieving hbase dependency...
Hadoop 3.0.0-cdh6.3.2
Retrieving hadoop conf dir...
Retrieving kafka dependency...
Retrieving Spark dependency...
报错03：
spark not found, set SPARK_HOME, or run bin/download-spark.sh
若成功启动，则提示如下：
A new Kylin instance is started by root. To stop it, run 'kylin.sh stop'
Check the log at /opt/module/apache-kylin-3.0.2-bin-cdh60/logs/kylin.log
Web UI is at http://cdh632-worker03:7070/kylin
按照其提示，此时查看日志
[root@cdh632-worker03 ~]# vi /opt/module/apache-kylin-3.0.2-bin-cdh60/logs/kylin.log
发现报错：
2020-06-25 12:57:54,410 WARN  [localhost-startStop-1] support.XmlWebApplicationContext:551 : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping': Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: org/apache/commons/configuration/ConfigurationException
2020-06-25 12:57:54,416 ERROR [localhost-startStop-1] context.ContextLoader:350 : Context initialization failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping': Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: org/apache/commons/configuration/ConfigurationException
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1628)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
        at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
        at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
        at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4699)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5165)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:743)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:719)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:705)
        at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:970)
        at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1841)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/configuration/ConfigurationException
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
        at java.lang.Class.getDeclaredMethods(Class.java:1975)
        at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:613)
        at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:524)
        at org.springframework.core.MethodIntrospector.selectMethods(MethodIntrospector.java:68)
        at org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.detectHandlerMethods(AbstractHandlerMethodMapping.java:230)
        at org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.initHandlerMethods(AbstractHandlerMethodMapping.java:214)
        at org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.afterPropertiesSet(AbstractHandlerMethodMapping.java:184)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping.afterPropertiesSet(RequestMappingHandlerMapping.java:127)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1687)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1624)
        ... 25 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.configuration.ConfigurationException
        at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1358)
        at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1180)
        ... 37 more

经查issues虽报错不完全相同，但可以察觉是缺jar包，参考网友《kylin启动打不开web UI》，于是解决方法为

[root@cdh632-worker03 apache-kylin-3.0.2-bin-cdh60]# find / -name commons-configuration*
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/hadoop/client/commons-configuration2-2.1.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/hadoop/client/commons-configuration2.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/hadoop/lib/commons-configuration2-2.1.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/parquet/lib/commons-configuration2-2.1.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/search/lib/commons-configuration2-2.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/search/lib/search-crunch/commons-configuration2-2.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/solr/server/solr-webapp/webapp/WEB-INF/lib/commons-configuration2-2.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/kite/lib/commons-configuration2-2.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/sentry/lib/commons-configuration2-2.1.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/hbase-solr/lib/commons-configuration2-2.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/oozie/lib/commons-configuration2-2.1.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/oozie/embedded-oozie-server/webapp/WEB-INF/lib/commons-configuration2-2.1.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/oozie/libtools/commons-configuration2-2.1.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/oozie/oozie-sharelib-yarn/lib/git/commons-configuration2-2.1.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/oozie/oozie-sharelib-yarn/lib/hcatalog/commons-configuration2-2.1.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/oozie/oozie-sharelib-yarn/lib/pig/commons-configuration2-2.1.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/hadoop-hdfs/lib/commons-configuration2-2.1.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/spark/jars/commons-configuration2-2.1.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/hbase/lib/commons-configuration2-2.1.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib/impala/lib/commons-configuration2-2.1.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/jars/commons-configuration2-2.1.1.jar
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/jars/commons-configuration2-2.1.jar
/opt/cloudera/cm/cloudera-navigator-audit-server/commons-configuration-1.9.jar
/opt/cloudera/cm/cloudera-navigator-server/jars/commons-configuration-1.9.jar
/opt/cloudera/cm/cloudera-navigator-server/libs/cdh6/commons-configuration2-2.1.1.jar
/opt/cloudera/cm/cloudera-scm-telepub/jars/commons-configuration-1.9.jar
/opt/cloudera/cm/cloudera-scm-telepub/libs/cdh6/commons-configuration2-2.1.1.jar
/opt/cloudera/cm/common_jars/commons-configuration-1.9.cf57559743f64f0b3a504aba449c9649.jar
/opt/cloudera/cm/common_jars/commons-configuration2-2.1.1.0e1d8ed885ef559731c58f49f370fd6e.jar
/opt/cloudera/cm/lib/cdh6/commons-configuration2-2.1.1.jar
/opt/cloudera/cm/lib/commons-configuration-1.9.jar
/opt/module/apache-atlas-sources-2.0.0/addons/storm-bridge/target/dependency/hook/storm/atlas-storm-plugin-impl/commons-configuration2-2.2.jar
/opt/module/apache-atlas-sources-2.0.0/addons/storm-bridge/target/dependency/hook/storm/atlas-storm-plugin-impl/commons-configuration-1.10.jar
/opt/module/apache-atlas-sources-2.0.0/addons/hbase-bridge/target/dependency/hook/hbase/atlas-hbase-plugin-impl/commons-configuration-1.10.jar
/opt/module/apache-atlas-sources-2.0.0/addons/hive-bridge/target/dependency/hook/hive/atlas-hive-plugin-impl/commons-configuration-1.10.jar
/opt/module/apache-atlas-sources-2.0.0/addons/kafka-bridge/target/dependency/hook/kafka/atlas-kafka-plugin-impl/commons-configuration-1.10.jar
/opt/module/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-kafka-hook/apache-atlas-kafka-hook-2.0.0/hook/kafka/atlas-kafka-plugin-impl/commons-configuration-1.10.jar
/opt/module/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-hive-hook/apache-atlas-hive-hook-2.0.0/hook/hive/atlas-hive-plugin-impl/commons-configuration-1.10.jar
/opt/module/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-bin/apache-atlas-2.0.0/hook/storm/atlas-storm-plugin-impl/commons-configuration2-2.2.jar
/opt/module/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-bin/apache-atlas-2.0.0/hook/storm/atlas-storm-plugin-impl/commons-configuration-1.10.jar
/opt/module/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-bin/apache-atlas-2.0.0/hook/hive/atlas-hive-plugin-impl/commons-configuration-1.10.jar
/opt/module/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-bin/apache-atlas-2.0.0/hook/hbase/atlas-hbase-plugin-impl/commons-configuration-1.10.jar
/opt/module/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-server/apache-atlas-2.0.0/server/webapp/atlas/WEB-INF/lib/commons-configuration-1.10.jar
/opt/module/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-server/apache-atlas-2.0.0/server/webapp/atlas/WEB-INF/lib/commons-configuration2-2.1.1.jar
/opt/module/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-server/apache-atlas-2.0.0/hook/hive/atlas-hive-plugin-impl/commons-configuration-1.10.jar
/opt/module/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-storm-hook/apache-atlas-storm-hook-2.0.0/hook/storm/atlas-storm-plugin-impl/commons-configuration2-2.2.jar
/opt/module/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-storm-hook/apache-atlas-storm-hook-2.0.0/hook/storm/atlas-storm-plugin-impl/commons-configuration-1.10.jar
/opt/module/apache-atlas-sources-2.0.0/distro/target/apache-atlas-2.0.0-hbase-hook/apache-atlas-hbase-hook-2.0.0/hook/hbase/atlas-hbase-plugin-impl/commons-configuration-1.10.jar
/opt/module/apache-atlas-sources-2.0.0/notification/target/dependency/hook/kafka-topic-setup/commons-configuration-1.10.jar
/opt/module/apache-atlas-sources-2.0.0/webapp/target/atlas-webapp-2.0.0/WEB-INF/lib/commons-configuration-1.10.jar
/opt/module/apache-atlas-sources-2.0.0/webapp/target/atlas-webapp-2.0.0/WEB-INF/lib/commons-configuration2-2.1.1.jar
/opt/apache-atlas-2.0.0/hook/hive/atlas-hive-plugin-impl/commons-configuration-1.10.jar
/opt/apache-atlas-2.0.0/hook/hbase/atlas-hbase-plugin-impl/commons-configuration-1.10.jar
/opt/apache-atlas-2.0.0/hook/storm/atlas-storm-plugin-impl/commons-configuration2-2.2.jar
/opt/apache-atlas-2.0.0/hook/storm/atlas-storm-plugin-impl/commons-configuration-1.10.jar
/yarn/nm/filecache/10/3.0.0-cdh6.3.2-mr-framework.tar.gz/commons-configuration2-2.1.1.jar
/yarn/nm/usercache/hdfs/filecache/15/__spark_libs__8545128267598157076.zip/commons-configuration2-2.1.1.jar
/yarn/nm/usercache/hdfs/filecache/19/__spark_libs__4380644035805851635.zip/commons-configuration2-2.1.1.jar
/yarn/nm/usercache/hdfs/filecache/23/__spark_libs__5818005512619487656.zip/commons-configuration2-2.1.1.jar
/yarn/nm/usercache/hdfs/filecache/27/__spark_libs__4877829929501500423.zip/commons-configuration2-2.1.1.jar
/yarn/nm/usercache/hdfs/filecache/31/__spark_libs__3407831335315735358.zip/commons-configuration2-2.1.1.jar
/yarn/nm/usercache/hdfs/filecache/35/__spark_libs__3620082684691436317.zip/commons-configuration2-2.1.1.jar
/yarn/nm/usercache/hdfs/filecache/39/__spark_libs__7765409084065077980.zip/commons-configuration2-2.1.1.jar
考虑到我之前配置的Hive On Spark，那么先尝试spark的commons-configuration2-2.1.1.jar却失败了（报错相同）
[root@cdh632-worker03 ~]# $KYLIN_HOME/bin/kylin.sh stop
Stopping Kylin: 34619
Kylin with pid 34619 has been stopped.
[root@cdh632-worker03 ~]# cp /yarn/nm/filecache/10/3.0.0-cdh6.3.2-mr-framework.tar.gz/commons-configuration2-2.1.1.jar $KYLIN_HOME/lib/
于是老老实实使用commons-configuration-1.9.jar吧
[root@cdh632-worker03 ~]# $KYLIN_HOME/bin/kylin.sh stop
[root@cdh632-worker03 ~]# cp /opt/cloudera/cm/lib/commons-configuration-1.9.jar $KYLIN_HOME/lib/
[root@cdh632-worker03 ~]# $KYLIN_HOME/bin/kylin.sh start

按照启动控制台日志上的提示，打开首页：http://cdh632-worker03:7070/kylin，输入账号/密码：ADMIN/KYLIN

注意：首页url必须带kylin否则还会报404错误，账号和密码均为大写。

参考：

CDH 6.3.1 部署 Kylin 3.0.1

kylin在启动后web ui无法打开

问题一：由于本人Hive On Spark，在创建cube时“Advanced Setting”的“Cube Engine”处Engine Type应选Spark而非默认的mapreduce，但选择spark后，“actions”选择build后状态显示ERROR，此时发现运行到第七步“Build Cube with Spark”报错如下：

20/06/26 11:56:09 ERROR spark.SparkContext: Error initializing SparkContext.
java.lang.IllegalArgumentException: Required executor memory (4096), overhead (1024 MB), and PySpark memory (0 MB) is above the max threshold (2048 MB) of this cluster! Please check the values of 'yarn.scheduler.maximum-allocation-mb' and/or 'yarn.nodemanager.resource.memory-mb'.

解决方案：按照日志中的提示，在yarn页面的“配置”处搜索此两个参数并将其设置为5G（两个都要修改且至少为5G）。

问题二：kylin官网文档Use Dashboard显示此功能在2.3.0版本之后就支持了，但我按官网配置后chart显示报错，待解决。。。

[root@cdh632-worker03 ~]# $KYLIN_HOME/bin/kylin.sh stop
[root@cdh632-worker03 ~]# vi /opt/module/apache-kylin-3.0.2-bin-cdh60/conf/kylin.properties
随便找一个web设置处添加：kylin.web.dashboard-enabled=true
[root@cdh632-worker03 ~]# $KYLIN_HOME/bin/kylin.sh start