Kylin实践（二）--Cube构建

接上篇：Kylin实践（一）--Hadoop环境搭建

https://blog.csdn.net/isscici06/article/details/80624797

-----------------------------------------------------------------------------------------------------------------------------------------------------------

集群搭建好后，各项服务都正常启动，看似没问题，但在cube构建过程中，还是出现了不少问题，以下详细记录一下：

1、用stop-all.sh关闭hadoop，再用start-all.sh打开hadoop的方式重启hadoop，重启后，启动kylin报如下错误：

Exception in thread "main" java.lang.IllegalArgumentException: Failed to find metadata store by url: kylin_metadata@hbase

经排查，是因为hadoop关闭后，会将HBASE也关掉，启动后，应该用jps查看一下，是否有HMaster和HRegionServer进程，没有的话就应该重新将HBASE启起来，直接 执行./start-hbase.sh启动hbase就行。

 

2.启动kylin后，无法构建cube，在step 1就报错，而且有时有报错信息，有时连报错信息都没有，只能看kylin的日志，错误如下：

Number of reduce tasks is set to 0 since there's no reduce operator

java.io.IOException: org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException: Invalid resource request, requested memory < 0, or requested memory > max configured, requestedMemory=1536, maxMemory=1024

at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.validateResourceRequest(SchedulerUtils.java:272)

at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.normalizeAndValidateRequest(SchedulerUtils.java:228)

at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.normalizeAndValidateRequest(SchedulerUtils.java:236)

at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.validateAndCreateResourceRequest(RMAppManager.java:385)

at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:330)

at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication(RMAppManager.java:282)

at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitApplic。。。。。

主要的原因是yarn-site.xml中的资源分配配置有问题，分配太少，之前yarn.nodemanager.resource.memory-mb这一项配的是768，yarn.scheduler.maximum-allocation-mb没有配，默认值是1024.

在yarn-site.xml中添加如下配置后，问题得以解决：

   

        yarn.nodemanager.resource.memory-mb

        2048

    

    

        yarn.scheduler.maximum-allocation-mb

        2048

    

 

 

3.

java.lang.NoClassDefFoundError: org/apache/hadoop/hive/ql/session/SessionState

at org.apache.kylin.source.hive.HiveClientFactory.getHiveClient(HiveClientFactory.java:27)

at org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.computeRowCount(HiveMRInput.java:271)

at org.apache.kylin.source.hive.HiveMRInput$RedistributeFlatHiveTableStep.doWork(HiveMRInput.java:316)

at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)

at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:65)

at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:125)

at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:141)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)

Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.session.SessionState

at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1858)

at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1701)

... 10 more

解决方法：

修改bin下面的kylin.sh

在HBASE_CLASSPATH中添加“:${hive_dependency}”,如下图所示：

网上看到说在HBASE_CLASSPATH_PREFIX中添加hive依赖，如下图所示，如果再有类似问题，修改HBASE_CLASSPATH解决不了的话，可以试试修改HBASE_CLASSPATH_PREFIX。

 

4.Exception: java.net.ConnectException: Call From Centos7/10.0.86.95 to master:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused  java.net.ConnectException: Call From Centos7/10.0.86.95 to master:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused

解决方法：

参考网页：https://www.cnblogs.com/luogankun/p/4019303.html

这是jobhistory的权限不够，通过http://master:19888/jobhistory/job/job_1528353326049_0002也访问不了job页面。

history-server启动之后，在hdfs上会生成两个目录，用如下命令进行查看：

hadoop fs -ls /history

 

drwxrwx--- - spark supergroup 0 2014-10-11 15:11 /history/done

drwxrwxrwt - spark supergroup 0 2014-10-11 15:16 /history/done_intermediate

在本集群中，完整的目录为/tmp/hadoop-yarn/staging/history/done、/tmp/hadoop-yarn/staging/history/done_intermediate。

通过hadoop dfs -chmod -R 755 /tmp将/tmp文件夹的权限修改为755问题就解决了。

原以为问题到此就解决了，但是后来又出问题了，原因还是tmp的权限问题，/tmp/hive 755权限不够，如下错误提示所示：

OS command error exit with 1 -- hive -e "USE default;

DROP TABLE IF EXISTS default.kylin_intermediate_kylin_sales_cube_00b46d2c_fde8_4e10_8343_4d8a201c28bd;

" --hiveconf hive.merge.mapredfiles=false --hiveconf hive.auto.convert.join=true --hiveconf dfs.replication=2 --hiveconf hive.exec.compress.output=true --hiveconf hive.auto.convert.join.noconditionaltask=true --hiveconf mapreduce.job.split.metainfo.maxsize=-1 --hiveconf hive.merge.mapfiles=false --hiveconf hive.auto.convert.join.noconditionaltask.size=100000000 --hiveconf hive.stats.autogather=true

 

Logging initialized using configuration in jar:file:/home/hive/apache-hive-1.2.2-bin/lib/hive-common-1.2.2.jar!/hive-log4j.properties

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rwxr-xr-x

at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)

at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)

at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

直接将/tmp的权限改成777了，这样cube总算构建成功。至此，从集群搭建到cube建立顺利走完。