CDH环境下Kylin集群模式安装部署

一、Kylin简介

Apache Kylin™是一个开源的分布式分析引擎，提供Hadoop/Spark之上的SQL查询接口及多维分析（OLAP）能力以支持超大规模数据，最初由eBay Inc. 开发并贡献至开源社区。它能在亚秒内查询巨大的Hive表。

二、安装前准备

软件版本：

1.apache-kylin-2.5.2-bin-cdh60.tar.gz
2.CDH6.0.0
3.JDK_1.8.0_141

安装机器：

172.17.194.17   dev_cdh_001v  all
172.17.194.18   dev_cdh_002v  query
172.17.194.19   dev_cdh_003v  query

三、集群模式安装

1.在dev_cdh_001v上解压部署apache-kylin-2.5.2-bin-cdh60.tar.gz

tar zxvf apache-kylin-2.5.2-bin-cdh60.tar.gz
cp apache-kylin-2.5.2-bin-cdh60 /usr/local

2.在dev_cdh_001v修改配置
(1)配置环境变量

vi /etc/profile

#for Kylin
export KYLIN_HOME=/usr/local/apache-kylin-2.5.2-bin-cdh60
export HBASE_HOME=/opt/cloudera/parcels/CDH/lib/hbase
export HADOOP_HOME=/opt/cloudera/parcels/CDH
export HIVE_HOME=/opt/cloudera/parcels/CDH/lib/hive
export HADOOP_CMD=/opt/cloudera/parcels/CDH/lib/hadoop/bin/hadoop
export HCAT_HOME=/opt/cloudera/parcels/CDH/lib/hive-hcatalog

使配置生效

source /etc/profile

(2)配置kylin

vi $KYLIN_HOME/conf/kylin.properties

kylin.metadata.url= kylin_metadata@hbase
kylin.server.mode=all    // 只能有一个节点是all，其他节点为query
kylin.server.cluster-servers=dev_cdh_001v:7070,dev_cdh_002v:7070,dev_cdh_003v:7070

(3)配置加载hbase common lib

vi /opt/cloudera/parcels/CDH/lib/hbase/bin/hbase

改
CLASSPATH=${CLASSPATH}:$JAVA_HOME/lib/tools.jar
成
CLASSPATH=${CLASSPATH}:$JAVA_HOME/lib/tools.jar:/opt/cloudera/parcels/CDH/lib/hbase/lib/*

3.将/usr/local/apache-kylin-2.5.2-bin-cdh60拷贝 dev_cdh_002v和 dev_cdh_003v

scp -r /usr/local/apache-kylin-2.5.2-bin-cdh60 root@dev_cdh_002v:/usr/local/
scp -r /usr/local/apache-kylin-2.5.2-bin-cdh60 root@dev_cdh_003v:/usr/local/

4.在dev_cdh_002v和dev_cdh_003v进行以下操作
(1)修改 /opt/cloudera/parcels/CDH/lib/hbase/bin/hbase

vi /opt/cloudera/parcels/CDH/lib/hbase/bin/hbase

改
CLASSPATH=${CLASSPATH}:$JAVA_HOME/lib/tools.jar
成
CLASSPATH=${CLASSPATH}:$JAVA_HOME/lib/tools.jar:/opt/cloudera/parcels/CDH/lib/hbase/lib/*

(2)修改kylin的配置

vi $KYLIN_HOME/conf/kylin.properties
kylin.server.mode=query

5.在dev_cdh_001v上的nginx上添加一下配置
(1)upstream.conf

vi /usr/local/nginx/conf/upstream.conf

upstream kylin-cluster {
ip_hash;
server 172.17.194.17:7070 max_fails=2 fail_timeout=5s;
server 172.17.194.18:7070 max_fails=2 fail_timeout=5s;
server 172.17.194.19:7070 max_fails=2 fail_timeout=5s;
}

(2)nginx.conf

vi /usr/local/nginx/conf/nginx.conf
server {
       listen       7001;
       server_name  10.240.9.132;

       default_type 'text/html';
       charset utf-8;
       access_log  /data/logs/kylin/kylin_nginx.log  main;

       location ^~ / {
                        proxy_pass http://kylin-cluster;
                        proxy_connect_timeout 60s;
                        proxy_send_timeout 1800s;
                        proxy_read_timeout 1800s;
                        proxy_ignore_client_abort on;
         }
     }

(3)reload nginx

四、启动kylin
1.检查配置依赖是否正确

 $KYLIN_HOME/bin/check-env.sh

root@dev_cdh_001v:/usr/local/nginx/conf
#  $KYLIN_HOME/bin/check-env.sh
Retrieving hadoop conf dir...
KYLIN_HOME is set to /usr/local/apache-kylin-2.5.2-bin-cdh60

2.在dev_cdh_001v，dev_cdh_002v和dev_cdh_003v启动kylin

$KYLIN_HOME/bin/kylin.sh start

3.登录kylin

http://dev_cdh_001v:7001/kylin
ADMIN/KYLIN(默认账号密码)

image.png