Kylo集群安装详解

---

说明

• 集群由dpnode05和dpnode08两个节点组成

---

>>>下面在dpnode05上操作

创建kylo安装目录

• mkdir /opt/kylo

将装装包解压到安装目录

• tar xvf /home/wcm/sources/kylo-0.10.0.tar -C /opt/kylo

安装前执行预处理脚本, 这里以root用户安装

• /opt/kylo/setup/install/post-install.sh /opt/kylo root root

PS: 三个参数依次为: kylo安装目录, 安装程序的用户, 安装程序的用户组

修改配置文件

• vim /opt/kylo/kylo-services/conf/application.properties

  spring.datasource.url=jdbc:mysql://dpnode05:3306/kylo
  spring.datasource.username=root
  spring.datasource.password=123456
  spring.datasource.driverClassName=com.mysql.jdbc.Driver
  

复制Mysql驱动

• cp /home/wcm/mysql-connector-java-8.0.15.jar /opt/kylo/kylo-services/lib/

生成sql脚本

• cd /opt/kylo/setup/sql/mysql
• ./setup-mysql.sh dpnode05 root 123456
• cd /opt/kylo/setup/sql
• ./generate-update-ql.sh

  # 生成的两个脚本文件
  kylo-db-update-script.sql
  kylo-db-update-script.sql.bac
  

连接Mysql客户端执行Kylo sql脚本

use kylo;
source kylo-db-update-script.sql;

下载Quartz

• 下载Quartz: http://d2zwv9pap9ylyd.cloudfront.net/quartz-2.2.3-distribution.tar.gz
• 解压: tar -zxvf quartz-2.2.3-distribution.tar.gz
• 进入Quartz解压目录: cd quartz-2.2.3/docs/dbTables/
• 进入Mysql客户端执行脚本Quartz sql脚本

  use kylo;
  source tables_mysql.sql;
  

集群配置

• ModeShape配置

  # vim /opt/kylo/kylo-services/conf/metadata-repository.json
  # 在最后一个追加如下内容
  
  ,"clustering": {
      "clusterName":"kylo-modeshape-cluster",
      "configuration":"modeshape-jgroups-config.xml",
      "locking":"db"
  }
  

• Kylo配置

  # vim  /opt/kylo/kylo-services/conf/application.properties
  # 最后添加如下内容
  
  kylo.cluster.jgroupsConfigFile=kylo-cluster-jgroups-config.xml
  kylo.cluster.nodeCount=2
  
  # 修改activemq jms连接
  jms.activemq.broker.url=tcp://dpnode05:61616
  
  # 修改elasticsearch jms连接
  config.elasticsearch.jms.url=tcp://dpnode05:61616
  
  # 修改nifi配置
  nifi.rest.host=dpnode05
  nifi.rest.port=8079
  

• Elasticsearch配置

  # vim /opt/kylo/kylo-services/conf/elasticsearch-rest.properties
  # 修改连接配置
  search.rest.host=dpnode05
  search.rest.port=9200
  

• Quartz配置

  # cp /opt/kylo/setup/config/kylo-cluster/quartz-cluster-example.properties /opt/kylo/kylo-services/conf/quartz.properties
  

• kylo-cluster-jgroups-config

  # cp /opt/kylo/setup/config/kylo-cluster/kylo-cluster-jgroups-config-example.xml /opt/kylo/kylo-services/conf/kylo-cluster-jgroups-config.xml
  # vim /opt/kylo/kylo-services/conf/kylo-cluster-jgroups-config.xml
  

• modeshape-jgroups-config

  # cp /opt/kylo/setup/config/kylo-cluster/modeshape-local-test-jgroups-config.xml /opt/kylo/kylo-services/conf/modeshape-jgroups-config.xml
  # vim modeshape-jgroups-config.xml
  

• 启动脚本配置

  # vim /opt/kylo/kylo-services/bin/run-kylo-services.sh
  # 添加启动参数 -Djava.net.preferIPv4Stack=true
  java -Djava.net.preferIPv4Stack=true -Dorg.springframework.boot.logging.LoggingSystem=none $KYLO_SERVICES_OPTS $KYLO_SPRING_PROFILES_OPTS -cp /opt/kylo/kylo-services/conf:$HADOOP_CONF_DIR:/opt/kylo/kylo-services/lib/*:/opt/kylo/kylo-services/lib/${KYLO_NIFI_PROFILE}/*:/opt/kylo/kylo-services/plugin/* com.thinkbiganalytics.server.KyloServerApplication --pgrep-marker=kylo-services-pgrep-marker > /var/log/kylo-services/std.out 2>/var/log/kylo-services/std.err &
  

添加集群服务监控依赖插件

• cp /opt/kylo/setup/plugins/kylo-service-monitor-kylo-cluster-0.10.0.jar /opt/kylo/kylo-services/plugin/

通信测试(可省略)

• dpnode05 上执行, 阻塞并接受消息

  java -Djava.net.preferIP4Stack=true -cp  /opt/kylo/kylo-services/conf:/opt/kylo/kylo-services/lib/*:/opt/kylo/kylo-services/plugin/* org.jgroups.tests.McastReceiverTest -bind_addr dpnode05 -port 7900
  

• dpnode08 上执行, 发送消息

  java -Djava.net.preferIP4Stack=true -cp  /opt/kylo/kylo-services/conf:/opt/kylo/kylo-services/lib/*:/opt/kylo/kylo-services/plugin/* org.jgroups.tests.McastSenderTest -bind_addr dpnode08 -port 7900
  

PS: dpnode05能收到消息则正常

复制节点

• cd /opt
• scp -r /opt dpnode08://$PWD

---

>>>下面在dpnode08上操作

执行预处理脚本, 这里以root用户安装

• /opt/kylo/setup/install/post-install.sh /opt/kylo root root

集群配置

• 启动脚本配置

  # vim /opt/kylo/kylo-services/bin/run-kylo-services.sh
  # 添加启动参数 -Djava.net.preferIPv4Stack=true
  java -Djava.net.preferIPv4Stack=true -Dorg.springframework.boot.logging.LoggingSystem=none $KYLO_SERVICES_OPTS $KYLO_SPRING_PROFILES_OPTS -cp /opt/kylo/kylo-services/conf:$HADOOP_CONF_DIR:/opt/kylo/kylo-services/lib/*:/opt/kylo/kylo-services/lib/${KYLO_NIFI_PROFILE}/*:/opt/kylo/kylo-services/plugin/* com.thinkbiganalytics.server.KyloServerApplication --pgrep-marker=kylo-services-pgrep-marker > /var/log/kylo-services/std.out 2>/var/log/kylo-services/std.err &
  

• kylo-cluster-jgroups-config

  # cp /opt/kylo/setup/config/kylo-cluster/kylo-cluster-jgroups-config-example.xml /opt/kylo/kylo-services/conf/kylo-cluster-jgroups-config.xml
  # vim /opt/kylo/kylo-services/conf/kylo-cluster-jgroups-config.xml
  

• modeshape-jgroups-config

  # cp /opt/kylo/setup/config/kylo-cluster/modeshape-local-test-jgroups-config.xml /opt/kylo/kylo-services/conf/modeshape-jgroups-config.xml
  # vim modeshape-jgroups-config.xml
  

---

>>>下面操作分别在两个节点上执行

启动服务

• kylo-services start

在浏览器访问kylo-ui

• http://dpnode05:8400/index.html#!/dashboard

• http://dpnode08:8400/index.html#!/dashboard

参考文章:

• https://www.jianshu.com/p/1575f9bf7840

问题

• Admin -> Connectors -> Hive/JDBC/... 访问不了
  • 原因: kylo 访问控制分为两层, service-level(kylo-wide) 和 entity-level, entity-level层权限控制默认是关闭的! 而这个UI选项就属于entity层权限,所以访问不了
  • 解决方法
    • 修改 /opt/kylo/kylo-services/conf/application.properties
    • 修改属性 security.entity.access.controlled=true
    • 重启服务即可

---

集群原理

Modeshape uses jgroups internally for Kylo Cluster management.

Modeshape Replicated with a shared database. Modeshape Cluster

This is the only clustering model which will be supported by ModeShape 5.

A cluster in this model can have any number of members each with it's own in-memory cache but all using a shared database for persisting and reading the content. Binary stores and Indexes can be configured to be either local to each member or shared across all members, depending on the chosen implementation.

Updates in the cluster are sent to each of the members in the form of JGroups messages representing the various events that caused that data to mutate. Each cluster member will update their own local state in response to these events.

This works great for small- to medium-sized repositories, even when the available memory on each process is not large enough to hold all of the nodes and binary values at one time.