Apache DolphinScheduler版本2.0.5分布式集群的安装
目录
- 1. 部署模式说明
- 2. DolphinScheduler分布式集群部署
-
- 2.1 服务器部署规划
- 2.2 下载解压(dolphin1上)
- 2.3 修改conf/config/install_config.conf(dolphin1上)
- 2.4 创建元数据库和添加jar包(dolphin1上)
- 2.5 初始化元数据库(dolphin1上)
- 2.6 启动DolphinScheduler(dolphin1上)
- 2.7 登录Web页面
1. 部署模式说明
DolphinScheduler支持多种部署模式，包括单机模式(Standalone)、伪集群模式(Pseudo-Cluster)、集群模式(Cluster)等
单机模式:
单机模式(standalone)模式下，所有服务均集中于一个StandaloneServer进程中，并且其中内置了注册中心Zookeeper和数据库H2。只需配置JDK环境，就可一键启动DolphinScheduler，快速体验其功能
伪集群模式:
伪集群模式(Pseudo-Cluster)是在单台机器部署DolphinScheduler各项服务，该模式下master、worker、api server、logger server等服务都只在同一台机器上，但位于不同的进程。Zookeeper和数据库需单独安装并进行相应配置
集群模式:
集群模式(Cluster)与伪集群模式的区别就是在多台机器部署DolphinScheduler各项服务，并且Master、Worker等服务可配置多个
2. DolphinScheduler分布式集群部署
2.1 服务器部署规划
3台服务器之间，配置无密码登录。这样部署脚本就可以通过scp的方式将安装需要的资源传输到其它服务器上
| 服务 | 服务器 | 备注 |
|---|---|---|
| Java | dolphin1、dolphin2、dolphin3 | 参考centos7同时安装java8和openJdk11、windows同时安装java8和openJdk11 |
| Zookeeper | dolphin1、dolphin2、dolphin3 | 参考基于Centos7分布式安装Zookeeper3.6.3 |
| Mysql | dolphin3 | centos7安装mysql8.0.25版本 |
| psmisc | dolphin1、dolphin2、dolphin3 | Centos7安装命令：yum install psmisc |
| masters | dolphin1、dolphin2、dolphin3 | |
| workers | dolphin1、dolphin2、dolphin3 | |
| alertServer | dolphin1 | |
| apiServers | dolphin2 | |
| pythonGatewayServers | dolphin3 |
2.2 下载解压(dolphin1上)
[root@dolphin1 ~]# wget https://downloads.apache.org/dolphinscheduler/2.0.5/apache-dolphinscheduler-2.0.5-bin.tar.gz
[root@dolphin1 ~]#
[root@dolphin1 ~]# tar -zxvf apache-dolphinscheduler-2.0.5-bin.tar.gz
[root@dolphin1 ~]#
[root@dolphin1 ~]# cd apache-dolphinscheduler-2.0.5-bin
[root@dolphin1 apache-dolphinscheduler-2.0.5-bin]#
2.3 修改conf/config/install_config.conf(dolphin1上)
修改如下：
# ---------------------------------------------------------
# INSTALL MACHINE
# ---------------------------------------------------------
# ips为DolphinScheduler集群的所有服务器
ips="dolphin1,dolphin2,dolphin3"
masters="dolphin1,dolphin2,dolphin3"
workers="dolphin1:default,dolphin2:default,dolphin3:default"
alertServer="dolphin1"
apiServers="dolphin2"
pythonGatewayServers="dolphin3"
# 路径不存在，会自动创建
installPath="/root/dolphinscheduler-2.0.5-install"
deployUser="root"
# ---------------------------------------------------------
# DolphinScheduler ENV
# ---------------------------------------------------------
javaHome="/root/jdk1.8.0_201"
# ---------------------------------------------------------
# Database
# ---------------------------------------------------------
DATABASE_TYPE="mysql"
SPRING_DATASOURCE_URL="jdbc:mysql://dolphin3:3306/dolphinScheduler?useUnicode=true&characterEncoding=UTF-8"
SPRING_DATASOURCE_USERNAME="root"
SPRING_DATASOURCE_PASSWORD="Root_123"
# ---------------------------------------------------------
# Registry Server
# ---------------------------------------------------------
registryServers="dolphin1:2181,dolphin2:2181,dolphin3:2181"
2.4 创建元数据库和添加jar包(dolphin1上)
连接Mysql数据库，创建DolphinScheduler的数据库
mysql> create database dolphinScheduler default character set utf8 default collate utf8_general_ci;
Query OK, 1 row affected, 2 warnings (0.05 sec)
mysql>
下载Mysql的驱动包，到DolphinScheduler的lib目录下
[root@dolphin1 ~]# wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.25/mysql-connector-java-8.0.25.jar -P apache-dolphinscheduler-2.0.5-bin/lib
[root@dolphin1 ~]#
2.5 初始化元数据库(dolphin1上)
运行脚本，进行元数据库的初始化
[root@dolphin1 ~]# sh apache-dolphinscheduler-2.0.5-bin/script/create-dolphinscheduler.sh
2.6 启动DolphinScheduler(dolphin1上)
必须在apache-dolphinscheduler-2.0.5-bin目录下，运行install.sh脚本
[root@dolphin1 apache-dolphinscheduler-2.0.5-bin]#
[root@dolphin1 apache-dolphinscheduler-2.0.5-bin]# pwd
/root/apache-dolphinscheduler-2.0.5-bin
[root@dolphin1 apache-dolphinscheduler-2.0.5-bin]#
[root@dolphin1 apache-dolphinscheduler-2.0.5-bin]# sh install.sh
[root@dolphin1 apache-dolphinscheduler-2.0.5-bin]#
会根据ips和installPath参数，自动在所有服务器上创建目录，并上传安装所需要的资源
日志文件存放在每台服务器的/root/dolphinscheduler-2.0.5-install/logs目录下
通过以下命令查询所有服务的状态
[root@dolphin1 ~]# sh dolphinscheduler-2.0.5-install/bin/status-all.sh
[root@dolphin1 ~]#
可能有的服务没有启动，先stop所有服务，再start所有服务
[root@dolphin1 ~]#
[root@dolphin1 ~]# sh dolphinscheduler-2.0.5-install/bin/stop-all.sh
[root@dolphin1 ~]#
[root@dolphin1 ~]# sh dolphinscheduler-2.0.5-install/bin/start-all.sh
[root@dolphin1 ~]#
如果还有服务没启动起来，可以到对应的服务器运行命令，启动单个服务，参考如下：
# 启停Master
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start master-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop master-server
# 启停Worker
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start worker-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop worker-server
# 启停Api
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start api-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop api-server
# 启停Logger
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start logger-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop logger-server
# 启停Alert
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start alert-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop alert-server
# 启停Python Gateway
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh start python-gateway-server
sh dolphinscheduler-2.0.5-install/bin/dolphinscheduler-daemon.sh stop python-gateway-server
2.7 登录Web页面
登录http://dolphin2:12345/dolphinscheduler，默认的账号/密码是admin/dolphinscheduler123。登录后的界面如下所示
其中数据源中心，能定义各种数据库的连接，这样SQL任务就能进行引用了