说明:
本次部署内容为:
mysql-5.7.33-1.el7
azkaban-3.89.0
一、安装MySql数据库
1.官网下载地址:MySQL :: Download MySQL Community Server (Archived Versions)
2.下载上传至服务器并解压
3.按顺序安装
①、检查服务器是否存在mariadb,如果有,请删除
# yum remove mariadb-libs-5.5.68-1.el7.x86_64
②、按此顺序进行安装部署
Ⅰ、# rpm -ivh mysql-community-common-5.7.33-1.el7.x86_64.rpm
# sudo yum -y install mysql-community-common-5.7.33-1.el7.x86_64.rpm
Ⅱ、# rpm -ivh mysql-community-libs-5.7.33-1.el7.x86_64.rpm
# sudo yum -y install mysql-community-libs-5.7.33-1.el7.x86_64.rpm
Ⅲ、# rpm -ivh mysql-community-client-5.7.33-1.el7.x86_64.rpm
# sudo yum -y install mysql-community-client-5.7.33-1.el7.x86_64.rpm
Ⅳ、# yum install -y mysql-community-server-5.7.33-1.el7.x86_64.rpm
# sudo yum -y install mysql-community-server-5.7.33-1.el7.x86_64.rpm
4、在/etc/my.cnf 文件中加入默认字符集
default-character-ser=utf-8
5、设置开机启动mysql服务
# chkconfig --add mysqld
6、修改配置文件 /etc/my.cnf
在配置文件中添加:
skip-grant-tables
可以登录数据库后设置root用户密码,设置完成后,再删除此配置项。
7、启动mysql
# service mysqld start
8、设置修改root用户的密码
mysql> alter user 'root'@'localhost' identified by 'password';
9、创建azkaban用户以及数据库
①、创建azkaban数据库
mysql> create database azkaban character set utf8;
②、创建azkaban用户
mysql> create user 'azkaban'@'%' identified by 'azkaban';
③、授权给azkaban用户
mysql> grant all privileges on azkaban.* to azkaban;
④、刷新,使得生效
mysql> flush privileges;
二、部署azkaban
1、下载地址:https://github.com/azkaban/azkaban/releases
2、安装git、gcc-c++
# yum install -y git gcc-c++
3、编译源码
上传azkaban的包进行编译
# ./gradlew build installDist -x test
# ./gradlew build
如果编译过程失败
可能会因为get不到依赖而编译失败
尝试:修改gradlew文件,在
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script
这一行下加一行:
GRADLE_OPTS=-Dgradle.user.home=/mnt/e/temp
即指定gradle临时目录,把下载失败的各种资源 (pom文件或jar文件)直接通过浏览器下载到这个指定的临时目录中
如果还是失败,在网络好的时候多试几次
4、把编译好的azkaban的包解压
5、再mysql的azkaban数据库下执行azkaban的建表sql
/azkaban/create-all-sql-0.1.0-SNAPSHOT.sql
6、登录创建好的azkaban库
# mysql -u azkaban -p
7、执行create-all-sql-0.1.0-SNAPSHOT.sql中的建表语句:
mysql> source /home/bigdata/create-all-sql-0.1.0-SNAPSHOT.sql
8、往executors 表插入一行数据
mysql> insert into executors values (1,'localhost',3306,1);
9、配置web和exec下的conf/azkaban.propertie
①、web
# Azkaban Personalization Settings
azkaban.name=Test
azkaban.label=My Local Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
default.timezone.id=America/Los_Angeles
# Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=conf/azkaban-users.xml
# Loader for projects
executor.global.properties=conf/global.properties
azkaban.project.dir=projects
# Velocity dev mode
velocity.dev.mode=false
# Azkaban Jetty server properties.
jetty.use.ssl=false
jetty.maxThreads=25
jetty.port=8081
# Azkaban Executor settings
# mail settings
mail.sender=
mail.host=
# User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users.
# enduser -> myazkabanhost:443 -> proxy -> localhost:8081
# when this parameters set then these parameters are used to generate email links.
# if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.
# azkaban.webserver.external_hostname=myazkabanhost.com
# azkaban.webserver.external_ssl_port=443
# azkaban.webserver.external_port=8081
job.failure.email=
job.success.email=
lockdown.create.projects=false
cache.directory=cache
# JMX stats
jetty.connector.stats=true
executor.connector.stats=true
# Azkaban mysql settings by default. Users should configure their own username and password.
database.type=mysql
mysql.port=3306
mysql.host=localhost
mysql.database=azkaban
mysql.user=azkaban
mysql.password=azkaban
mysql.numconnections=100
#Multiple Executor
azkaban.use.multiple.executors=true
azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus
azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1
azkaban.executorselector.comparator.Memory=1
azkaban.executorselector.comparator.LastDispatched=1
azkaban.executorselector.comparator.CpuUsage=1
②、exec
# Azkaban Personalization Settings
azkaban.name=Test
azkaban.label=My Local Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
default.timezone.id=America/Los_Angeles
# Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=conf/azkaban-users.xml
# Loader for projects
executor.global.properties=conf/global.properties
azkaban.project.dir=projects
# Velocity dev mode
velocity.dev.mode=false
# Azkaban Jetty server properties.
jetty.use.ssl=false
jetty.maxThreads=25
jetty.port=8081
# Where the Azkaban web server is located
azkaban.webserver.url=http://localhost:8081
# mail settings
mail.sender=
mail.host=
# User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users.
# enduser -> myazkabanhost:443 -> proxy -> localhost:8081
# when this parameters set then these parameters are used to generate email links.
# if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.
# azkaban.webserver.external_hostname=myazkabanhost.com
# azkaban.webserver.external_ssl_port=443
# azkaban.webserver.external_port=8081
job.failure.email=
job.success.email=
lockdown.create.projects=false
cache.directory=cache
# JMX stats
jetty.connector.stats=true
executor.connector.stats=true
# Azkaban plugin settings
azkaban.jobtype.plugin.dir=plugins/jobtypes
# Azkaban mysql settings by default. Users should configure their own username and password.
database.type=mysql
mysql.port=3306
mysql.host=localhost
mysql.database=azkaban
mysql.user=azkaban
mysql.password=azkaban
mysql.numconnections=100
# Azkaban Executor settings
executor.maxThreads=50
executor.flow.threads=30
10、启动azkaban
①、先启动exec /azkaban/azkaban-exec-server-0.1.0-SNAPSHOT/bin/start-exec.sh
# ./bin/start-exec.sh
②、再启动web /azkaban/azkaban-exec-server-0.1.0-SNAPSHOT/bin/start-web.sh
# ./bin/start-web.sh
③、可以查看对应的日志,是否启动成功
azkaban-exec-server和azkaban-web-server下的 logs/azkaban-XXXserver.log
11、验证azkaban登录网站
http://localhost:8081