背景
因业务需求， 想通过 ClouderaManager ( 以下简称 CM) 管理、监控自定义服务
准备工作
安装 CM 扩展工具
工具地址
https://github.com/cloudera/cm_ext
安装时需跳过单元测试
mvn install -DskipTests
操作步骤
创建 Extensions (Parcel & CSD)
Parcel
职责
负责分发部署安装包
原理
1. 当parcel 处于可用状态，建立符号链接，以便被系统识别(二进制或文件)
2. 为 CM管理的进程设置扩展的环境变量，当 CM启动一个已经配置好的Parcel 进程时，CM Agent将获取 Parcel 提供的shell脚本，此时脚本可以设置或修改正在启动进程的环境变量
目录结构
├── ECHO-1.0
│ └── meta
│ ├── echo_env.sh
│ └── parcel.json
├── ECHO-1.0-el6.parcel
├── ECHO-1.0-el6.parcel.sha1
└── manifest.json
创建步骤
创建 Parcel
Parcel 名称拟为 ECHO-1.0
mkdir -p ECHO-1.0/meta
创建 parcel.json
{
"schema_version":1,
"name":"ECHO",
"version":"1.0",
"setActiveSymlink":true,
"conflicts":"",
"provides":[
"ECHO"
],
"scripts":{
"defines":"echo_env.sh"
},
"packages":[
{
"name":"echo",
"version":"1.0"
}
],
"components":[
{
"name":"echo",
"version":"1.0",
"pkg_version":"1.0",
"pkg_release":"1.0"
}
],
"users":{},
"groups":[]
}
创建echo_env.sh
#!/bin/bash
ECHO_DIRNAME=${PARCEL_DIRNAME}
export CDH_ECHO_HOME=$PARCELS_ROOT/$ECHO_DIRNAME
验证文件
java -jar cm_ext/validator/target/validator.jar -p ECHO-1.0/meta/parcel.json
验证目录
java -jar cm_ext/validator/target/validator.jar -d ECHO-1.0
打包 parcel
tar czvf ECHO-1.0-el6.parcel ECHO-1.0 —owner=root —group=root
验证 parcel 包
java -jar cm_ext/validator/target/validator.jar -f ECHO-1.0-el6.parcel
对外提供 Parcel 服务
mkdir parcelService && cd parcelService
python cm_ext/make_manifest/make_manifest.py ./parcelService
把 parcel 包以及 manifest 放到 parcelServer目录
启动服务
python -m SimpleHTTPServer 8001
或以 Apache、Nginx 对外提供服务
CSD ( Custom Service Descriptor )
职责
负责管理和监控
目录结构
├── ECHO-1.0.jar
├── descriptor
│ └── service.sdl
└── scripts
└── control.sh
SDL ( Service Descriptor Language ) 模板
https://github.com/cloudera/cm_ext/wiki/Service-Descriptor-Language-Reference
创建步骤
mkdir csd && cd csd
mkdir descriptor
创建 service.sdl
{
"name" : "ECHO",
"label" : "Echo",
"description" : "The echo service",
"version" : "1.0",
"runAs" : {
"user" : "root",
"group" : "root"
},
"roles" : [
{
"name" : "ECHO_WEBSERVER",
"label" : "Web Server",
"pluralLabel" : "Web Servers",
"parameters" : [
{
"name" : "port_num",
"label" : "Webserver port",
"description" : "The web server port number",
"required" : "true",
"type" : "port",
"default" : 8080
}
],
"startRunner" : {
"program" : "scripts/control.sh",
"args" : [ "start" ],
"environmentVariables" : {
"WEBSERVER_PORT" : "${port_num}"
}
}
}
]
}
mkdir scripts
创建 control.sh
#!/bin/bash
CMD=$1
case $CMD in
(start)
echo "Starting the web server on port [$WEBSERVER_PORT]"
exec python -m SimpleHTTPServer $WEBSERVER_PORT
;;
(*)
echo "Don't understand [$CMD]"
;;
esac
验证
java -jar cm_ext/validator/target/validator.jar -s csd/descriptor/service.sdl
打jar包
jar -cvf ECHO-1.0.jar *
cp -a ECHO-1.0.jar cm主机/opt/cloudera/csd
重启服务
service cloudera-scm-server restart
参考
https://github.com/cloudera/cm_ext/wiki/CSD-Primer
界面操作
进入 CM
点击右上角 Parcels (礼物形状的 Icon)
点击配置
新增远程 Parcel 存储库 URL
URL必须配置域名，IP 地址无效
更新
重启服务
更新 CSD jar 包
监控
基线监控 Baseline Monitoring (自带，无需在 server.sdl 中配置)
程序指标(句柄、CPU、Memory等)
程序健康测试(进程停止提醒等)
图表
角色和主机指标
创建仪表盘并将其保存为角色页面的默认值
支持的系统版本
el5: Redhat Enterprise Linux 5 and clones (CentOS, Scientific Linux, etc)
el6: Redhat Enterprise Linux 6 and clones (CentOS, Scientific Linux, etc)
el7: Redhat Enterprise Linux 7 and clones (CentOS, Scientific Linux, etc)
sles11: SuSE Linux Enterprise Server 11.x
lucid: Ubuntu Linux 10.04 LTS (No CDH 5.x parcel provided)
precise: Ubuntu Linux 12.04 LTS
trusty: Ubuntu Linux 14.04 LTS (Newly supported in CM 5.2. No CDH 4.x parcel provided)
squeeze: Debian 6.x (No CDH 5.x parcel provided))
wheezy: Debian 7.x (Newly supported in CM 5.0. No CDH 4.x parcel provided)
命名需要加上系统版本
CDH-5.0.0-0.cdh5b1.p0.57-squeeze.parcel
IMPALA-1.2.3-1.p0.97-el6.parcel
GPLEXTRAS-5.5.0-1.cdh5.5.0.p0.7-el7.parcel
参考文档
https://github.com/cloudera/cm_ext
https://github.com/cloudera/cm_csds
如有错误，欢迎指正~