一 应用场景描述
网站的可用性监控对于运维来说是一项重要的工作。网站监控主要关心几个要点：域名解析时间、域名响应时间、域名访问响应代码、域名访问速度等。这里的域名应该是一个专门用于网站监控检测的URL。网站的可用性监控和网站性能监控有所区别，网站可用性监控只是简单地了解网站是否可用，网站性能监控就是要从用户的角度出发去深层挖掘从用户域名解析开始知道用户数据进入数据库的整个流程中各个环节的性能指标，也就是现在谈得最多的APM。性能监控不在本文的讨论之中。既然是网站的可用性监控，那么部署多个外网监控点就非常有必要，但是一般的公司没有那么多不同区域的服务器，所以没法从多个区域去监控网站的可用性，这种情况只能借助于第三方类似监控宝这种服务，因为他们有更多的监控点可以采集到不同的区域的访问情况，这个也是第三方工具的优势。
Zabbix自带有Web监控服务，但是如果人工批量添加域名监控就很麻烦，解决办法有三个：
1.编写脚本，直接通过修改数据来批量监控域名
2.编写脚本，通过Zabbix的Web监控API来批量监控域名
3.编写脚本，结合Zabbix agent或者sender来批量监控域名
二 Zabbix web监控相关源代码分析
1.分析Zabbix前端php代码以及SQL语句
Zabbix前端关于web监控的php代码有：
httpconf.php
httpdetails.php
httpmon.php
include/httptest.inc.php
include/classes/api/managers/CHttpTestManager.php
和web监控相关的MySQL表结构如下：
mysql> desc httpstep; +------------------+---------------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +------------------+---------------------+------+-----+---------+-------+ | httpstepid | bigint(20) unsigned | NO | PRI | NULL | | | httptestid | bigint(20) unsigned | NO | MUL | NULL | | | name | varchar(64) | NO | | | | | no | int(11) | NO | | 0 | | | url | varchar(2048) | NO | | | | | timeout | int(11) | NO | | 15 | | | posts | text | NO | | NULL | | | required | varchar(255) | NO | | | | | status_codes | varchar(255) | NO | | | | | variables | text | NO | | NULL | | | follow_redirects | int(11) | NO | | 1 | | | retrieve_mode | int(11) | NO | | 0 | | | headers | text | NO | | NULL | | +------------------+---------------------+------+-----+---------+-------+ 13 rows in set (0.01 sec)
mysql> desc httpstepitem; +----------------+---------------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------------+---------------------+------+-----+---------+-------+ | httpstepitemid | bigint(20) unsigned | NO | PRI | NULL | | | httpstepid | bigint(20) unsigned | NO | MUL | NULL | | | itemid | bigint(20) unsigned | NO | MUL | NULL | | | type | int(11) | NO | | 0 | | +----------------+---------------------+------+-----+---------+-------+ 4 rows in set (0.00 sec)
mysql> desc httptest; +------------------+---------------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +------------------+---------------------+------+-----+---------+-------+ | httptestid | bigint(20) unsigned | NO | PRI | NULL | | | name | varchar(64) | NO | | | | | applicationid | bigint(20) unsigned | YES | MUL | NULL | | | nextcheck | int(11) | NO | | 0 | | | delay | int(11) | NO | | 60 | | | status | int(11) | NO | MUL | 0 | | | variables | text | NO | | NULL | | | agent | varchar(255) | NO | | Zabbix | | | authentication | int(11) | NO | | 0 | | | http_user | varchar(64) | NO | | | | | http_password | varchar(64) | NO | | | | | hostid | bigint(20) unsigned | NO | MUL | NULL | | | templateid | bigint(20) unsigned | YES | MUL | NULL | | | http_proxy | varchar(255) | NO | | | | | retries | int(11) | NO | | 1 | | | ssl_cert_file | varchar(255) | NO | | | | | ssl_key_file | varchar(255) | NO | | | | | ssl_key_password | varchar(64) | NO | | | | | verify_peer | int(11) | NO | | 0 | | | verify_host | int(11) | NO | | 0 | | | headers | text | NO | | NULL | | +------------------+---------------------+------+-----+---------+-------+ 21 rows in set (0.05 sec)
mysql> desc httptestitem; +----------------+---------------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------------+---------------------+------+-----+---------+-------+ | httptestitemid | bigint(20) unsigned | NO | PRI | NULL | | | httptestid | bigint(20) unsigned | NO | MUL | NULL | | | itemid | bigint(20) unsigned | NO | MUL | NULL | | | type | int(11) | NO | | 0 | | +----------------+---------------------+------+-----+---------+-------+ 4 rows in set (0.01 sec)
根据httptestid 删除httptest相关的历史数据 deleteHistoryByHttpTestIds
现在假设想要删除一个web scenario，查找它的httptestid为14，一个web scenario对应唯一一个httptestid
a.先找出所有的itemid
SELECT hti.itemid FROM httptestitem hti WHERE httptestid=14 UNION ALL SELECT hsi.itemid FROM httpstep hs,httpstepitem hsi WHERE hs.httpstepid=hsi.httpstepid AND httptestid=14; +--------+ | itemid | +--------+ | 235244 | | 235245 | | 235246 | | 235247 | | 235248 | | 235249 | +--------+
注意这里使用的UNION ALL而不是UNION，UNION会去除掉重复的列，UNION ALL不会去重。
b.再删除所有的itemid
调用deleteHistoryByItemIds函数
DELETE FROM trends_uint WHERE itemid IN (235244,235245,235246,235247,235248,235249); DELETE FROM history_text WHERE itemid IN (235244,235245,235246,235247,235248,235249); DELETE FROM history_log WHERE itemid IN (235244,235245,235246,235247,235248,235249); DELETE FROM history_uint WHERE itemid IN (235244,235245,235246,235247,235248,235249); DELETE FROM history_str WHERE itemid IN (235244,235245,235246,235247,235248,235249); DELETE FROM history WHERE itemid IN (235244,235245,235246,235247,235248,235249);
这样这个httptest的相关历史数据就清掉了
根据httptestid查看httptest全部数据 get_httptest_by_httptestid
SELECT ht.* FROM httptest ht WHERE ht.httptestid=14\G
根据no查看httpstep get_httpstep_by_no
SELECT hs.* FROM httpstep hs WHERE hs.httptestid=14 AND hs.no=1\G
根据hostid查看主机所有的web监控 get_httptests_by_hostid
SELECT DISTINCT ht.* FROM httptest ht WHERE hostid=10328\G
这里需要特别说明一下，Zabbix将Template也当作一个host，当添加一个Template的时候会自动创建一个hostid，有了这个hostid才能创建模板内的Application,Trigger,Items等
获取httptests的parent templates
如果是通过模板为主机创建的Web监控，那么主机的Web监控就有个Parent web scenarios
SELECT ht.httptestid,ht.templateid,ht.hostid,h.name FROM httptest ht INNER JOIN hosts h ON h.hostid=ht.hostid ; +------------+------------+--------+------------------------+ | httptestid | templateid | hostid | name | +------------+------------+--------+------------------------+ | 1 | NULL | 10120 | Template Test URL | | 5 | NULL | 10120 | Template Test URL | | 8 | NULL | 10120 | Template Test URL | | 11 | NULL | 10120 | Template Test URL | | 14 | 1 | 10328 | cdn01 | | 15 | 5 | 10328 | cdn01 | | 16 | 8 | 10328 | cdn01 | | 17 | 11 | 10328 | cdn01 | | 18 | 1 | 10361 | proxy | | 19 | 5 | 10361 | proxy | | 20 | 8 | 10361 | proxy | | 21 | 11 | 10361 | proxy | +------------+------------+--------+------------------------+ 12 rows in set (0.00 sec)
可以看出Template本身也是一个host，name就是Template的名字，如果其他主机是使用的Template模板添加Web监控，那么就会有一个templateid,这个templateid是属于Web监控这个Template的
SELECT ht.httptestid,ht.templateid,ht.hostid,h.name FROM httptest ht INNER JOIN hosts h ON h.hostid=ht.hostid ; WHERE httptestid=14;
获取所有的http step items
SELECT i.value_type,i.valuemapid,i.units,i.itemid,hi.type AS httpitem_type,hs.httpstepid FROM items i,httpstepitem hi,httpstep hs WHERE hi.itemid=i.itemid AND hi.httpstepid=hs.httpstepid AND hs.httptestid=14;
2.分析Zabbix 后端C代码以及SQL语句
Web监控的代码位于这个目录下 src/zabbix_server/httppoller
Zabbix 执行Web监控的时候需要libcurl
三 Zabbix批量监控Web
1.直接修改数据库来批量添加web监控
Zabbix数据库有张ids表，这个表是用来记录存储各种表当前最大的id，下一个要创建的id就应该加1
mysql> desc ids; +------------+---------------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +------------+---------------------+------+-----+---------+-------+ | table_name | varchar(64) | NO | PRI | | | | field_name | varchar(64) | NO | PRI | | | | nextid | bigint(20) unsigned | NO | | NULL | | +------------+---------------------+------+-----+---------+-------+ 3 rows in set (0.00 sec)
获取当前最大的id
SELECT nextid FROM ids WHERE table_name='httptest' AND field_name='httptestid';
创建一个新的Web监控的httptestid应该加1
UPDATE ids SET nextid=nextid+1 WHERE table_name='httptest' AND field_name='httptestid';
创建一个Web监控需要一个hostid，如果将Web监控放到一个Application下，就需要再创建一个Application。假设放到名为URL TEST这个Application
获取当前最大的applicationid
SELECT nextid FROM ids WHERE table_name='applications' AND field_name='applicationid';
Zabbix的applications表用于存放各个host的applications
mysql> desc applications; +---------------+---------------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +---------------+---------------------+------+-----+---------+-------+ | applicationid | bigint(20) unsigned | NO | PRI | NULL | | | hostid | bigint(20) unsigned | NO | MUL | NULL | | | name | varchar(255) | NO | | | | | flags | int(11) | NO | | 0 | | +---------------+---------------------+------+-----+---------+-------+ 4 rows in set (0.06 sec)
INSERT INTO applications(applicationid,hostid,name) VALUES(9189,12308,'URL TEST');
这个hostid为12308的host就创建了一个名为URL TEST的Application
创建一个httptest
INSERT INTO httptest(httptestid,name,applicationid,delay,status,variables,agent,authentication,hostid,headers) VALUES(22,'www.baidu.com',9191,30,0,'',"Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)",0,10328,'');
httptestid 就是查找出ids表中的httptest的nextid加1，需要注意每次插入数据之前必须要先查一下这个ids表，要不然id可能分配错误
name 就是这个Web监控的名称，可以直接写成www.baidu.com 或者 check www.baidu.com 随便怎么写
applicationid 需要把这个Web监控放到哪个Application下，就需要哪个Application的applicationid
delay 监控间隔时间，默认是60s
status 是否开启监控，对应页面上显示就是Enabled或者Disabled,默认值是0(Enabled)
variables web监控变量设置，不能为NULL，如果没有就设置为空''
agent 使用哪种浏览器类型去检测
authentication 验证类型，0-none,1-basic HTTP authentication，2-NTLM authentication
获取当前最大的httpstepid
SELECT nextid FROM ids WHERE table_name='httpstep' AND field_name='httpstepid';
然后更新ids的nextid为当前值加1
UPDATE ids SET nextid=nextid+1 WHERE table_name='httpstep' AND field_name='httpstepid';
添加一个httpstep
INSERT INTO httpstep(httpstepid,httptestid,name,no,url,timeout,posts,required,variables,headers,status_codes) VALUES(22,22,'url index',1,'http://www.baidu.com/index.php',60,'','','','',200);
name 是这个step的名称
url 是这个步骤需要访问的详细url
timeout 执行步骤的超时时间，默认是60秒
posts HTTP POST变量
required http响应中必须要包含的内容
variables step中的变量设置
headers 设置HTTP头
status_codes 允许的HTTP状态代码，例如200,302,404
创建了httptest和httpstep后，下面就要为它们添加相应的items，如果不添加Zabbix执行Web监控的时候找不到对应的item，就不会执行
httpstep需要3个item
Download speed web.test.in Bps
Response time web.test.time
Response code web.test.rspcode
httptest需要3个item
Download speed web.test.in Bps
Failed step web.test.fail
Last error message web.test.error
添加httptest和httpstep的items涉及到4张表
ids
httptestitem
httpstepitem
items
先查出当前最大的id，然后再加1
SELECT nextid FROM ids WHERE table_name='items' AND field_name='itemid'; UPDATE ids SET nextid=nextid+1 WHERE table_name='items' AND field_name='itemid';
没添加一个item之前重复执行这两个SQL语句
INSERT INTO items(itemid, type, hostid, name,description , key_, delay, history, trends, status, value_type, trapper_hosts, units, multiplier, delta, formula, error,logtimefmt, valuemapid, delay_flex, params, data_type, mtime) VALUES (235886, 9, 10328, 'Download speed for step "$2" of scenario "$1".','', 'web.test.in[www.baidu.com,url index,bps]', 60, 30, 90, 0, 0, '', 'Bps', 0, 0, '', '', '',NULL,'', '',0, 0);
INSERT INTO items(itemid, type, hostid, name,description , key_, delay, history, trends, status, value_type, trapper_hosts, units, multiplier, delta, formula, error,logtimefmt, valuemapid, delay_flex, params, data_type, mtime) VALUES (235887, 9, 10328, 'Response time for step "$2" of scenario "$1".','', 'web.test.time[www.baidu.com,url index,resp]', 60, 30, 90, 0, 0, '', 's', 0, 0, '', '', '',NULL,'', '',0, 0)
INSERT INTO items(itemid, type, hostid, name,description , key_, delay, history, trends, status, value_type, trapper_hosts, units, multiplier, delta, formula, error,logtimefmt, valuemapid, delay_flex, params, data_type, mtime) VALUES (235888, 9, 10328, 'Response code for step "$2" of scenario "$1".','', 'web.test.rspcode[www.baidu.com,url index]', 60, 30, 90, 0, 0, '', '', 0, 0, '', '', '',NULL,'', '',0, 0);
查找当前最大的httpstepitemid，添加一个step之前就加1
SELECT nextid FROM ids WHERE table_name='httpstepitem' AND field_name='httpstepitemid'; UPDATE ids SET nextid=nextid+1 WHERE table_name='httpstepitem' AND field_name='httpstepitemid';
INSERT INTO httpstepitem(httpstepitemid, httpstepid, itemid, type) VALUES(64,22,235886,2); INSERT INTO httpstepitem(httpstepitemid, httpstepid, itemid, type) VALUES(65,22,235887,1); INSERT INTO httpstepitem(httpstepitemid, httpstepid, itemid, type) VALUES(66,22,235888,0);
Type of the item.
Possible values:
0 - Zabbix agent;
1 - SNMPv1 agent;
2 - Zabbix trapper;
3 - simple check;
4 - SNMPv2 agent;
5 - Zabbix internal;
6 - SNMPv3 agent;
7 - Zabbix agent (active);
8 - Zabbix aggregate;
9 - web item;
10 - external check;
11 - database monitor;
12 - IPMI agent;
13 - SSH agent;
14 - TELNET agent;
15 - calculated;
16 - JMX agent;
17 - SNMP trap.
Type of information of the item.
Possible values:
0 - numeric float;
1 - character;
2 - log;
3 - numeric unsigned;
4 - text.
Value that will be stored.
Possible values:
0 - (default) as is;
1 - Delta, speed per second;
2 - Delta, simple change.
INSERT INTO items(itemid, type, hostid, name,description , key_, delay, history, trends, status, value_type, trapper_hosts, units, multiplier, delta, formula, error,logtimefmt, valuemapid, delay_flex, params, data_type, mtime) VALUES (235889, 9, 10328, 'Download speed for scenario "$1".','', 'web.test.in[www.baidu.com,,bps]', 60, 30, 90, 0, 0, '', 'Bps', 0, 0, '', '', '',NULL,'', '',0, 0);
INSERT INTO items(itemid, type, hostid, name,description , key_, delay, history, trends, status, value_type, trapper_hosts, units, multiplier, delta, formula, error,logtimefmt, valuemapid, delay_flex, params, data_type, mtime) VALUES (235890, 9, 10328, 'Failed step of scenario "$1".','', 'web.test.fail[www.baidu.com]', 60, 30, 90, 0, 0, '', '', 0, 0, '', '', '',NULL,'', '',0, 0);
INSERT INTO items(itemid, type, hostid, name,description , key_, delay, history, trends, status, value_type, trapper_hosts, units, multiplier, delta, formula, error,logtimefmt, valuemapid, delay_flex, params, data_type, mtime) VALUES (235891, 9, 10328, 'Last error message of scenario "$1".','', 'web.test.error[www.baidu.com]', 60, 30, 90, 0, 0, '', '', 0, 0, '', '', '',NULL,'', '',0, 0);
然后更新httptestitem表
查找当前最大的httptestitemid,然后加1
SELECT nextid FROM ids WHERE table_name='httptestitem' AND field_name='httptestitemid'; UPDATE ids SET nextid=nextid+1 WHERE table_name='httptestitem' AND field_name='httptestitemid';
INSERT INTO httptestitem(httptestitemid, httptestid, itemid, type) VALUES(64,22,235889,2); INSERT INTO httptestitem(httptestitemid, httptestid, itemid, type) VALUES(65,22,235890,3); INSERT INTO httptestitem(httptestitemid, httptestid, itemid, type) VALUES(66,22,235891,4);
好了，整个Web监控的数据库操作就完成了，现在应该可以在Zabbix页面看到监控图了
假设有很多网站需要监控，每个网站有不同的URL，现在就可以编写脚本来实现批量监控
baidu 1 index.php https://www.baidu.com/index.php baidu 2 index.htm https://www.baidu.com/index.html baidu 3 index.htm https://www.baidu.com/index.htm taobao 1 index https://world.taobao.com/?spm=a21bp.8077467.1417485807582.1.Jx0OM4 taobao 2 login https://world.taobao.com/markets/all/login tencent 1 index http://www.tencent.com/zh-cn/index.shtml tencent 2 about http://www.tencent.com/zh-cn/at/abouttencent.shtml
第一列就是Web scenarios的名称，第二列是检查步骤，第三列是执行步骤名称，第四列是检查的URL，可以根据自己需要再添加几列，比如执行步骤变量，返回码等
脚本根据https://www.zabbix.org/wiki/Docs/howto/script_web_checks 这个改编而来
add_web_checks.sh
#!/bin/bash
######
# This script adds WEB checks from a file to a zabbix host by inputting the urls directly into the database."
# test on zabbix database 3.0
# written by John Wang on 28/07/2016
# this script is based on https://www.zabbix.org/wiki/Docs/howto/script_web_checks which is not suitable for zabbix 3.0
### warning !!!!!!! this script can be dangerous as it directly operate the zabbix database,so before execute this script,please ensure that you know this risk!!!!!
PROGNAME=$(basename $0)
###### Helper functions
function usage
{
echo "This script adds WEB checks from a file to a zabbix host or multiple hosts in a group by inputting the urls directly into the database."
echo "USE WITH CARE!!"
echo ""
echo "usage: ${PROGNAME}"
echo ""
echo "MySQL connection options:"
echo "-u | --dbuser mysql username to connect to zabbix database (Required)"
echo "-p | --dbpass mysql password to connect to zabbix database (Required)"
echo "-H | --dbhost hostname of the mysql server (Default: localhost)"
echo "-D | --dbname databasename of the zabbix mysql database (Default: zabbix)"
echo ""
echo "Webcheck options:"
echo "-t | --timeout timeout in seconds before a http request times out (Default: 60)"
echo "-d | --delay number of seconds between two WEB checks (Default: 60)"
echo "-i | --history number of days to keep all values (Default: 90)"
echo "-t | --trends number of days to keep trends (Default: 360)"
echo ""
echo "-o | --hostid the zabbix hostid of the host these WEB checks will be added to(only required one of hostid,groupid and groupname)"
echo "-g | --groupid the zabbix hostid of the host these WEB checks will be added to(only required one of hostid,groupid and groupname)"
echo "-G | --groupname the zabbix hostid of the host these WEB checks will be added to(only required one of hostid,groupid and groupname)"
echo "-A | --appname zabbix application the WEB checks will be added to (Required)"
echo "-a | --agent user agent the WEB check will be done as (Default: 'Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)')"
echo ""
echo "-f | --urlfile file with urls to monitor. One url per line (Required)"
echo "--delete delete web checks"
echo ""
echo "Help:"
echo "-h | --help this message"
}
function error_exit
{
#
## ----------------------------------------------------------------
## Function for exit due to fatal program error
## Accepts 1 argument:
## string containing descriptive error message
## Example call of the error_exit function. Note the inclusion
## of the LINENO environment variable. It contains the current
## line number.
##
# error_exit "$LINENO: An error has occurred."
##
## ----------------------------------------------------------------
echo -e "\e[035m${PROGNAME}: ${1:-"Unknown Error"}\e[0m" 1>&2
exit 1
}
#
function log
{
echo -e "\e[035m${PROGNAME}: $1\e[0m"
}
#
##### Parameter processing
# These are the host and hostgroup that all web checks will be added to
HOSTID=
GROUPID=
GROUPNAME=
APPLNAME=
USERAGENT='Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)'
HOSTFILE=
DELETE=false
# timeout in seconds before a http request times out
TIMEOUT=60
# number of seconds between two checks
DELAY=60
# number of days to keep all values
HISTORY=90
# number of days to keep trends
TRENDS=360
MYSQLUSER=zabbix
MYSQLPASS=
MYSQLHOST=
MYSQLDB=zabbix
while [ "$1" != "" ]; do
case $1 in
-u | --dbuser ) shift
MYSQLUSER=$1
;;
-p | --dbpass ) shift
MYSQLPASS=$1
;;
-H | --dbhost ) shift
MYSQLHOST=$1
;;
-D | --dbname ) shift
MYSQLDB=$1
;;
-t | --timeout ) shift
TIMEOUT=$1
;;
-d | --delay ) shift
DELAY=$1
;;
-i | --history ) shift
HISTORY=$1
;;
-t | --trends ) shift
TRENDS=$1
;;
-o | --hostid ) shift
HOSTID=$1
;;
-g | --groupid ) shift
GROUPID=$1
;;
-G | --groupname ) shift
GROUPNAME=$1
;;
-f | --urlfile ) shift
HOSTFILE=$1
;;
--delete ) shift
DELETE=$1
;;
-a | --agent ) shift
USERAGENT=$1
;;
-A | --appname ) shift
APPLNAME=$1
;;
-h | --help ) usage
exit
;;
* ) usage
exit 1
esac
shift
done
if [ -z "$MYSQLUSER" ]
then
error_exit "$LINENO: Required parameter not found: -u | --dbuser. See -h | --help for more information"
fi
if [ -z "$MYSQLPASS" ]
then
error_exit "$LINENO: Required parameter not found: -p | --dbpass. See -h | --help for more information"
fi
if [ -z "$HOSTFILE" ]
then
error_exit "$LINENO: Required parameter not found: -f | --urlfile. See -h | --help for more information"
fi
if [ -z "$APPLNAME" ]
then
error_exit "$LINENO: Required parameter not found: -A | --appname. See -h | --help for more information"
fi
# Executes an sql command, call like so:
# mysql_cmd "select * from zabbix.items where hostid = ${HOSTID}"
mysql_cmd () {
if [ -z "$1" ] # Is parameter #1 zero length?
then
error_exit "-Parameter #1 is zero length in mysql_cmd.-\n" # Or no parameter passed.
fi
# --skip-columna-names for parseable output assumes we don't use select *
# so you already know which column names are in which order
# --batch output results in record-per-line tab-delimited way
#log "mysql -u${MYSQLUSER} -p${MYSQLPASS} -D${MYSQLDB} -h${MYSQLHOST} --batch --skip-column-name -e\"$1\""
result=`mysql -u"${MYSQLUSER}" -p"${MYSQLPASS}" -D${MYSQLDB} -h${MYSQLHOST} --batch --skip-column-name -e"$1"`
if [ "$?" = "0" ]; then
# in bash you cannot return a value so we echo it, the calling function can
# then just capture that using `` or $()
echo $result
else
error_exit "MySQL command failed: $1 (-u${MYSQLUSER} -p${MYSQLPASS} -D${MYSQLDB} -h${MYSQLHOST})"
fi
}
zabbix_get_next_id () {
table="$1"
field="$2"
# Get the current id and the next id and compare them. If the difference is 1, use it.
curr_id=`mysql_cmd "SELECT nextid FROM ids WHERE table_name='${table}' AND field_name='${field}';"`
next_id=`mysql_cmd "UPDATE ids SET nextid=nextid+1 WHERE table_name='${table}' AND field_name='${field}';SELECT nextid FROM ids WHERE table_name='${table}' AND field_name='${field}';"`
if [ `expr $curr_id - $next_id` -eq -1 ] ;
then
echo $next_id
return 1
else
return 0
fi
}
########### add triggers
### only add trigger for response code
function add_web_trigger() {
itemid="$1"
key_="$2"
description="$3"
url="$4"
##### check trigger
functionid=$(mysql_cmd "select functionid from functions where itemid=${itemid} and function='last' and parameter='#3' ")
if [ -z "$functionid" ];then
echo "functionid not exists!"
echo "creating trigger '$description' on $HOSTID"
functionid=$(zabbix_get_next_id 'functions' 'functionid')
triggerid=$(zabbix_get_next_id 'triggers' 'triggerid')
expression="{$functionid}<>200"
#echo $expression
mysql_cmd "insert into triggers(triggerid,expression,description,url,priority,comments) values (${triggerid},'${expression}','${description}','${url}',4,'')"
mysql_cmd "insert into functions(functionid,itemid,triggerid,function,parameter) values (${functionid},${itemid},${triggerid},'last','#3')"
else
expression="{$functionid}<>200"
triggerid=$(mysql_cmd "select triggerid from functions where functionid=${functionid}")
trigger_expression=$(mysql_cmd "select expression from triggers where triggerid=${triggerid}")
trigger_description=$(mysql_cmd "select description from triggers where triggerid=${triggerid}")
#echo $trigger_expression
#echo $expression
if [ "$trigger_expression" != "$expression" -o "$trigger_description" != "$description" ];then
echo "trigger expression is not right,delete it "
mysql_cmd "delete from triggers where triggerid=${triggerid}"
echo "creating trigger '$description' on $HOSTID"
mysql_cmd "insert into triggers(triggerid,expression,description,url,priority,comments) values (${triggerid},'${expression}','${description}','${url}',4,'')"
else
echo "trigger for ${key_} already exists!"
fi
fi
}
##################### ADD WEB CHECK FOR HOST ########################
function add_web_check_for_host() {
# Add an application if it doesn't exist and get it's appid returned
app_id=`mysql_cmd "select applicationid from applications where hostid = '${HOSTID}' and name = '${APPLNAME}'"`
if [ -z "$app_id" ]
then
app_id=`zabbix_get_next_id 'applications' 'applicationid'`
mysql_cmd "insert into applications(applicationid, hostid, name) values (${app_id}, ${HOSTID}, '${APPLNAME}')"
fi
# Loop through all sites to be added
while read line; do
######
### baidu 1 index.php https://www.baidu.com/index.php
####
log "== Starting ${line}"
site_name=$(echo $line|awk '{print $1}')
step_no=$(echo $line|awk '{print $2}')
url_name=$(echo $line|awk '{print $3}')
step_url=$(echo $line|awk '{print $4}')
test_name="check website ${site_name} on {HOST.NAME}"
step_name="check url ${url_name}"
####### for adding trigger
key_="web.test.rspcode[${test_name},${step_name}]"
description="${site_name} url ${url_name} on {HOST.NAME} is down"
# Add an httptest if it doesn't exist and get it's httptestid returned
httptest_id=`mysql_cmd "select httptestid from httptest where hostid='${HOSTID}' and name='${test_name}'"`
if [ -z "$httptest_id" ];then
httptest_id=`zabbix_get_next_id 'httptest' 'httptestid'`
mysql_cmd "INSERT INTO httptest(httptestid,name,applicationid,delay,status,variables,agent,authentication,hostid,headers) VALUES(${httptest_id},'${test_name}',${app_id},60,0,'','${USERAGENT}',0,${HOSTID},'')"
fi
# Add an httpstep if it doesn't exist and get it's httpstepid returned
httpstep_id=`mysql_cmd "select httpstepid from httpstep where httptestid='${httptest_id}' and no='${step_no}'"`
if [ -z "$httpstep_id" ];then
httpstep_id=`zabbix_get_next_id 'httpstep' 'httpstepid'`
mysql_cmd "INSERT INTO httpstep(httpstepid,httptestid,name,no,url,timeout,posts,required,variables,headers,status_codes) VALUES(${httpstep_id},${httptest_id},'${step_name}',${step_no},'${step_url}',${TIMEOUT},'','','','',200)"
else
stepname=`mysql_cmd "select name from httpstep where httpstepid=${httpstep_id}"`
stepurl=`mysql_cmd "select url from httpstep where httpstepid=${httpstep_id}"`
itemids=$(mysql_cmd "SELECT hsi.itemid FROM httpstep hs,httpstepitem hsi,items i WHERE hsi.itemid=i.itemid and hs.httpstepid=hsi.httpstepid and hs.httpstepid=${httpstep_id} ")
if [ -z "$(echo $itemids|grep ' ')" ];then
##### only one item
itemids1=$itemids
else
itemids1=$(echo $itemids|sed -n 's/ \|\t/,/gp')
fi
triggerids=$(mysql_cmd "select triggerid from functions where itemid in ($(echo $itemids1))")
if [ -z "$(echo $triggerids|grep ' ')" ];then
##### only one item
triggerids1=$triggerids
else
triggerids1=$(echo $triggerids|sed -n 's/ \|\t/,/gp')
fi
eventids=$(mysql_cmd "select eventid from escalations where itemid in ($(echo $itemids1))")
if [ -z "$(echo $eventids|grep ' ')" ];then
##### only one item
eventids1=$eventids
else
eventids1=$(echo $eventids|sed -n 's/ \|\t/,/gp')
fi
if [ "$stepname" != "${step_name}" -o "$stepurl" != "${step_url}" ];then
echo "${site_name} step ${step_no} has been changed,updating it"
mysql_cmd "delete from httpstep where httpstepid=${httpstep_id} and no=${step_no}"
mysql_cmd "delete from httpstepitem where itemid in ($(echo $itemids1))"
mysql_cmd "delete from functions where itemid in ($(echo $itemids1))"
mysql_cmd "delete from triggers where triggerid in ($(echo $triggerids1))"
if [ ! -z "$triggerids1" ];then
mysql_cmd "delete from escalations where triggerid in ($(echo $triggerids1))"
fi
if [ ! -z "$eventids1" ];then
mysql_cmd "delete from events where eventid in ($(echo $eventids1))"
fi
mysql_cmd "delete from items where itemid in ($(echo $itemids1))"
mysql_cmd "delete from items_applications where itemid in ($(echo $itemids1))"
httpstep_id=`zabbix_get_next_id 'httpstep' 'httpstepid'`
mysql_cmd "INSERT INTO httpstep(httpstepid,httptestid,name,no,url,timeout,posts,required,variables,headers,status_codes) VALUES(${httpstep_id},${httptest_id},'${step_name}',${step_no},'${step_url}',${TIMEOUT},'','','','',200)"
fi
fi
log "Create 3 items for response time, response code and download speed"
# download speed
item_id=$(mysql_cmd "select itemid from items where hostid=${HOSTID} and key_='web.test.in[${test_name},${step_name},bps]'")
httpstepitem_id=$(mysql_cmd "select httpstepitemid from httpstepitem where httpstepid=${httpstep_id} and type=2 ")
if [ -z "$item_id" -o -z "$httpstepitem_id" ];then
mysql_cmd "delete from items where hostid=${HOSTID} and key_='web.test.in[${test_name},${step_name},bps]'"
mysql_cmd "delete from httpstepitem where httpstepid=${httpstep_id} and type=2 "
item_id=`zabbix_get_next_id 'items' 'itemid'`
mysql_cmd "delete from items where hostid=${HOSTID} and key_='web.test.in[${test_name},${step_name},bps]'"
mysql_cmd "INSERT INTO items(itemid, type, hostid, name,description , key_, delay, history, trends, status, value_type, trapper_hosts, units, multiplier, delta, formula, error,logtimefmt, valuemapid, delay_flex, params, data_type, mtime) VALUES (${item_id}, 9, ${HOSTID}, 'Download speed for step \"\$2\" of scenario \"\$1\".','', 'web.test.in[${test_name},${step_name},bps]', ${DELAY}, ${HISTORY}, ${TRENDS}, 0, 0, '', 'Bps', 0, 0, '', '', '',NULL,'', '',0, 0)"
httpstepitem_id=`zabbix_get_next_id 'httpstepitem' 'httpstepitemid'`
mysql_cmd "insert into httpstepitem(httpstepitemid, httpstepid, itemid, type) values (${httpstepitem_id}, ${httpstep_id}, ${item_id}, 2)"
itemapp_id=`zabbix_get_next_id 'items_applications' 'itemappid'`
mysql_cmd "insert into items_applications(itemappid,applicationid,itemid) values (${itemapp_id},${app_id},${item_id})"
fi
# response time
item_id=$(mysql_cmd "select itemid from items where hostid=${HOSTID} and key_='web.test.time[${test_name},${step_name},resp]'")
httpstepitem_id=$(mysql_cmd "select httpstepitemid from httpstepitem where httpstepid=${httpstep_id} and type=1 ")
if [ -z "$item_id" -o -z "$httpstepitem_id" ];then
item_id=`zabbix_get_next_id 'items' 'itemid'`
mysql_cmd "delete from items where hostid=${HOSTID} and key_='web.test.time[${test_name},${step_name},resp]'"
mysql_cmd "delete from httpstepitem where httpstepid=${httpstep_id} and type=1 "
mysql_cmd "INSERT INTO items(itemid, type, hostid, name,description , key_, delay, history, trends, status, value_type, trapper_hosts, units, multiplier, delta, formula, error,logtimefmt, valuemapid, delay_flex, params, data_type, mtime) VALUES (${item_id}, 9, ${HOSTID}, 'Response time for step \"\$2\" of scenario \"\$1\".','', 'web.test.time[${test_name},${step_name},resp]', ${DELAY}, ${HISTORY}, ${TRENDS}, 0, 0, '', 's', 0, 0, '', '', '',NULL,'', '',0, 0)"
httpstepitem_id=`zabbix_get_next_id 'httpstepitem' 'httpstepitemid'`
mysql_cmd "insert into httpstepitem(httpstepitemid, httpstepid, itemid, type) values (${httpstepitem_id}, ${httpstep_id}, ${item_id}, 1)"
itemapp_id=`zabbix_get_next_id 'items_applications' 'itemappid'`
mysql_cmd "insert into items_applications(itemappid,applicationid,itemid) values (${itemapp_id},${app_id},${item_id})"
fi
# response code
item_id=$(mysql_cmd "select itemid from items where hostid=${HOSTID} and key_='web.test.rspcode[${test_name},${step_name}]'")
httpstepitem_id=$(mysql_cmd "select httpstepitemid from httpstepitem where httpstepid=${httpstep_id} and type=0 ")
if [ -z "$item_id" -o -z "$httpstepitem_id" ];then
mysql_cmd "delete from items where hostid=${HOSTID} and key_='web.test.rspcode[${test_name},${step_name}]'"
mysql_cmd "delete from httpstepitem where httpstepid=${httpstep_id} and type=0 "
item_id=`zabbix_get_next_id 'items' 'itemid'`
mysql_cmd "INSERT INTO items(itemid, type, hostid, name,description , key_, delay, history, trends, status, value_type, trapper_hosts, units, multiplier, delta, formula, error,logtimefmt, valuemapid, delay_flex, params, data_type, mtime) VALUES (${item_id}, 9, ${HOSTID}, 'Response code for step \"\$2\" of scenario \"\$1\".','', 'web.test.rspcode[${test_name},${step_name}]', ${DELAY}, ${HISTORY}, ${TRENDS}, 0, 3, '', '', 0, 0, '', '', '',NULL,'', '',0, 0)"
httpstepitem_id=`zabbix_get_next_id 'httpstepitem' 'httpstepitemid'`
mysql_cmd "insert into httpstepitem(httpstepitemid, httpstepid, itemid, type) values (${httpstepitem_id}, ${httpstep_id}, ${item_id}, 0)"
itemapp_id=`zabbix_get_next_id 'items_applications' 'itemappid'`
mysql_cmd "insert into items_applications(itemappid,applicationid,itemid) values (${itemapp_id},${app_id},${item_id})"
##### add web trigger
#### here must use "" instead of ''
add_web_trigger ${item_id} "${key_}" "${description}" "${step_url}"
fi
##### add web trigger
add_web_trigger ${item_id} "${key_}" "${description}" "${step_url}"
######### only need to add one for each website
if [ $step_no -eq 1 ];then
log "Create 3 items for download speed ,fail test of the whole scenario and last error message of scenario"
# download speed
item_id=$(mysql_cmd "select itemid from items where hostid=${HOSTID} and key_='web.test.in[${test_name},,bps]'")
httptestitem_id=$(mysql_cmd "select httptestitemid from httptestitem where httptestid=${httptest_id} ")
if [ -z "$item_id" -o -z "$httpstepitem_id" ];then
item_id=`zabbix_get_next_id 'items' 'itemid'`
mysql_cmd "INSERT INTO items(itemid, type, hostid, name,description , key_, delay, history, trends, status, value_type, trapper_hosts, units, multiplier, delta, formula, error,logtimefmt, valuemapid, delay_flex, params, data_type, mtime) VALUES (${item_id}, 9, ${HOSTID}, 'Download speed for scenario \"\$1\".','', 'web.test.in[${test_name},,bps]', ${DELAY}, ${HISTORY}, ${TRENDS}, 0, 0, '', 'Bps', 0, 0, '', '', '',NULL,'', '',0, 0)"
httptestitem_id=`zabbix_get_next_id 'httptestitem' 'httptestitemid'`
mysql_cmd "insert into httptestitem(httptestitemid, httptestid, itemid, type) values (${httptestitem_id}, ${httptest_id}, ${item_id}, 2)"
itemapp_id=`zabbix_get_next_id 'items_applications' 'itemappid'`
mysql_cmd "insert into items_applications(itemappid,applicationid,itemid) values (${itemapp_id},${app_id},${item_id})"
fi
# fail test
item_id=$(mysql_cmd "select itemid from items where hostid=${HOSTID} and key_='web.test.fail[${test_name}]'")
httptestitem_id=$(mysql_cmd "select httptestitemid from httptestitem where httptestid=${httptest_id} ")
if [ -z "$item_id" -o -z "$httpstepitem_id" ];then
item_id=`zabbix_get_next_id 'items' 'itemid'`
mysql_cmd "INSERT INTO items(itemid, type, hostid, name,description , key_, delay, history, trends, status, value_type, trapper_hosts, units, multiplier, delta, formula, error,logtimefmt, valuemapid, delay_flex, params, data_type, mtime) VALUES (${item_id}, 9, ${HOSTID}, 'Failed step of scenario \"\$1\".','', 'web.test.fail[${test_name}]',${DELAY}, ${HISTORY}, ${TRENDS}, 0, 3, '', '', 0, 0, '', '', '',NULL,'', '',0, 0)"
httptestitem_id=`zabbix_get_next_id 'httptestitem' 'httptestitemid'`
mysql_cmd "insert into httptestitem(httptestitemid, httptestid, itemid, type) values (${httptestitem_id}, ${httptest_id}, ${item_id}, 3)"
itemapp_id=`zabbix_get_next_id 'items_applications' 'itemappid'`
mysql_cmd "insert into items_applications(itemappid,applicationid,itemid) values (${itemapp_id},${app_id},${item_id})"
fi
# last error message
item_id=$(mysql_cmd "select itemid from items where hostid=${HOSTID} and key_='web.test.error[${test_name}]'")
httptestitem_id=$(mysql_cmd "select httptestitemid from httptestitem where httptestid=${httptest_id}")
if [ -z "$item_id" -o -z "$httpstepitem_id" ];then
item_id=`zabbix_get_next_id 'items' 'itemid'`
mysql_cmd "INSERT INTO items(itemid, type, hostid, name,description , key_, delay, history, trends, status, value_type, trapper_hosts, units, multiplier, delta, formula, error,logtimefmt, valuemapid, delay_flex, params, data_type, mtime) VALUES (${item_id}, 9, ${HOSTID}, 'Last error message of scenario \"\$1\".','', 'web.test.error[${test_name}]', ${DELAY}, ${HISTORY}, ${TRENDS}, 0, 4, '', '', 0, 0, '', '', '',NULL,'', '',0, 0)"
httptestitem_id=`zabbix_get_next_id 'httptestitem' 'httptestitemid'`
mysql_cmd "insert into httptestitem(httptestitemid, httptestid, itemid, type) values (${httptestitem_id}, ${httptest_id}, ${item_id}, 4)"
itemapp_id=`zabbix_get_next_id 'items_applications' 'itemappid'`
mysql_cmd "insert into items_applications(itemappid,applicationid,itemid) values (${itemapp_id},${app_id},${item_id})"
fi
fi
done < ${HOSTFILE}
}
################################ DELETE WEB CHECK FOR HOST ############################
function delete_web_check_for_host() {
# Loop through all sites to be added
while read line; do
######
### baidu 1 index.php https://www.baidu.com/index.php
####
site_name=$(echo $line|awk '{print $1}')
step_no=$(echo $line|awk '{print $2}')
url_name=$(echo $line|awk '{print $3}')
step_url=$(echo $line|awk '{print $4}')
test_name="check website ${site_name} on {HOST.NAME}"
step_name="check url ${url_name}"
#delete httptest if it exists!
httptest_id=`mysql_cmd "select httptestid from httptest where hostid='${HOSTID}' and name='${test_name}'"`
if [ -z "$httptest_id" ];then
echo "$test_name on $HOSTID not exists!"
else
echo -e "\e[035m DELETING ${test_name} RELATED ITEMS ON ${HOSTID}\e[0m"
itemids=$(mysql_cmd "SELECT hti.itemid FROM httptestitem hti,items i WHERE hti.itemid=i.itemid and httptestid=${httptest_id} UNION ALL SELECT hsi.itemid FROM httpstep hs,httpstepitem hsi,items i WHERE hsi.itemid=i.itemid and hs.httpstepid=hsi.httpstepid AND httptestid=${httptest_id}")
if [ -z "$itemids" ];then
echo "no need to delete items data"
else
####### delete items , history data
if [ -z "$(echo $itemids|grep ' ')" ];then
##### only one item
itemids1=$itemids
else
itemids1=$(echo $itemids|sed -n 's/ \|\t/,/gp')
fi
triggerids=$(mysql_cmd "select triggerid from functions where itemid in ($(echo $itemids1))")
if [ -z "$(echo $triggerids|grep ' ')" ];then
##### only one item
triggerids1=$triggerids
else
triggerids1=$(echo $triggerids|sed -n 's/ \|\t/,/gp')
fi
eventids=$(mysql_cmd "select eventid from escalations where itemid in ($(echo $itemids1))")
if [ -z "$(echo $eventids|grep ' ')" ];then
##### only one item
eventids1=$eventids
else
eventids1=$(echo $eventids|sed -n 's/ \|\t/,/gp')
fi
mysql_cmd "delete from items_applications where itemid in ($(echo $itemids1))"
mysql_cmd "delete from items where itemid in ($(echo $itemids1))"
mysql_cmd "delete from trends_uint where itemid in ($(echo $itemids1))"
mysql_cmd "delete from history_text where itemid in ($(echo $itemids1))"
mysql_cmd "delete from history_log where itemid in ($(echo $itemids1))"
mysql_cmd "delete from history_uint where itemid in ($(echo $itemids1))"
mysql_cmd "delete from history_str where itemid in ($(echo $itemids1))"
mysql_cmd "delete from history where itemid in ($(echo $itemids1))"
mysql_cmd "delete from functions where itemid in ($(echo $itemids1))"
if [ ! -z "$triggerids1" ];then
mysql_cmd "delete from escalations where triggerid in ($(echo $triggerids1))"
mysql_cmd "delete from triggers where triggerid in ($(echo $triggerids1))"
fi
if [ ! -z "$eventids1" ];then
mysql_cmd "delete from events where eventid in ($(echo $eventids1))"
fi
fi
fi
#######
#### after all items deleted,delete the httptest
echo -e "\e[035m DELETING ${test_name} ON ${HOSTID}\e[0m"
mysql_cmd "delete from httptest where hostid=${HOSTID} and name='${test_name}'"
done < ${HOSTFILE}
}
#######################################################################
######################################################################
function main_add () {
if [ "$HOSTID" != "" ];then
HOSTID=$HOSTID
echo -e "\e[036m################### ADDING WEB CHECK FOR $HOSTID ###################\e[0m"
add_web_check_for_host
fi
if [ "$GROUPID" != "" ];then
hostids=$(mysql_cmd "select hg.hostid from hosts_groups hg,hosts h where hg.hostid=h.hostid and groupid=$GROUPID")
for i in $(echo $hostids)
do
HOSTID=$i
echo -e "\e[036m################### ADDING WEB CHECK FOR $HOSTID ###################\e[0m"
add_web_check_for_host
done
fi
if [ "$GROUPNAME" != "" ];then
GROUPID=$(mysql_cmd "select groupid from groups where name='${GROUPNAME}'")
hostids=$(mysql_cmd "select hg.hostid from hosts_groups hg,hosts h where hg.hostid=h.hostid and groupid=$GROUPID")
for i in $(echo $hostids)
do
HOSTID=$i
echo -e "\e[036m################### ADDING WEB CHECK FOR $HOSTID ###################\e[0m"
add_web_check_for_host
done
fi
}
################################
function main_delete () {
if [ "$HOSTID" != "" ];then
HOSTID=$HOSTID
delete_web_check_for_host
fi
if [ "$GROUPID" != "" ];then
hostids=$(mysql_cmd "select hg.hostid from hosts_groups hg,hosts h where hg.hostid=h.hostid and groupid=$GROUPID")
for i in $(echo $hostids)
do
HOSTID=$i
delete_web_check_for_host
done
fi
if [ "$GROUPNAME" != "" ];then
GROUPID=$(mysql_cmd "select groupid from groups where name='${GROUPNAME}'")
hostids=$(mysql_cmd "select hg.hostid from hosts_groups hg,hosts h where hg.hostid=h.hostid and groupid=$GROUPID")
for i in $(echo $hostids)
do
HOSTID=$i
delete_web_check_for_host
done
fi
}
#######################################################
####### add or delete
if [ "$DELETE" == "true" -o "$DELETE" == "True" -o "$DELETE" == "TRUE" ];then
main_delete
else
main_add
fi
使用方法：
添加Web监控，可以是 -o 10328为单个主机添加Web监控，也可以是-g 23 或者 -g 'EC2 Servers' 为一个主机组添加Web监控，文件格式如上
sh add_web_checks.sh -u 'zabbix' -H localhost -p abc123 -A "Check Websites" -G 'EC2 Servers' -f zabbix_web_check_files.txt
删除Web监控
sh add_web_checks.sh -u 'zabbix' -H localhost -p abc123 -A "Check Websites" -G 'EC2 Servers' -f zabbix_web_check_files.txt --delete true
如果想要删除相关的Web监控可以添加一个 --delete true 参数，就可以直接删除相关的数据了
在脚本为每个url的返回码设置了一个trigger，不为200就报警
这种直接更改数据库的方法看着有点麻烦，因为必须要了解Zabbix内部是如何存储数据的才能去更改相应的表。也比较容易出错，在以上的脚本中我们没有使用模板的方式来添加Web监控，操作会比较麻烦些。
参考文档：
http://sfzhang88.blog.51cto.com/4995876/1826763
http://john88wang.blog.51cto.com/2165294/1403754
https://www.zabbix.org/wiki/Docs/howto/script_web_checks
https://www.zabbix.com/documentation/3.0/manual/api/reference/item/object