公司IDC出口流量是按照1分钟的时候间隔取值的。第三方运营商(CDN、IDC)一般都是按照5分钟的时间间隔统计流量的,计费方式是每月按照95计费或者峰值计费,不同的运营商计费方式不一样。
Zabbix监控流量定期通过API获取数据存入本地MySQL数据库当中,方便后期数据统计和分析。下面通过SQL语句根据第三方标准(5min取值)统计出口流量的95计费或者峰值计费。
1)由于公司每条链路都是两根线接入,每根线都有流量,如果一根线路出现问题,流量会全部切到另外一条线路上面。先用SQL语句统计两根线路流量的总和,按照每分钟统计。
mysql> select RunDate,RunTime,round((bjtel_line1_out + bjtel_line2_out)/1000/1000,2) as Totalsum from t_netmx where RunDate>='2018-12-01' and RunDate<'2019-01-01' group by 1,2 limit 30;
+------------+---------+--------+
| RunDate | RunTime | Totalsum |
+------------+---------+--------+
| 2018-12-01 | 00:00 | 133.14 |
| 2018-12-01 | 00:01 | 190.65 |
| 2018-12-01 | 00:02 | 152.80 |
| 2018-12-01 | 00:03 | 137.99 |
| 2018-12-01 | 00:04 | 162.81 |
| 2018-12-01 | 00:05 | 129.08 |
| 2018-12-01 | 00:06 | 152.86 |
| 2018-12-01 | 00:07 | 143.34 |
| 2018-12-01 | 00:08 | 139.36 |
| 2018-12-01 | 00:09 | 133.31 |
| 2018-12-01 | 00:10 | 121.75 |
| 2018-12-01 | 00:11 | 147.79 |
| 2018-12-01 | 00:12 | 129.35 |
| 2018-12-01 | 00:13 | 137.92 |
| 2018-12-01 | 00:14 | 117.28 |
| 2018-12-01 | 00:15 | 127.25 |
| 2018-12-01 | 00:16 | 266.33 |
| 2018-12-01 | 00:17 | 214.06 |
| 2018-12-01 | 00:18 | 241.38 |
| 2018-12-01 | 00:19 | 220.78 |
| 2018-12-01 | 00:20 | 168.99 |
| 2018-12-01 | 00:21 | 146.52 |
| 2018-12-01 | 00:22 | 108.88 |
| 2018-12-01 | 00:23 | 100.35 |
| 2018-12-01 | 00:24 | 112.49 |
| 2018-12-01 | 00:25 | 121.06 |
| 2018-12-01 | 00:26 | 111.48 |
| 2018-12-01 | 00:27 | 111.46 |
| 2018-12-01 | 00:28 | 111.30 |
| 2018-12-01 | 00:29 | 115.38 |
+------------+---------+--------+
30 rows in set (0.00 sec)
2)根据第三方的标准取值,每5min取一个平均值,然后打印输出,这里非常感谢我们公司两位非常帅气的小哥哥帮我写这条SQL语句。
mysql> select FROM_UNIXTIME(unix_timestamp(str_to_date(concat(rundate,' ',runtime),'%Y-%m-%d %H:%i')) - (unix_timestamp(str_to_date(concat(rundate,' ',runtime),'%Y-%m-%d %H:%i')) % (5 * 60))) as DateTime,round(avg(bjtel_line1_out + bjtel_line2_out)/1000/1000,2) as avg5min from t_netmx where RunDate>='2018-12-01' and RunDate<'2019-01-01' group by DateTime limit 6;
+---------------------+---------+
| DateTime | avg5min |
+---------------------+---------+
| 2018-12-01 00:00:00 | 155.48 |
| 2018-12-01 00:05:00 | 139.59 |
| 2018-12-01 00:10:00 | 130.82 |
| 2018-12-01 00:15:00 | 213.96 |
| 2018-12-01 00:20:00 | 127.45 |
| 2018-12-01 00:25:00 | 114.14 |
+---------------------+---------+
6 rows in set (0.32 sec)
3)把avg5min的值按照从大到小排序,通过limit取出流量的峰值或者9/5值,2018年12月份流量的峰值为1215.04M,9/5值为646.85M。
mysql> select FROM_UNIXTIME(unix_timestamp(str_to_date(concat(rundate,' ',runtime),'%Y-%m-%d %H:%i')) - (unix_timestamp(str_to_date(concat(rundate,' ',runtime),'%Y-%m-%d %H:%i')) % (5 * 60))) as DateTime,round(avg(bjtel_line1_out + bjtel_line2_out)/1000/1000,2) as avg5min from t_netmx where RunDate>='2018-12-01' and RunDate<'2019-01-01' group by DateTime order by avg5min desc limit 1,1;
+---------------------+---------+
| DateTime | avg5min |
+---------------------+---------+
| 2018-12-25 11:00:00 | 1215.04 |
+---------------------+---------+
1 row in set (0.37 sec)
mysql> select FROM_UNIXTIME(unix_timestamp(str_to_date(concat(rundate,' ',runtime),'%Y-%m-%d %H:%i')) - (unix_timestamp(str_to_date(concat(rundate,' ',runtime),'%Y-%m-%d %H:%i')) % (5 * 60))) as DateTime,round(avg(bjtel_line1_out + bjtel_line2_out)/1000/1000,2) as avg5min from t_netmx where RunDate>='2018-12-01' and RunDate<'2019-01-01' group by DateTime order by avg5min desc limit 446,1;
+---------------------+---------+
| DateTime | avg5min |
+---------------------+---------+
| 2018-12-17 16:20:00 | 646.85 |
+---------------------+---------+
1 row in set (0.36 sec)
4)通过SHELL脚本自动获取上个月IDC出口流量的峰值和9/5值。
[root@monitor idc]# cat idc_pk_95_values.sh
#!/bin/sh
#create by sfzhang 20180131
PATH=$PATH:/usr/local/mysql/bin
CurrentMouthDay=`date +%Y-%m-01`
LastMouthStartDay=`date -d "$CurrentMouthDay last month" +%Y-%m-%d`
LastMouthEndDay=`date -d "$CurrentMouthDay last day" +%Y-%m-%d`
LastMouthStartDayToNum=`echo $LastMouthStartDay|sed 's/-//g'`
LastMouthEndDayToNum=`echo $LastMouthEndDay|sed 's/-//g'`
DiffDays=`expr $LastMouthEndDayToNum - $LastMouthStartDayToNum + 1`
P95=$(echo "scale=0;$[$DiffDays*24*12*5]/100" | bc)
echo $LastMouthStartDayToNum-$LastMouthEndDayToNum "Total":$DiffDays "days" "P95":$P95
function getIDCNetflowFromDB
{
BJTELPkMonth=$(mysql -hx.x.x.x -uxxxx -pxxxx -D xxxx -e "select FROM_UNIXTIME(unix_timestamp(str_to_date(concat(rundate,' ',runtime),'%Y-%m-%d %H:%i')) - (unix_timestamp(str_to_date(concat(rundate,' ',runtime),'%Y-%m-%d %H:%i')) % (5 * 60))) as DateTime,round(avg(bjtel_line1_out + bjtel_line2_out)/1000/1000,2) as avg5min from xxxxxx where RunDate>='$LastMouthStartDay' and RunDate<'$CurrentMouthDay' group by DateTime order by avg5min desc limit 1,1"|awk '{print $NF}' |egrep "^[0-9]")
BJTEL95Month=$(mysql -hx.x.x.x -uxxxx -pxxxx -D xxxx -e "select FROM_UNIXTIME(unix_timestamp(str_to_date(concat(rundate,' ',runtime),'%Y-%m-%d %H:%i')) - (unix_timestamp(str_to_date(concat(rundate,' ',runtime),'%Y-%m-%d %H:%i')) % (5 * 60))) as DateTime,round(avg(bjtel_line1_out + bjtel_line2_out)/1000/1000,2) as avg5min from xxxxxx where RunDate>='$LastMouthStartDay' and RunDate<'$CurrentMouthDay' group by DateTime order by avg5min desc limit $P95,1"|awk '{print $NF}' |egrep "^[0-9]")
printf "%s %.2f %s\n" BJTELPkMonth: $BJTELPkMonth Mbps
printf "%s %.2f %s\n" BJTEL95Month: $BJTEL95Month Mbps
}
getIDCNetflowFromDB
5)执行脚本获取上个月IDC出口流量的峰值和9/5值。
[root@monitor idc]# sh idc_pk_95_values.sh
20181201-20181231 Total:31 days P95:446
BJTELPkMonth: 1215.04 Mbps
BJTEL95Month: 646.85 Mbps
下面的SQL语句统计的是上个月IDC出口流量的均值流量。
select avg(avg5min) from (select FROM_UNIXTIME(unix_timestamp(str_to_date(concat(rundate,' ',runtime),'%Y-%m-%d %H:%i')) - (unix_timestamp(str_to_date(concat(rundate,' ',runtime),'%Y-%m-%d %H:%i')) % (5 * 60))) as DateTime,round(avg(wxl_3850_out + wxl_3850_2_out)/1000/1000,2) as avg5min from t_netmx where RunDate>='$LastMouthStartDay' and RunDate<'$CurrentMouthDay' group by DateTime order by avg5min) as temp_table;`