TCP的连接状态对于我们web服务器来说是至关重要的，尤其是并发量ESTAB；或者是syn_recv值，假如这个值比较大的话我们可以认为是不是受到了攻击，或是是time_wait值比较高的话，我们要考虑看我们内核是否需要调优，太高的time_wait值的话会占用太多端口，要是端口少的话后果不堪设想
一、TCP状态介绍
1、man netstat查看TCP的各种状态信息描述：
LISTEN -侦听来自远方TCP端口的连接请求；
SYN-SENT -在发送连接请求后等待匹配的连接请求；
SYN-RECEIVED -在收到和发送一个连接请求后等待对连接请求的确认；
ESTABLISHED -代表一个打开的连接，数据可以传送给用户；
FIN-WAIT-1 -等待远程TCP的连接中断请求，或先前的连接中断请求的确认；
FIN-WAIT-2 -从远程TCP等待连接中断请求；
CLOSE-WAIT -等待从本地用户发来的连接中断请求；
CLOSING -等待远程TCP对连接中断的确认；
LAST-ACK -等待原来发向远程TCP的连接中断请求的确认；
TIME-WAIT -等待足够的时间以确保远程TCP接收到连接中断请求的确认；
CLOSED -没有任何连接状态；
2、监控原理
[root@Node1 ~]# /bin/netstat -an|awk '/^tcp/{++S[$NF]}END{for(a in S) print a,S[a]}' //通过netstat获取相关值 LISTEN 10 ESTABLISHED 1 TIME_WAIT 178 [root@Node1 ~]#
二、监控脚本编写
1、编写脚本，放于 /etc/zabbix/zabbix_agentd.d/目录下
1 [root@Node1 zabbix_agentd.d]# cat tcp_status.sh 2 #!/bin/bash 3 #This script is used to get tcp and udp connetion status 4 #tcp status 5 metric=$1 6 tmp_file=/tmp/tcp_status.txt 7 /bin/netstat -an|awk '/^tcp/{++S[$NF]}END{for(a in S) print a,S[a]}' > $tmp_file 8 case $metric in 9 closed) 10 output=$(awk '/CLOSED/{print $2}' $tmp_file) 11 if [ "$output" == "" ];then 12 echo 0 13 else 14 echo $output 15 fi 16 ;; 17 listen) 18 output=$(awk '/LISTEN/{print $2}' $tmp_file) 19 if [ "$output" == "" ];then 20 echo 0 21 else 22 echo $output 23 fi 24 ;; 25 synrecv) 26 output=$(awk '/SYN_RECV/{print $2}' $tmp_file) 27 if [ "$output" == "" ];then 28 echo 0 29 else 30 echo $output 31 fi 32 ;; 33 synsent) 34 output=$(awk '/SYN_SENT/{print $2}' $tmp_file) 35 if [ "$output" == "" ];then 36 echo 0 37 else 38 echo $output 39 fi 40 ;; 41 established) 42 output=$(awk '/ESTABLISHED/{print $2}' $tmp_file) 43 if [ "$output" == "" ];then 44 echo 0 45 else 46 echo $output 47 fi 48 ;; 49 timewait) 50 output=$(awk '/TIME_WAIT/{print $2}' $tmp_file) 51 if [ "$output" == "" ];then 52 echo 0 53 else 54 echo $output 55 fi 56 ;; 57 closing) 58 output=$(awk '/CLOSING/{print $2}' $tmp_file) 59 if [ "$output" == "" ];then 60 echo 0 61 else 62 echo $output 63 fi 64 ;; 65 closewait) 66 output=$(awk '/CLOSE_WAIT/{print $2}' $tmp_file) 67 if [ "$output" == "" ];then 68 echo 0 69 else 70 echo $output 71 fi 72 ;; 73 lastack) 74 output=$(awk '/LAST_ACK/{print $2}' $tmp_file) 75 if [ "$output" == "" ];then 76 echo 0 77 else 78 echo $output 79 fi 80 ;; 81 finwait1) 82 output=$(awk '/FIN_WAIT1/{print $2}' $tmp_file) 83 if [ "$output" == "" ];then 84 echo 0 85 else 86 echo $output 87 fi 88 ;; 89 finwait2) 90 output=$(awk '/FIN_WAIT2/{print $2}' $tmp_file) 91 if [ "$output" == "" ];then 92 echo 0 93 else 94 echo $output 95 fi 96 ;; 97 *) 98 echo -e "\e[033mUsage: sh $0 [closed|closing|closewait|synrecv|synsent|finwait1|finwait2|listen|established|lastack|timewait]\e[0m" 99 esac
赋予脚本执行权限：
[root@Node1 ~]# chmod a+x /etc/zabbix/zabbix_agentd.d/tcp_status.sh //赋予执行权限 [root@Node1 ~]# ll /etc/zabbix/zabbix_agentd.d/tcp_status.sh -rwxr-xr-x 1 root root 2502 Jan 18 09:48 /etc/zabbix/zabbix_agentd.d/tcp_status.sh [root@Node1 ~]#
2、添加zabbix配置文件，放于 /etc/zabbix/zabbix_agentd.d/目录下（agent的配置文件 /etc/zabbix/zabbix_agentd.conf 中定义了其他key的包含目录）创建配置文件tcp_status.conf
[root@Node1 ~]# cat /etc/zabbix/zabbix_agentd.d/tcp_status.conf UserParameter=tcp.status[*],/etc/zabbix/zabbix_agentd.d/tcp_status.sh "$1" //脚本路径 [root@Node1 ~]#
3、确保配置Agent配置文件开启自定义参数UnsafeUserParameters=1
[root@Node1 ~]# grep -n "^[a-Z]" /etc/zabbix/zabbix_agentd.conf 13:PidFile=/var/run/zabbix/zabbix_agentd.pid 32:LogFile=/var/log/zabbix/zabbix_agentd.log 43:LogFileSize=0 57:DebugLevel=3 97:Server=172.17.21.208 138:ServerActive=172.17.21.208 149:Hostname=Node1.contoso.com 267:Include=/etc/zabbix/zabbix_agentd.d/*.conf 286:UnsafeUserParameters=1 //1代表允许，0代表关闭
4、重启zabbix-agent服务
[root@Node1 ~]# systemctl restart zabbix-agent.service
备注：因为脚本是把tcp的一些信息存放在/tmp/下，为了zabbix可以读取到我们设置zabbix可以读的权限，确保属主与属组都为zabbix即可
[root@Node1 ~]# chown zabbix.zabbix /tmp/tcp_status.txt //改变属主与属主 [root@Node1 ~]# ll /tmp/tcp_status.txt -rw-rw-r-- 1 zabbix zabbix 38 Jan 18 11:32 /tmp/tcp_status.txt
5、在zabbix servere服务器上测试,是否能正常获取数据
[root@Node3 ~]# zabbix_get -s 172.17.21.206 -p 10050 -k "tcp.status[listen]" 0 [root@Node3 ~]# zabbix_get -s 172.17.21.206 -p 10050 -k "tcp.status[timewait]" 183 [root@Node3 ~]# zabbix_get -s 172.17.21.206 -p 10050 -k "tcp.status[established]" 2 [root@Node3 ~]#
三、WEB界面配置
1、导入事先准备好的TCP Connection Status模板，Configuration-->Templates-->Import，选择相应的模板，点击Import即可
2、关联模板，Configuration-->Hosts-->Node1--Templates，选择模板,点击Updtae即可
3、查看TCP状态：
Monitoring-->Graphs-->Host(Node1)-->Grahp(TCP Status)
附模板：Template_TCP_Status.xml
xml version="1.0" encoding="UTF-8"?>
<zabbix_export>
<version>3.4version>
<date>2018-01-18T11:10:22Zdate>
<groups>
<group>
<name>Templatesname>
group>
groups>
<templates>
<template>
<template>Template TCP Connection Statustemplate>
<name>Template TCP Connection Statusname>
<description/>
<groups>
<group>
<name>Templatesname>
group>
groups>
<applications>
<application>
<name>TCP Statusname>
application>
applications>
<items>
<item>
<name>CLOSEDname>
<type>0type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[closed]key>
<delay>60sdelay>
<history>90dhistory>
<trends>365dtrends>
<status>0status>
<value_type>3value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0snmpv3_securitylevel>
<snmpv3_authprotocol>0snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0inventory_link>
<applications>
<application>
<name>TCP Statusname>
application>
applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
item>
<item>
<name>CLOSE_WAITname>
<type>0type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[closewait]key>
<delay>60sdelay>
<history>90dhistory>
<trends>365dtrends>
<status>0status>
<value_type>3value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0snmpv3_securitylevel>
<snmpv3_authprotocol>0snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0inventory_link>
<applications>
<application>
<name>TCP Statusname>
application>
applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
item>
<item>
<name>CLOSINGname>
<type>0type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[closing]key>
<delay>60sdelay>
<history>90dhistory>
<trends>365dtrends>
<status>0status>
<value_type>3value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0snmpv3_securitylevel>
<snmpv3_authprotocol>0snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0inventory_link>
<applications>
<application>
<name>TCP Statusname>
application>
applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
item>
<item>
<name>ESTABLISHEDname>
<type>0type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[established]key>
<delay>60sdelay>
<history>90dhistory>
<trends>365dtrends>
<status>0status>
<value_type>3value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0snmpv3_securitylevel>
<snmpv3_authprotocol>0snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0inventory_link>
<applications>
<application>
<name>TCP Statusname>
application>
applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
item>
<item>
<name>FIN_WAIT1name>
<type>0type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[finwait1]key>
<delay>60sdelay>
<history>90dhistory>
<trends>365dtrends>
<status>0status>
<value_type>3value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0snmpv3_securitylevel>
<snmpv3_authprotocol>0snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0inventory_link>
<applications>
<application>
<name>TCP Statusname>
application>
applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
item>
<item>
<name>FIN_WAIT2name>
<type>0type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[finwait2]key>
<delay>60sdelay>
<history>90dhistory>
<trends>365dtrends>
<status>0status>
<value_type>3value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0snmpv3_securitylevel>
<snmpv3_authprotocol>0snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0inventory_link>
<applications>
<application>
<name>TCP Statusname>
application>
applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
item>
<item>
<name>LAST_ACKname>
<type>0type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[lastack]key>
<delay>60sdelay>
<history>90dhistory>
<trends>365dtrends>
<status>0status>
<value_type>3value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0snmpv3_securitylevel>
<snmpv3_authprotocol>0snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0inventory_link>
<applications>
<application>
<name>TCP Statusname>
application>
applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
item>
<item>
<name>LISTENname>
<type>0type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[listen]key>
<delay>60sdelay>
<history>90dhistory>
<trends>365dtrends>
<status>0status>
<value_type>3value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0snmpv3_securitylevel>
<snmpv3_authprotocol>0snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0inventory_link>
<applications>
<application>
<name>TCP Statusname>
application>
applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
item>
<item>
<name>SYN_RECVname>
<type>0type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[synrecv]key>
<delay>60sdelay>
<history>90dhistory>
<trends>365dtrends>
<status>0status>
<value_type>3value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0snmpv3_securitylevel>
<snmpv3_authprotocol>0snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0inventory_link>
<applications>
<application>
<name>TCP Statusname>
application>
applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
item>
<item>
<name>SYN_SENTname>
<type>0type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[synsent]key>
<delay>60sdelay>
<history>90dhistory>
<trends>365dtrends>
<status>0status>
<value_type>3value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0snmpv3_securitylevel>
<snmpv3_authprotocol>0snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0inventory_link>
<applications>
<application>
<name>TCP Statusname>
application>
applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
item>
<item>
<name>TIME_WAITname>
<type>0type>
<snmp_community/>
<snmp_oid/>
<key>tcp.status[timewait]key>
<delay>60sdelay>
<history>90dhistory>
<trends>365dtrends>
<status>0status>
<value_type>3value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0snmpv3_securitylevel>
<snmpv3_authprotocol>0snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0inventory_link>
<applications>
<application>
<name>TCP Statusname>
application>
applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
item>
items>
<discovery_rules/>
<httptests/>
<macros/>
<templates/>
<screens/>
template>
templates>
<triggers>
<trigger>
<expression>{Template TCP Connection Status:tcp.status[listen].last()}>500expression>
<recovery_mode>0recovery_mode>
<recovery_expression/>
<name>There are too many TCP LISTEN statusname>
<correlation_mode>0correlation_mode>
<correlation_tag/>
<url/>
<status>0status>
<priority>4priority>
<description/>
<type>0type>
<manual_close>0manual_close>
<dependencies/>
<tags/>
trigger>
<trigger>
<expression>{Template TCP Connection Status:tcp.status[timewait].last()}>10000expression>
<recovery_mode>0recovery_mode>
<recovery_expression/>
<name>There are too many TCP TIME_WAIT statusname>
<correlation_mode>0correlation_mode>
<correlation_tag/>
<url/>
<status>0status>
<priority>4priority>
<description/>
<type>0type>
<manual_close>0manual_close>
<dependencies/>
<tags/>
trigger>
triggers>
<graphs>
<graph>
<name>TCP Statusname>
<width>900width>
<height>200height>
<yaxismin>0.0000yaxismin>
<yaxismax>100.0000yaxismax>
<show_work_period>1show_work_period>
<show_triggers>1show_triggers>
<type>0type>
<show_legend>1show_legend>
<show_3d>0show_3d>
<percent_left>0.0000percent_left>
<percent_right>0.0000percent_right>
<ymin_type_1>0ymin_type_1>
<ymax_type_1>0ymax_type_1>
<ymin_item_1>0ymin_item_1>
<ymax_item_1>0ymax_item_1>
<graph_items>
<graph_item>
<sortorder>0sortorder>
<drawtype>0drawtype>
<color>C80000color>
<yaxisside>0