监控io性能
监控系统状态
[root@localhost ~]# iostat 1
Linux 3.10.0-693.el7.x86_64 (localhost.localdomain) 2018年03月06日 _x86_64_ (2 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
0.06 0.00 0.09 0.00 0.00 99.84
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 0.23 1.49 4.48 332604 998066
sdb 0.00 0.02 0.00 4748 0
scd0 0.00 0.01 0.00 2056 0
dm-0 0.00 0.01 0.00 2072 0
avg-cpu: %user %nice %system %iowait %steal %idle
0.00 0.00 0.00 0.00 0.00 100.00
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 0.00 0.00 0.00 0 0
sdb 0.00 0.00 0.00 0 0
scd0 0.00 0.00 0.00 0 0
dm-0 0.00 0.00 0.00 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
0.00 0.00 0.00 0.00 0.00 100.00
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 0.00 0.00 0.00 0 0
sdb 0.00 0.00 0.00 0 0
scd0 0.00 0.00 0.00 0 0
dm-0 0.00 0.00 0.00 0 0
^C
[root@localhost ~]# sar -b
Linux 3.10.0-693.el7.x86_64 (localhost.localdomain) 2018年03月06日 _x86_64_ (2 CPU)
[root@localhost ~]# sar -b 1
Linux 3.10.0-693.el7.x86_64 (localhost.localdomain) 2018年03月06日 _x86_64_ (2 CPU)
16时25分00秒 tps rtps wtps bread/s bwrtn/s
16时25分01秒 0.00 0.00 0.00 0.00 0.00
16时25分02秒 0.00 0.00 0.00 0.00 0.00
16时25分03秒 0.00 0.00 0.00 0.00 0.00
16时25分04秒 0.00 0.00 0.00 0.00 0.00
^C
16时25分04秒 0.00 0.00 0.00 0.00 0.00
平均时间: 0.00 0.00 0.00 0.00 0.00
iostat -x
关注%util，太高就是磁盘传输速度跟不上
[root@localhost ~]# iostat -x
Linux 3.10.0-693.el7.x86_64 (localhost.localdomain) 2018年03月06日 _x86_64_ (2 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
0.06 0.00 0.09 0.00 0.00 99.84
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 0.04 0.05 0.18 1.49 4.48 52.83 0.00 0.94 0.47 1.07 0.32 0.01
sdb 0.00 0.00 0.00 0.00 0.02 0.00 30.44 0.00 0.16 0.16 0.00 0.12 0.00
scd0 0.00 0.00 0.00 0.00 0.01 0.00 114.22 0.00 0.94 0.94 0.00 0.69 0.00
dm-0 0.00 0.00 0.00 0.00 0.01 0.00 49.93 0.00 0.13 0.13 0.00 0.10 0.00
查看哪个进程IO读写繁忙，安装工具iotop
[root@localhost ~]# yum install iotop
Total DISK READ : 22.04 M/s | Total DISK WRITE : 227.93 K/s
Actual DISK READ: 22.37 M/s | Actual DISK WRITE: 243.95 K/s
TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
10824 be/4 www-data 2.45 M/s 0.00 B/s 0.00 % 99.99 % apache2 -k start
6608 be/4 www-data 2.23 M/s 0.00 B/s 0.00 % 99.99 % apache2 -k start
7829 be/4 www-data 0.00 B/s 0.00 B/s 0.00 % 99.99 % apache2 -k start
10825 be/4 www-data 2.34 M/s 0.00 B/s 0.00 % 99.65 % apache2 -k start
21111 be/4 www-data 2.23 M/s 0.00 B/s 0.00 % 99.10 % apache2 -k start
10811 be/4 www-data 2.34 M/s 0.00 B/s 0.00 % 98.73 % apache2 -k start
21112 be/4 www-data 2.23 M/s 0.00 B/s 0.00 % 98.72 % apache2 -k start
10691 be/4 www-data 2.23 M/s 0.00 B/s 0.00 % 98.11 % apache2 -k start
10968 be/4 www-data 2.23 M/s 0.00 B/s 0.00 % 97.94 % apache2 -k start
7874 be/4 www-data 2.23 M/s 0.00 B/s 0.00 % 97.77 % apache2 -k start
6609 be/4 www-data 1595.49 K/s 0.00 B/s 0.00 % 45.60 % apache2 -k start
1420 be/3 root 0.00 B/s 213.68 K/s 0.00 % 13.12 % [jbd2/sda2-8]
101 be/3 root 0.00 B/s 14.25 K/s 0.00 % 0.80 % [jbd2/mmcblk0p2-]
1 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % init splash
2 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kthreadd]
3 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/0]
5 be/0 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kworker/0:0H]
1030 be/4 www-data 0.00 B/s 0.00 B/s 0.00 % 0.00 % php-fpm: pool www
7 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [rcu_sched]
8 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [rcu_bh]
9 rt/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [migration
free命令
数据经过CPU计算，即将要写入磁盘，这时用的内存为buffer；CPU需要计算时，需要把数据从磁盘中读出来，临时先放到内存中，这部分内存就是cache。
[root@localhost ~]# free
total used free shared buff/cache available
Mem: 1867048 150404 779552 17576 937092 1476320
Swap: 4194300 0 4194300
[root@localhost ~]# free -m
total used free shared buff/cache available
Mem: 1823 146 761 17 915 1441
Swap: 4095 0 4095
[root@localhost ~]# free -h
total used free shared buff/cache available
Mem: 1.8G 146M 761M 17M 915M 1.4G
Swap: 4.0G 0B 4.0G
公式：total=used+free+buff/cache
avaliable包含free和buffer/cache剩余部分，buffer/cache只是被分配了，并不一定就被使用了
ps命令
image.png
ps aux静态的列出所有的进程
[root@localhost ~]# ps aux|head
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.3 46052 6384 ? Ss 3月04 0:09 /usr/lib/systemd/systemd --system --deserialize 17
root 2 0.0 0.0 0 0 ? S 3月04 0:00 [kthreadd]
root 3 0.0 0.0 0 0 ? S 3月04 0:00 [ksoftirqd/0]
root 5 0.0 0.0 0 0 ? S< 3月04 0:00 [kworker/0:0H]
root 7 0.0 0.0 0 0 ? S 3月04 0:00 [migration/0]
root 8 0.0 0.0 0 0 ? S 3月04 0:00 [rcu_bh]
root 9 0.0 0.0 0 0 ? S 3月04 0:06 [rcu_sched]
root 10 0.0 0.0 0 0 ? S 3月04 0:01 [watchdog/0]
root 11 0.0 0.0 0 0 ? S 3月04 0:01 [watchdog/1]
[root@localhost ~]# ps -elf|head
F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD
4 S root 1 0 0 80 0 - 11513 ep_pol 3月04 ? 00:00:09 /usr/lib/systemd/systemd --system --deserialize 17
1 S root 2 0 0 80 0 - 0 kthrea 3月04 ? 00:00:00 [kthreadd]
1 S root 3 2 0 80 0 - 0 smpboo 3月04 ? 00:00:00 [ksoftirqd/0]
1 S root 5 2 0 60 -20 - 0 worker 3月04 ? 00:00:00 [kworker/0:0H]
1 S root 7 2 0 -40 - - 0 smpboo 3月04 ? 00:00:00 [migration/0]
1 S root 8 2 0 80 0 - 0 rcu_gp 3月04 ? 00:00:00 [rcu_bh]
1 S root 9 2 0 80 0 - 0 rcu_gp 3月04 ? 00:00:06 [rcu_sched]
5 S root 10 2 0 -40 - - 0 smpboo 3月04 ? 00:00:01 [watchdog/0]
5 S root 11 2 0 -40 - - 0 smpboo 3月04 ? 00:00:01 [watchdog/1]
加上管道符 | grep 可以检查某项进程和服务
root@raspberrypi:/home/pi# ps aux|grep php
root 548 0.0 0.0 140140 720 ? Ss 3月05 0:06 php-fpm: master process (/etc/php/7.0/fpm/php-fpm.conf)
www-data 1029 0.0 0.0 140140 76 ? S 3月05 0:00 php-fpm: pool www
www-data 1030 0.0 0.0 140140 76 ? S 3月05 0:00 php-fpm: pool www
root 20421 0.0 0.0 6528 580 pts/0 S+ 22:31 0:00 grep php
PID进程ID可以在杀掉进程时使用
杀死进程可以用kill命令
[root@localhost ~]# ps aux|grep qmgr
postfix 1191 0.0 0.2 89716 4008 ? S 3月04 0:00 qmgr -l -t unix -u
root 42792 0.0 0.0 112676 980 pts/0 R+ 22:43 0:00 grep --color=auto qmgr
[root@localhost ~]# kill 1191
[root@localhost ~]# ps aux|grep qmgr
root 42794 0.0 0.0 112676 980 pts/0 R+ 22:44 0:00 grep --color=auto qmgr
STAT：进程状态。
D:不能中断的进程(通常为1O)
R(run):正在运行中的进程,其中包括了等待CPU时间片的进程。
S(sleep):已经中断的进程。通常情况下,系统的大部分进程都是这个状态。
T:已经停止或者暂停的进程。如果我们正在运行一个命令,比如说
seep10,我们按一下cm+z暂停进程时,用ps命令查看就会显示这个状
态。
W:(内核26xx以后不可用),没有足够的内存页分配
X:已经死掉的进程(这个好像从来不会出现
Z:僵尸进程,即杀不掉、打不死的垃圾进程,占用系统一点资源,不过没
有关系。如果占用太多(一般不会出现),就需要重视
<:高优先级进程。
N:低优先级进程
L:在内存中被锁了内存分页
s:主进程,后面阿铭讲到 nginx或者php-fm服务的时候,你就能更好地理解它了。
l:多线程进程
+:在前台运行的进程。
查看网络状态
系统监控状态
主要查看端口监听
[root@localhost ~]# netstat -lnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 12459/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1185/master
tcp6 0 0 :::22 :::* LISTEN 12459/sshd
tcp6 0 0 ::1:25 :::* LISTEN 1185/master
udp 0 0 127.0.0.1:323 0.0.0.0:* 584/chronyd
udp6 0 0 ::1:323 :::* 584/chronyd
Active UNIX domain sockets (only servers)
Proto RefCnt Flags Type State I-Node PID/Program name Path
unix 2 [ ACC ] STREAM LISTENING 19547 1185/master private/tlsmgr
unix 2 [ ACC ] STREAM LISTENING 19550 1185/master private/rewrite
unix 2 [ ACC ] STREAM LISTENING 19553 1185/master private/bounce
unix 2 [ ACC ] STREAM LISTENING 19556 1185/master private/defer
unix 2 [ ACC ] STREAM LISTENING 19559 1185/master private/trace
unix 2 [ ACC ] STREAM LISTENING 19562 1185/master private/verify
unix 2 [ ACC ] STREAM LISTENING 19568 1185/master private/proxymap
unix 2 [ ACC ] STREAM LISTENING 16475 574/VGAuthService /var/run/vmware/guestServicePipe
unix 2 [ ACC ] STREAM LISTENING 19540 1185/master public/cleanup
unix 2 [ ACC ] STREAM LISTENING 19543 1185/master public/qmgr
unix 2 [ ACC ] STREAM LISTENING 19565 1185/master public/flush
unix 2 [ ACC ] STREAM LISTENING 19580 1185/master public/showq
unix 2 [ ACC ] STREAM LISTENING 34437 1/systemd /run/systemd/private
unix 2 [ ACC ] SEQPACKET LISTENING 34447 1/systemd /run/udev/control
unix 2 [ ACC ] STREAM LISTENING 19536 1185/master public/pickup
unix 2 [ ACC ] STREAM LISTENING 9146 1/systemd /run/systemd/journal/stdout
unix 2 [ ACC ] STREAM LISTENING 19571 1185/master private/proxywrite
unix 2 [ ACC ] STREAM LISTENING 19574 1185/master private/smtp
unix 2 [ ACC ] STREAM LISTENING 19577 1185/master private/relay
unix 2 [ ACC ] STREAM LISTENING 19583 1185/master private/error
unix 2 [ ACC ] STREAM LISTENING 15064 1/systemd /var/run/dbus/system_bus_socket
unix 2 [ ACC ] STREAM LISTENING 19586 1185/master private/retry
unix 2 [ ACC ] STREAM LISTENING 19589 1185/master private/discard
unix 2 [ ACC ] STREAM LISTENING 13530 1/systemd /run/lvm/lvmetad.socket
unix 2 [ ACC ] STREAM LISTENING 19592 1185/master private/local
unix 2 [ ACC ] STREAM LISTENING 19595 1185/master private/virtual
unix 2 [ ACC ] STREAM LISTENING 19598 1185/master private/lmtp
unix 2 [ ACC ] STREAM LISTENING 19601 1185/master private/anvil
unix 2 [ ACC ] STREAM LISTENING 19604 1185/master private/scache
unix 2 [ ACC ] STREAM LISTENING 13548 1/systemd /run/lvm/lvmpolld.socket
查看TCP、UDP
[root@localhost ~]# netstat -ltnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 12459/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1185/master
tcp6 0 0 :::22 :::* LISTEN 12459/sshd
tcp6 0 0 ::1:25 :::* LISTEN 1185/master
[root@localhost ~]# netstat -ltunp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 12459/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1185/master
tcp6 0 0 :::22 :::* LISTEN 12459/sshd
tcp6 0 0 ::1:25 :::* LISTEN 1185/master
udp 0 0 127.0.0.1:323 0.0.0.0:* 584/chronyd
udp6 0 0 ::1:323 :::* 584/chronyd
查看与服务器连接的个数
[root@localhost ~]# netstat -an | awk '/^tcp/ {++sta[$NF]} END {for(key in sta) print key, "\t",sta[key]}'
LISTEN 4
ESTABLISHED 1 (通信个数)
[root@localhost ~]# ss -an | head
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port
nl UNCONN 0 0 0:0 *
nl UNCONN 4352 0 4:43599 *
nl UNCONN 768 0 4:0 *
nl UNCONN 0 0 6:0 *
nl UNCONN 0 0 7:1 *
nl UNCONN 0 0 7:571 *
nl UNCONN 0 0 7:0 *
nl UNCONN 0 0 7:1 *
nl UNCONN 0 0 7:571 *
Linux下抓包
监控系统状态
安装tcpdump
-nn选项的作用是让第3和4列显示成“IP + 端口号的形式”，如果不加则显示“主机名 + 服务名称”
[root@localhost ~]# yum install tcpdump
[root@localhost ~]# tcpdump -nn -i ens33
一般是tcp链接
udp多的话会容易是攻击
抓取指定的条件和抓取包数
（指定22端口并指定抓5个包）
[root@localhost ~]# tcpdump -nn -i ens33 port 22 -c 5
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), capture size 262144 bytes
21:20:46.013749 IP 172.16.79.140.22 > 172.16.79.1.62692: Flags [P.], seq 2469168778:2469168966, ack 4257817397, win 313, options [nop,nop,TS val 280397395 ecr 190515902], length 188
21:20:46.013960 IP 172.16.79.1.62692 > 172.16.79.140.22: Flags [.], ack 188, win 4090, options [nop,nop,TS val 190515927 ecr 280397395], length 0
21:20:46.014097 IP 172.16.79.140.22 > 172.16.79.1.62692: Flags [P.], seq 188:552, ack 1, win 313, options [nop,nop,TS val 280397396 ecr 190515927], length 364
21:20:46.014189 IP 172.16.79.1.62692 > 172.16.79.140.22: Flags [.], ack 552, win 4084, options [nop,nop,TS val 190515927 ecr 280397396], length 0
21:20:46.014319 IP 172.16.79.140.22 > 172.16.79.1.62692: Flags [P.], seq 552:900, ack 1, win 313, options [nop,nop,TS val 280397396 ecr 190515927], length 348
5 packets captured
6 packets received by filter
0 packets dropped by kernel
[root@localhost ~]# tcpdump -nn -i ens33 tcp and port 22 and not port 53 -c 5
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), capture size 262144 bytes
21:23:03.281779 IP 172.16.79.140.22 > 172.16.79.1.62692: Flags [P.], seq 2469178410:2469178598, ack 4257824933, win 313, options [nop,nop,TS val 280529101 ecr 190653093], length 188
21:23:03.281948 IP 172.16.79.1.62692 > 172.16.79.140.22: Flags [.], ack 188, win 4090, options [nop,nop,TS val 190653121 ecr 280529101], length 0
21:23:03.282071 IP 172.16.79.140.22 > 172.16.79.1.62692: Flags [P.], seq 188:552, ack 1, win 313, options [nop,nop,TS val 280529101 ecr 190653121], length 364
21:23:03.282181 IP 172.16.79.1.62692 > 172.16.79.140.22: Flags [.], ack 552, win 4084, options [nop,nop,TS val 190653121 ecr 280529101], length 0
21:23:03.282305 IP 172.16.79.140.22 > 172.16.79.1.62692: Flags [P.], seq 552:900, ack 1, win 313, options [nop,nop,TS val 280529101 ecr 190653121], length 348
5 packets captured
6 packets received by filter
0 packets dropped by kernel
保存抓取的数据包
[root@localhost ~]# tcpdump -nn -i ens33 tcp and port 22 and not port 53 -c 5 -w /tmp/1.cap
tcpdump: listening on ens33, link-type EN10MB (Ethernet), capture size 262144 bytes
5 packets captured
5 packets received by filter
0 packets dropped by kernel
[root@localhost ~]# file /tmp/1.cap
/tmp/1.cap: tcpdump capture file (little-endian) - version 2.4 (Ethernet, capture length 262144)
[root@localhost ~]# tcpdump -r /tmp/1.cap
reading from file /tmp/1.cap, link-type EN10MB (Ethernet)
21:29:10.767689 IP localhost.localdomain.ssh > 172.16.79.1.62692: Flags [P.], seq 2469180650:2469180774, ack 4257825561, win 313, options [nop,nop,TS val 280889853 ecr 191024097], length 124
21:29:10.767915 IP 172.16.79.1.62692 > localhost.localdomain.ssh: Flags [.], ack 124, win 4092, options [nop,nop,TS val 191024122 ecr 280889853], length 0
21:29:33.561721 IP 172.16.79.1.51894 > localhost.localdomain.ssh: Flags [SEW], seq 2336007396, win 65535, options [mss 1460,nop,wscale 5,nop,nop,TS val 191041985 ecr 0,sackOK,eol], length 0
21:29:33.561806 IP localhost.localdomain.ssh > 172.16.79.1.51894: Flags [S.E], seq 1178967180, ack 2336007397, win 28960, options [mss 1460,sackOK,TS val 280906701 ecr 191041985,nop,wscale 7], length 0
21:29:33.561995 IP 172.16.79.1.51894 > localhost.localdomain.ssh: Flags [.], ack 1, win 4117, options [nop,nop,TS val 191041985 ecr 280906701], length 0
安装wireshark
使用 tshark 查看web服务器
[root@localhost ~]# tshark -n -t a -R http.request -T fields -e "frame.time" -e "ip.src" -e "http.host" -e "http.request.method" -e "http.request.uri"
扩展
扩展tcp三次握手四次挥手 http://www.doc88.com/p-9913773324388.html
tshark几个用法：http://www.aminglinux.com/bbs/thread-995-1-1.html