【性能优化】在容器环境使用 tcpdump 抓包

1.tcpdump 使用简介

$ tcpdump -h
tcpdump version 4.9.3
libpcap version 1.9.1 (with TPACKET_V3)
OpenSSL 1.1.1f  31 Mar 2020
Usage: tcpdump [-aAbdDefhHIJKlLnNOpqStuUvxX#] [ -B size ] [ -c count ]
                [ -C file_size ] [ -E algo:secret ] [ -F file ] [ -G seconds ]
                [ -i interface ] [ -j tstamptype ] [ -M secret ] [ --number ]
                [ -Q in|out|inout ]
                [ -r file ] [ -s snaplen ] [ --time-stamp-precision precision ]
                [ --immediate-mode ] [ -T type ] [ --version ] [ -V file ]
                [ -w file ] [ -W filecount ] [ -y datalinktype ] [ -z postrotate-command ]
                [ -Z user ] [ expression ]

# -n :不解析地址,直接以 IP 及 port number 显示,而非主机名与服务名称
# -i :后面接要『监听』的网络接口,例如 eth0, lo, ppp0 等等
# -c :抓包数量,如果没有这个参数, tcpdump 会持续不断的监听,直到使用者输入 [ctrl]-c 为止   
# -e :输出dump的报文的链路层(OSI 第二层)信息(包括源目MAC和以太类型)
# -q :quick mode,仅列出较为简短的封包信息,每一行的内容比较精简
# -s :抓包的长度,默认是96字节。如果要把报文保存下来分析,需配置此参数
# -A :抓包内容以 ASCII 显示,做WWW 的网页很有用
# -X :可以列出十六进制 (hex) 以及 ASCII 的抓包内容,对于观察内容很有用
# -r :从后面接的文件中读取数据,这个『文件』是由 -w 所生成的
# -w :把抓取数据保存到文件,后面接文件名
# -v :verbose output 打印详细的输出
# -vv :more verbose output,打印更加详细的输出

常用的expression
host:匹配v4/v6地址,可以匹配源和目的。
src host:匹配源v4/v6地址
dst host:匹配目的v4/v6地址
ether src:匹配源MAC
ether dst:匹配目的MAC
net:匹配网络地址范围
src net:匹配源网络地址范围
dst net: 匹配目的网络地址范围
port:匹配端口号
src port:匹配源端口号
dst port:匹配目的端口号
ip proto:匹配IP协议号,可以跟数字或者知名的协议名
ether proto: 匹配以太类型
vlan: 匹配VLAN号
缩写: ip ip6 arp stp tcp udp icmp(ether proto和ip proto的缩写)
逻辑符:not and or,not的优先级最高,and和or的优先级相同,从左往右依次生效

tcpdump 的表达式样例

# 1.截取 1.1.1.1 主机所有收到的和发出的的报文,并把信息打印到 stdout
$ tcpdump 'host 1.1.1.1'

# 2. 截取主机 1.1.1.1 和主机 1.1.1.2 或 192.168.111.3 的通信,并且不做解析
$ tcpdump 'host 1.1.1.1 and 1.1.1.2 or 192.168.111.3' -n

# 3.截取源 IP 为 210.2.2.4 主机给目的IP为 192.168.0.0 的报文:
$ tcpdump 'src host 210.2.2.4 and dst net 192.168.0.0'

# 4.截取主机为 210.2.2.4 和端口号为 8081 的报文:
$ tcpdump 'host 210.2.2.4 and port 8081'

# 5.截取源 IP 是 210.2.2.4 主机发送给目的 IP 和端口为 192.168.0.0:8081 的报文,并把它保存到 tcp_dump_data.cap 文件中
$ tcpdump 'src host 210.2.2.4 and dst host 192.168.0.0 and dst port 8081'  -w tcp_dump_data.cap

# 6.截取源 IP 是 210.2.2.4,发送的所有的经过 eth0 网卡的所有报文
tcpdump 'src host 210.2.2.4' -i eth0

2.在宿主机上使用 tcpdump 抓取容器的报文

关键点:在宿主机监听容器 IP 和 端口,需要利用工具 nsenter 进入容器进程的 network namesapce,例如 nsenter -t 14885 -n -p 表示进入进程 14885 的 network namespace、pid namespace

# 1.在宿主机上获取容器Id
# 查询 svc 端口
$  kubectl get svc -ntest -owide |grep flink
service-flink-jobmanager           ClusterIP   10.96.0.62            8123/TCP,8124/TCP,8089/TCP,50010/TCP                                                                 27d     app=flink,component=jobmanager

# ssh到容器所在的宿主机
$ kubectl get pod -ntest -owide |grep flink-jobmanager
flink-jobmanager-5bdf56747-g4bsj    1/1     Running   0          5d17h   10.244.3.81     worker-0010
$ ssh worker-0010
$ docker ps |grep flink-jobmanager-7b58565dc8-msgpp
04d1b1899c2a        192.168.31.37:5000/flink                      "bash -cx /opt/flink/s…"   3 weeks ago         Up 3 weeks                              k8s_flink-jobmanager-7b58565dc8-msgpp_4b6d15d8-7b54-41fb-bf46-ffa91aa33963_0
567ba6d9a31f        192.168.31.37:5000/pause:3.2                           "/pause"                 3 weeks ago         Up 3 weeks                              k8s_POD_flink-jobmanager-7b58565dc8-msgpp_4b6d15d8-7b54-41fb-bf46-ffa91aa33963_0

# 2.根据容器Id查询容器在宿主机上的Pid
$ docker inspect 04d1b1899c2a |grep Pid
            "Pid": 63046,
            "PidMode": "",
            "PidsLimit": 0,

# 3.进入容器Pid所在的network namespace
$ nsenter -t 63046 -n

# 4.使用 tcpdump 抓包
tcpdump 'src host 10.244.3.81 and src port 8089'
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
01:54:49.563413 IP worker-0010.8089 > _gateway.20444: Flags [S.], seq 2899382800, ack 16865042, win 27960, options [mss 1410,sackOK,TS val 3697771296 ecr 1082429947,nop,wscale 7], length 0
01:54:49.563708 IP worker-0010.8089 > _gateway.20444: Flags [.], ack 262, win 227, options [nop,nop,TS val 3697771296 ecr 1082429947], length 0
01:54:49.565766 IP worker-0010.8089 > _gateway.20444: Flags [P.], seq 1:1278, ack 262, win 227, options [nop,nop,TS val 3697771298 ecr 1082429947], length 1277
01:54:49.566762 IP worker-0010.8089 > _gateway.20444: Flags [P.], seq 1278:1329, ack 355, win 227, options [nop,nop,TS val 3697771299 ecr 1082429950], length 51
01:54:49.567561 IP worker-0010.8089 > _gateway.20444: Flags [P.], seq 1329:1461, ack 509, win 236, options [nop,nop,TS val 3697771300 ecr 1082429951], length 132
01:54:49.567849 IP worker-0010.8089 > _gateway.20444: Flags [P.], seq 1461:1492, ack 541, win 236, options [nop,nop,TS val 3697771300 ecr 1082429951], length 31
01:54:51.987146 IP worker-0010.8089 > _gateway.20486: Flags [S.], seq 242954397, ack 3505559798, win 27960, options [mss 1410,sackOK,TS val 3697773720 ecr 1082432371,nop,wscale 7], length 0
01:54:51.987423 IP worker-0010.8089 > _gateway.20486: Flags [.], ack 262, win 227, options [nop,nop,TS val 3697773720 ecr 1082432371], length 0
01:54:51.989664 IP worker-0010.8089 > _gateway.20486: Flags [P.], seq 1:1278, ack 262, win 227, options [nop,nop,TS val 3697773722 ecr 1082432371], length 1277
01:54:51.990511 IP worker-0010.8089 > _gateway.20486: Flags [P.], seq 1278:1329, ack 355, win 227, options [nop,nop,TS val 3697773723 ecr 1082432374], length 51
01:54:51.991106 IP worker-0010.8089 > _gateway.20486: Flags [P.], seq 1329:1461, ack 508, win 236, options [nop,nop,TS val 3697773724 ecr 1082432374], length 132
01:54:51.991288 IP worker-0010.8089 > _gateway.20486: Flags [P.], seq 1461:1492, ack 540, win 236, options [nop,nop,TS val 3697773724 ecr 1082432375], length 31
01:54:52.925539 IP worker-0010.8089 > 10.244.3.195.54930: Flags [S.], seq 2047958668, ack 2355521248, win 27960, options [mss 1410,sackOK,TS val 2189379360 ecr 489972800,nop,wscale 7], length 0
01:54:52.927597 IP worker-0010.8089 > 10.244.3.195.54930: Flags [.], ack 317, win 227, options [nop,nop,TS val 2189379362 ecr 489972802], length 0
01:54:52.928100 IP worker-0010.8089 > 10.244.3.195.54930: Flags [P.], seq 1:148, ack 317, win 227, options [nop,nop,TS val 2189379363 ecr 489972802], length 147
01:54:52.929826 IP worker-0010.8089 > 10.244.3.195.54930: Flags [.], ack 1721, win 249, options [nop,nop,TS val 2189379364 ecr 489972804], length 0
01:54:52.944145 IP worker-0010.8089 > 10.244.3.195.54930: Flags [P.], seq 148:324, ack 2116, win 271, options [nop,nop,TS val 2189379379 ecr 489972805], length 176
01:54:52.944241 IP worker-0010.8089 > 10.244.3.195.54930: Flags [P.], seq 324:1892, ack 2116, win 271, options [nop,nop,TS val 2189379379 ecr 489972805], length 1568
01:54:52.944373 IP worker-0010.8089 > 10.244.3.195.54930: Flags [P.], seq 1892:1926, ack 2116, win 271, options [nop,nop,TS val 2189379379 ecr 489972805], length 34
01:54:52.947224 IP worker-0010.8089 > 10.244.3.195.54932: Flags [S.], seq 3172374436, ack 1388865001, win 27960, options [mss 1410,sackOK,TS val 2189379382 ecr 489972822,nop,wscale 7], length 0
01:54:52.959977 IP worker-0010.8089 > 10.244.3.195.54932: Flags [.], ack 317, win 227, options [nop,nop,TS val 2189379395 ecr 489972835], length 0
...省略

参考
1.Linux tcpdump 命令
2.如何学习 tcpdump 表达式?

你可能感兴趣的:(【性能优化】在容器环境使用 tcpdump 抓包)