从上午10:41分开始试验,一直到18:17,7个半小时左右。
从4台nginx服务器拷贝流量到测试服务器,由于lvs有两个虚拟ip,在在线服务器执行命令如下:
[root@bgp176-141 work]# ./tcpcopy xxx.xxx.xxx.140 80 xxx.xxx.xxx.148 18080
root@bgp176-142 work]# ./tcpcopy xxx.xxx.xxx.137 80 xxx.xxx.xxx.148 18080
[root@bgp176-143 work]# ./tcpcopy xxx.xxx.xxx.140 80 xxx.xxx.xxx.148 18080
[root@bgp176-144 work]# ./tcpcopy xxx.xxx.xxx.137 80 xxx.xxx.xxx.148 18080
上述4台服务器所copy的流量大概是在线服务器的2倍流量
测试服务器中途遇到ip queue full的情况,修改ip queue length后就没有再出现
echo 4096 > /proc/sys/net/ipv4/ip_queue_maxlen
cat /var/log/messages部分数据如下:
Nov 2 10:45:13 bgp176-148 kernel: printk: 118 messages suppressed.
Nov 2 10:45:13 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243413
Nov 2 10:45:13 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243414
Nov 2 10:45:13 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243415
Nov 2 10:45:13 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243416
Nov 2 10:45:13 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243417
Nov 2 10:45:13 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243418
Nov 2 10:45:13 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243419
Nov 2 10:45:13 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243420
Nov 2 10:45:13 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243421
Nov 2 10:45:13 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243422
Nov 2 10:56:53 bgp176-148 kernel: printk: 274 messages suppressed.
Nov 2 10:56:53 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243697
Nov 2 10:56:53 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243698
Nov 2 10:56:53 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243699
Nov 2 10:56:53 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243700
Nov 2 10:56:53 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243701
Nov 2 10:56:53 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243702
Nov 2 10:56:53 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243703
Nov 2 10:56:53 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243704
Nov 2 10:56:53 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243705
Nov 2 10:56:53 bgp176-148 kernel: ip_queue: full at 1024 entries, dropping packets(s). Dropped: 2243706
Nov 2 11:18:42 bgp176-148 ntpd[1851]: time reset +0.212364 s
Nov 2 11:22:56 bgp176-148 ntpd[1851]: synchronized to LOCAL(0), stratum 10
Nov 2 11:23:33 bgp176-148 ntpd[1851]: synchronized to 220.181.28.157, stratum 2
Nov 2 11:27:15 bgp176-148 ntpd[1851]: synchronized to 122.226.192.4, stratum 2
执行到18点多的时候,我们来分析使用情况
root@bgp176-148 logs]# date
Wed Nov 2 18:17:02 CST 2011root 7855 34.6 4.4 94052 92708 pts/1 S+ 10:40 158:09 ./interception
root 27198 21.3 0.7 40952 28068 pts/0 Rl+ 10:41 97:24 ./tcpcopy xxx.xxx.xxx.140 80 xxx.xxx.xxx.148 18080
测试服务器数据如下:
7830 adrun 15 0 22340 18m 932 S 30.8 0.9 202:34.59 nginx
7832 adrun 15 0 22456 19m 932 S 30.8 0.9 206:58.98 nginx
7829 adrun 15 0 21848 18m 932 S 29.7 0.9 181:29.62 nginx
7831 adrun 15 0 22084 18m 932 S 27.4 0.9 192:06.58 nginx
7855 root 15 0 94052 90m 400 S 25.1 4.5 159:41.30 interception
5363 root 18 0 37764 23m 744 S 17.1 1.2 155:03.92 asyn_server
我们分析请求处理情况:
[wangbin@bgp176-141 logs]$ grep '2011:18:35' access.log |wc -l[root@bgp176-148 logs]# tail -n 1000000 access.log |grep '2011:18:35' |wc -l
139653
2倍流量=69737×2=139474,接近在线流量的2倍
中间的情况如下:
[root@bgp176-148 logs]# tail -n 1000000 access.log |grep '2011:13:36' |wc -l
191913
[root@bgp176-148 logs]# tail -n 1000000 access.log |grep '2011:13:37' |wc -l
190548
[root@bgp176-148 logs]# tail -n 1000000 access.log |grep '2011:13:44' |wc -l
190721
[root@bgp176-148 logs]# tail -n 1000000 access.log |grep '2011:13:47' |wc -l
189788
[wangbin@bgp176-141 logs]$ grep '2011:13:37' access.log |wc -l
97498
[wangbin@bgp176-141 logs]$ grep '2011:13:36' access.log |wc -l
97693
[wangbin@bgp176-141 logs]$ grep '2011:13:44' access.log |wc -l
94891
[wangbin@bgp176-141 logs]$ grep '2011:13:47' access.log |wc -l
94464
两倍=97498×2=194996
两倍=97693×2=195386
两倍=94891×2=189782
两倍=94464×2=188928
我们可以得出结论,通过tcpcopy,nginx目前能够承受2倍的在线压力