连接失效问题
例子
其中,Redis常见的报错就是:
配置项:timeout
报错信息:Error while reading line from the server
Redis可以配置如果客户端经过多少秒还不给Redis服务器发送数据,那么就会把连接close掉。
MySQL常见的报错:
配置项:wait_timeout & interactive_timeout
报错信息:has gone away
和Redis服务器一样,MySQL也会定时的去清理掉没用的连接。
如何解决
1、用的时候进行重连
2、定时发送心跳维持连接
用的时候进行重连
优点是简单,缺点是面临短连接的问题。
定时发送心跳维持连接
推荐。
如何维持长连接
tcp协议中实现的tcp_keepalive
操作系统底层提供了一组tcp的keepalive配置:
1 tcp_keepalive_time (integer; default: 7200; since Linux 2.2) 2 The number of seconds a connection needs to be idle before TCP 3 begins sending out keep-alive probes. Keep-alives are sent only 4 when the SO_KEEPALIVE socket option is enabled. The default 5 value is 7200 seconds (2 hours). An idle connection is 6 terminated after approximately an additional 11 minutes (9 7 probes an interval of 75 seconds apart) when keep-alive is 8 enabled. 9 10 Note that underlying connection tracking mechanisms and 11 application timeouts may be much shorter. 12 13 tcp_keepalive_intvl (integer; default: 75; since Linux 2.4) 14 The number of seconds between TCP keep-alive probes. 15 16 tcp_keepalive_probes (integer; default: 9; since Linux 2.2) 17 The maximum number of TCP keep-alive probes to send before 18 giving up and killing the connection if no response is obtained 19 from the other end. 20 8
Swoole
底层把这些配置开放出来了,例如:
1 ?php 2 3 $server = new \Swoole\Server('127.0.0.1', 6666, SWOOLE_PROCESS); 4 5 $server->set([ 6 'worker_num' => 1, 7 'open_tcp_keepalive' => 1, 8 'tcp_keepidle' => 4, // 对应tcp_keepalive_time 9 'tcp_keepinterval' => 1, // 对应tcp_keepalive_intvl 10 'tcp_keepcount' => 5, // 对应tcp_keepalive_probes 11 ]);
其中:
1 'open_tcp_keepalive' => 1, // 总开关,用来开启tcp_keepalive 2 'tcp_keepidle' => 4, // 4s没有数据传输就进行检测 3 // 检测的策略如下: 4 'tcp_keepinterval' => 1, // 1s探测一次,即每隔1s给客户端发一个包(然后客户端可能会回一个ack的包,如果服务端收到了这个ack包,那么说明这个连接是活着的) 5 'tcp_keepcount' => 5, // 探测的次数,超过5次后客户端还没有回ack包,那么close此连接
我们来实战测试体验一下,服务端脚本如下:
1 php 2 3 $server = new \Swoole\Server('127.0.0.1', 6666, SWOOLE_PROCESS); 4 5 $server->set([ 6 'worker_num' => 1, 7 'open_tcp_keepalive' => 1, // 开启tcp_keepalive 8 'tcp_keepidle' => 4, // 4s没有数据传输就进行检测 9 'tcp_keepinterval' => 1, // 1s探测一次 10 'tcp_keepcount' => 5, // 探测的次数,超过5次后还没有回包close此连接 11 ]); 12 13 $server->on('connect', function ($server, $fd) { 14 var_dump("Client: Connect $fd"); 15 }); 16 17 $server->on('receive', function ($server, $fd, $reactor_id, $data) { 18 var_dump($data); 19 }); 20 21 $server->on('close', function ($server, $fd) { 22 var_dump("close fd $fd"); 23 }); 24 25 $server->start();
我们启动这个服务器:
1 ~/codeDir/phpCode/hyperf-skeleton # php server.php
然后通过tcpdump
进行抓包:
~/codeDir/phpCode/hyperf-skeleton # tcpdump -i lo port 6666 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes
我们此时正在监听lo
上的6666
端口的数据包。
然后我们用客户端去连接它:
1 ~/codeDir/phpCode/hyperf-skeleton # nc 127.0.0.1 6666
此时服务端会打印出消息:
~/codeDir/phpCode/hyperf-skeleton # php server.php string(17) "Client: Connect 1"
tcpdump
的输出信息如下:
1 01:48:40.178439 IP localhost.33933 > localhost.6666: Flags [S], seq 43162537, win 43690, options [mss 65495,sackOK,TS val 9833698 ecr 0,nop,wscale 7], length 0 2 01:48:40.178484 IP localhost.6666 > localhost.33933: Flags [S.], seq 1327460565, ack 43162538, win 43690, options [mss 65495,sackOK,TS val 9833698 ecr 9833698,nop,wscale 7], length 0 3 01:48:40.178519 IP localhost.33933 > localhost.6666: Flags [.], ack 1, win 342, options [nop,nop,TS val 9833698 ecr 9833698], length 0 4 01:48:44.229926 IP localhost.6666 > localhost.33933: Flags [.], ack 1, win 342, options [nop,nop,TS val 9834104 ecr 9833698], length 0 5 01:48:44.229951 IP localhost.33933 > localhost.6666: Flags [.], ack 1, win 342, options [nop,nop,TS val 9834104 ecr 9833698], length 0 6 01:48:44.229926 IP localhost.6666 > localhost.33933: Flags [.], ack 1, win 342, options [nop,nop,TS val 9834104 ecr 9833698], length 0 7 01:48:44.229951 IP localhost.33933 > localhost.6666: Flags [.], ack 1, win 342, options [nop,nop,TS val 9834104 ecr 9833698], length 0 8 01:48:44.229926 IP localhost.6666 > localhost.33933: Flags [.], ack 1, win 342, options [nop,nop,TS val 9834104 ecr 9833698], length 0 9 // 省略了其他的输出
我们会发现最开始的时候,会打印三次握手的包:
01:48:40.178439 IP localhost.33933 > localhost.6666: Flags [S], seq 43162537, win 43690, options [mss 65495,sackOK,TS val 9833698 ecr 0,nop,wscale 7], length 0 01:48:40.178484 IP localhost.6666 > localhost.33933: Flags [S.], seq 1327460565, ack 43162538, win 43690, options [mss 65495,sackOK,TS val 9833698 ecr 9833698,nop,wscale 7], length 0 01:48:40.178519 IP localhost.33933 > localhost.6666: Flags [.], ack 1, win 342, options [nop,nop,TS val 9833698 ecr 9833698], length 0
然后,停留了4s
没有任何包的输出。
之后,每隔1s
左右就会打印出一组:
1 01:52:54.359341 IP localhost.6666 > localhost.43101: Flags [.], ack 1, win 342, options [nop,nop,TS val 9859144 ecr 9858736], length 0 2 01:52:54.359377 IP localhost.43101 > localhost.6666: Flags [.], ack 1, win 342, options [nop,nop,TS val 9859144 ecr 9855887], length 0
其实这就是我们配置的策略:
1 'tcp_keepinterval' => 1, // 1s探测一次 2 'tcp_keepcount' => 5, // 探测的次数,超过5次后还没有回包close此连接
因为我们操作系统底层会自动的给客户端回ack
,所以这个连接不会在5
次探测后被关闭。操作系统底层会持续不断的发送这样的一组包:
1 01:52:54.359341 IP localhost.6666 > localhost.43101: Flags [.], ack 1, win 342, options [nop,nop,TS val 9859144 ecr 9858736], length 0 2 01:52:54.359377 IP localhost.43101 > localhost.6666: Flags [.], ack 1, win 342, options [nop,nop,TS val 9859144 ecr 9855887], length 0
如果我们要测试5
次探测后关闭这个连接,可以禁掉6666
端口的包:
1 ~/codeDir/phpCode/hyperf-skeleton # iptables -A INPUT -p tcp --dport 6666 -j DROP
这样会把所有从6666
端口进来的包给禁掉,自然,服务器就接收不到从客户端那一边发来的ack
包了。
然后服务器过5
秒就会打印出close
(服务端主动的调用了close
方法,给客户端发送了FIN
包):
1 ~/codeDir/phpCode/hyperf-skeleton # php server.php 2 string(17) "Client: Connect 1" 3 string(10) "close fd 1"
我们恢复一下iptables
的规则:
1 ~/codeDir/phpCode # iptables -D INPUT -p tcp -m tcp --dport 6666 -j DROP
即把我们设置的规则给删除了。
通过tcp_keepalive的方式实现心跳的功能,优点是简单,不要写代码就可以完成这个功能,并且发送的心跳包小。缺点是依赖于系统的网络环境,必须保证服务器和客户端都实现了这样的功能,需要客户端配合发心跳包。还有一个更为严重的缺点是如果客户端和服务器不是直连的,而是通过代理来进行连接的,例如socks5代理,它只会转发应用层的包,不会转发更为底层的tcp探测包,那这个心跳功能就失效了。
所以,Swoole就提供了其他的解决方案,一组检测死连接的配置。
1 'heartbeat_check_interval' => 1, // 1s探测一次 2 'heartbeat_idle_time' => 5, // 5s未发送数据包就close此连接
swoole实现的heartbeat
我们来测试一下:
1 php 2 3 $server = new \Swoole\Server('127.0.0.1', 6666, SWOOLE_PROCESS); 4 5 $server->set([ 6 'worker_num' => 1, 7 'heartbeat_check_interval' => 1, // 1s探测一次 8 'heartbeat_idle_time' => 5, // 5s未发送数据包就close此连接 9 ]); 10 11 $server->on('connect', function ($server, $fd) { 12 var_dump("Client: Connect $fd"); 13 }); 14 15 $server->on('receive', function ($server, $fd, $reactor_id, $data) { 16 var_dump($data); 17 }); 18 19 $server->on('close', function ($server, $fd) { 20 var_dump("close fd $fd"); 21 }); 22 23 $server->start();
然后启动服务器:
1 ~/codeDir/phpCode/hyperf-skeleton # php server.php
然后启动tcpdump
:
1 ~/codeDir/phpCode # tcpdump -i lo port 6666 2 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode 3 listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes
然后再启动客户端:
1 ~/codeDir/phpCode/hyperf-skeleton # nc 127.0.0.1 6666
此时服务器端打印:
1 ~/codeDir/phpCode/hyperf-skeleton # php server.php 2 string(17) "Client: Connect 1"
然后tcpdump打印:
1 02:48:32.516093 IP localhost.42123 > localhost.6666: Flags [S], seq 1088388248, win 43690, options [mss 65495,sackOK,TS val 10193342 ecr 0,nop,wscale 7], length 0 2 02:48:32.516133 IP localhost.6666 > localhost.42123: Flags [S.], seq 80508236, ack 1088388249, win 43690, options [mss 65495,sackOK,TS val 10193342 ecr 10193342,nop,wscale 7], length 0 3 02:48:32.516156 IP localhost.42123 > localhost.6666: Flags [.], ack 1, win 342, options [nop,nop,TS val 10193342 ecr 10193342], length 0
这是三次握手信息。
然后过了5s后,tcpdump会打印出:
1 02:48:36.985027 IP localhost.6666 > localhost.42123: Flags [F.], seq 1, ack 1, win 342, options [nop,nop,TS val 10193789 ecr 10193342], length 0 2 02:48:36.992172 IP localhost.42123 > localhost.6666: Flags [.], ack 2, win 342, options [nop,nop,TS val 10193790 ecr 10193789], length 0
也就是服务端发送了FIN包。因为客户端没有发送数据,所以Swoole关闭了连接。
然后服务器端会打印:
1 ~/codeDir/phpCode/hyperf-skeleton # php server.php 2 string(17) "Client: Connect 1" 3 string(10) "close fd 1"
所以,heartbeat和tcp keepalive还是有一定的区别的,tcp keepalive有保活连接的功能,但是heartbeat存粹是检测没有数据的连接,然后关闭它,并且只可以在服务端这边配置,如果需要保活,也可以让客户端配合发送心跳。
如果我们不想让服务端close掉连接,那么就得在应用层里面不断的发送数据包来进行保活,例如我在nc客户端里面不断的发送包:
1 ~/codeDir/phpCode/hyperf-skeleton # nc 127.0.0.1 6666 2 ping 3 ping 4 ping 5 ping 6 ping 7 ping 8 ping 9 ping 10 ping
我发送了9个ping包给服务器,tcpdump的输出如下:
1 // 省略了三次握手的包 2 02:57:53.697363 IP localhost.44195 > localhost.6666: Flags [P.], seq 1:6, ack 1, win 342, options [nop,nop,TS val 10249525 ecr 10249307], length 5 3 02:57:53.697390 IP localhost.6666 > localhost.44195: Flags [.], ack 6, win 342, options [nop,nop,TS val 10249525 ecr 10249525], length 0 4 02:57:55.309532 IP localhost.44195 > localhost.6666: Flags [P.], seq 6:11, ack 1, win 342, options [nop,nop,TS val 10249686 ecr 10249525], length 5 5 02:57:55.309576 IP localhost.6666 > localhost.44195: Flags [.], ack 11, win 342, options [nop,nop,TS val 10249686 ecr 10249686], length 0 6 02:57:58.395206 IP localhost.44195 > localhost.6666: Flags [P.], seq 11:16, ack 1, win 342, options [nop,nop,TS val 10249994 ecr 10249686], length 5 7 02:57:58.395239 IP localhost.6666 > localhost.44195: Flags [.], ack 16, win 342, options [nop,nop,TS val 10249994 ecr 10249994], length 0 8 02:58:01.858094 IP localhost.44195 > localhost.6666: Flags [P.], seq 16:21, ack 1, win 342, options [nop,nop,TS val 10250341 ecr 10249994], length 5 9 02:58:01.858126 IP localhost.6666 > localhost.44195: Flags [.], ack 21, win 342, options [nop,nop,TS val 10250341 ecr 10250341], length 0 10 02:58:04.132584 IP localhost.44195 > localhost.6666: Flags [P.], seq 21:26, ack 1, win 342, options [nop,nop,TS val 10250568 ecr 10250341], length 5 11 02:58:04.132609 IP localhost.6666 > localhost.44195: Flags [.], ack 26, win 342, options [nop,nop,TS val 10250568 ecr 10250568], length 0 12 02:58:05.895704 IP localhost.44195 > localhost.6666: Flags [P.], seq 26:31, ack 1, win 342, options [nop,nop,TS val 10250744 ecr 10250568], length 5 13 02:58:05.895728 IP localhost.6666 > localhost.44195: Flags [.], ack 31, win 342, options [nop,nop,TS val 10250744 ecr 10250744], length 0 14 02:58:07.150265 IP localhost.44195 > localhost.6666: Flags [P.], seq 31:36, ack 1, win 342, options [nop,nop,TS val 10250870 ecr 10250744], length 5 15 02:58:07.150288 IP localhost.6666 > localhost.44195: Flags [.], ack 36, win 342, options [nop,nop,TS val 10250870 ecr 10250870], length 0 16 02:58:08.349124 IP localhost.44195 > localhost.6666: Flags [P.], seq 36:41, ack 1, win 342, options [nop,nop,TS val 10250990 ecr 10250870], length 5 17 02:58:08.349156 IP localhost.6666 > localhost.44195: Flags [.], ack 41, win 342, options [nop,nop,TS val 10250990 ecr 10250990], length 0 18 02:58:09.906223 IP localhost.44195 > localhost.6666: Flags [P.], seq 41:46, ack 1, win 342, options [nop,nop,TS val 10251145 ecr 10250990], length 5 19 02:58:09.906247 IP localhost.6666 > localhost.44195: Flags [.], ack 46, win 342, options [nop,nop,TS val 10251145 ecr 10251145], length 0
有9组数据包的发送。(这里的Flags [P.]代表Push的含义)
此时服务器还没有close掉连接,实现了客户端保活连接的功能。然后我们停止发送ping,过了5秒后tcpdump就会输出一组:
02:58:14.811761 IP localhost.6666 > localhost.44195: Flags [F.], seq 1, ack 46, win 342, options [nop,nop,TS val 10251636 ecr 10251145], length 0
02:58:14.816420 IP localhost.44195 > localhost.6666: Flags [.], ack 2, win 342, options [nop,nop,TS val 10251637 ecr 10251636], length 0
服务端那边发送了FIN包,说明服务端close掉了连接。服务端的输出如下:
1 ~/codeDir/phpCode/hyperf-skeleton # php server.php 2 string(17) "Client: Connect 1" 3 string(5) "ping 4 " 5 string(5) "ping 6 " 7 string(5) "ping 8 " 9 string(5) "ping 10 " 11 string(5) "ping 12 " 13 string(5) "ping 14 " 15 string(5) "ping 16 " 17 string(5) "ping 18 " 19 string(5) "ping 20 " 21 string(10) "close fd 1"
然后我们在客户端那边ctrl + c来关闭连接:
1 ~/codeDir/phpCode/hyperf-skeleton # nc 127.0.0.1 6666 2 ping 3 ping 4 ping 5 ping 6 ping 7 ping 8 ping 9 ping 10 ping 11 ^Cpunt! 12 13 ~/codeDir/phpCode/hyperf-skeleton #
此时,tcpdump的输出如下:
1 03:03:02.257667 IP localhost.44195 > localhost.6666: Flags [F.], seq 46, ack 2, win 342, options [nop,nop,TS val 10280414 ecr 10251636], length 0 2 03:03:02.257734 IP localhost.6666 > localhost.44195: Flags [R], seq 2678621620, win 0, length 0
应用层心跳
1、制定ping/pong协议(mysql等自带ping协议)
2、客户端灵活的发送ping心跳包
3、服务端OnRecive检查可用性回复pong
例如:
1 $server->on('receive', function (\Swoole\Server $server, $fd, $reactor_id, $data) 2 { 3 if ($data == 'ping') 4 { 5 checkDB(); 6 checkServiceA(); 7 checkRedis(); 8 $server->send('pong'); 9 } 10 });
结论
1、tcp的keepalive最简单,但是有兼容性问题,不够灵活
2、swoole提供的keepalive最实用,但是需要客户端配合,复杂度适中
3、应用层的keepalive最灵活但是最麻烦