python httplib [Errno 104] Connection reset by peer问题

写了一个简单的http client 方法,代码如下

import httplib
import json
from common.logger import sys_logger

def http_req(address, path, method, data={}, headers={}, content_type='application/json'):
    conn = httplib.HTTPConnection(address)
    headers['Content-type'] = content_type
    conn.request(method, path, json.dumps(data), headers)
    response = conn.getresponse()
    status = response.status
    resp_data = response.read()
    if resp_data and resp_data != '':
        sys_logger.debug('get response data :' + resp_data)
        resp_data = json.loads(resp_data)
    conn.close()
    return status, resp_data


但是出现奇怪的问题:

同一个请求,时不时出现

[Errno 104] Connection reset by peer的问题,大多数时候请求成功。这非常影响功能稳定性,不能接受。

StackOverFlow上有位同学写了这么段话,大概就是说TCP就是这么设计的,你能奈何?


"Connection reset by peer" is the TCP/IP equivalent of slamming the phone back on the hook. It's more polite than merely not replying, leaving one hanging. But it's not the FIN-ACK expected of the truly polite TCP/IP converseur.


只好做成异常重试,改造后代码如下,

对于重试请求没有负面影响的场景,尽量都调用带重试的方法,基本正常工作了。

import httplib
import json
from common.logger import sys_logger
from socket import error as SocketError
import errno
import time

def http_req(address, path, method, data={}, headers={}, content_type='application/json'):
    conn = httplib.HTTPConnection(address)
    headers['Content-type'] = content_type
    conn.request(method, path, json.dumps(data), headers)
    response = conn.getresponse()
    status = response.status
    resp_data = response.read()
    if resp_data and resp_data != '':
        sys_logger.debug('get response data :' + resp_data)
        resp_data = json.loads(resp_data)
    conn.close()
    return status, resp_data

def http_req_retrys(address, path, method, data={}, headers={}, content_type='application/json',try_times=5):
    sys_logger.debug('retrys : %d' % try_times)
    if try_times <=0:
        raise Exception('http request failed after retrys')
    try:
        return http_req(address, path, method, data, headers, content_type)
    except SocketError as e:
        if e.errno != errno.ECONNRESET:
            raise
        time.sleep(0.5)
        return http_req_retrys(address, path, method, data, headers, content_type, try_times - 1)



你可能感兴趣的:(python)