聊聊HttpClient的KeepAlive
序
本文主要研究一下HttpClient的KeepAlive
ConnectionKeepAliveStrategy
org/apache/http/conn/ConnectionKeepAliveStrategy.java
public interface ConnectionKeepAliveStrategy {
/**
* Returns the duration of time which this connection can be safely kept
* idle. If the connection is left idle for longer than this period of time,
* it MUST not reused. A value of 0 or less may be returned to indicate that
* there is no suitable suggestion.
*
* When coupled with a {@link org.apache.http.ConnectionReuseStrategy}, if
* {@link org.apache.http.ConnectionReuseStrategy#keepAlive(
* HttpResponse, HttpContext)} returns true, this allows you to control
* how long the reuse will last. If keepAlive returns false, this should
* have no meaningful impact
*
* @param response
* The last response received over the connection.
* @param context
* the context in which the connection is being used.
*
* @return the duration in ms for which it is safe to keep the connection
* idle, or <=0 if no suggested duration.
*/
long getKeepAliveDuration(HttpResponse response, HttpContext context);
}
ConnectionKeepAliveStrategy接口定义了getKeepAliveDuration方法，用于返回该connection空间多久以内被复用是安全的
DefaultConnectionKeepAliveStrategy
org/apache/http/impl/client/DefaultConnectionKeepAliveStrategy.java
@Contract(threading = ThreadingBehavior.IMMUTABLE)
public class DefaultConnectionKeepAliveStrategy implements ConnectionKeepAliveStrategy {
public static final DefaultConnectionKeepAliveStrategy INSTANCE = new DefaultConnectionKeepAliveStrategy();
@Override
public long getKeepAliveDuration(final HttpResponse response, final HttpContext context) {
Args.notNull(response, "HTTP response");
final HeaderElementIterator it = new BasicHeaderElementIterator(
response.headerIterator(HTTP.CONN_KEEP_ALIVE));
while (it.hasNext()) {
final HeaderElement he = it.nextElement();
final String param = he.getName();
final String value = he.getValue();
if (value != null && param.equalsIgnoreCase("timeout")) {
try {
return Long.parseLong(value) * 1000;
} catch(final NumberFormatException ignore) {
}
}
}
return -1;
}
}
DefaultConnectionKeepAliveStrategy实现了ConnectionKeepAliveStrategy接口，它主要是从response的Keep-Alive的header读取timeout参数
ConnectionReuseStrategy
org/apache/http/ConnectionReuseStrategy.java
public interface ConnectionReuseStrategy {
/**
* Decides whether a connection can be kept open after a request.
* If this method returns {@code false}, the caller MUST
* close the connection to correctly comply with the HTTP protocol.
* If it returns {@code true}, the caller SHOULD attempt to
* keep the connection open for reuse with another request.
*
* One can use the HTTP context to retrieve additional objects that
* may be relevant for the keep-alive strategy: the actual HTTP
* connection, the original HTTP request, target host if known,
* number of times the connection has been reused already and so on.
*
*
* If the connection is already closed, {@code false} is returned.
* The stale connection check MUST NOT be triggered by a
* connection reuse strategy.
*
*
* @param response
* The last response received over that connection.
* @param context the context in which the connection is being
* used.
*
* @return {@code true} if the connection is allowed to be reused, or
* {@code false} if it MUST NOT be reused
*/
boolean keepAlive(HttpResponse response, HttpContext context);
}
ConnectionReuseStrategy接口定义了keepAlive方法，用于判断该connection是否保持连接继续复用，还是直接关闭
DefaultClientConnectionReuseStrategy
org/apache/http/impl/client/DefaultClientConnectionReuseStrategy.java
public class DefaultClientConnectionReuseStrategy extends DefaultConnectionReuseStrategy {
public static final DefaultClientConnectionReuseStrategy INSTANCE = new DefaultClientConnectionReuseStrategy();
@Override
public boolean keepAlive(final HttpResponse response, final HttpContext context) {
final HttpRequest request = (HttpRequest) context.getAttribute(HttpCoreContext.HTTP_REQUEST);
if (request != null) {
final Header[] connHeaders = request.getHeaders(HttpHeaders.CONNECTION);
if (connHeaders.length != 0) {
final TokenIterator ti = new BasicTokenIterator(new BasicHeaderIterator(connHeaders, null));
while (ti.hasNext()) {
final String token = ti.nextToken();
if (HTTP.CONN_CLOSE.equalsIgnoreCase(token)) {
return false;
}
}
}
}
return super.keepAlive(response, context);
}
}
DefaultClientConnectionReuseStrategy继承了DefaultConnectionReuseStrategy，其keepAlive方法先判断Connection这个header的值是不是Close，若是则返回false，其他逻辑复用父类的方法
MainClientExec
org/apache/http/impl/execchain/MainClientExec.java
@Override
public CloseableHttpResponse execute(
final HttpRoute route,
final HttpRequestWrapper request,
final HttpClientContext context,
final HttpExecutionAware execAware) throws IOException, HttpException {
//......
response = requestExecutor.execute(request, managedConn, context);
// The connection is in or can be brought to a re-usable state.
if (reuseStrategy.keepAlive(response, context)) {
// Set the idle duration of this connection
final long duration = keepAliveStrategy.getKeepAliveDuration(response, context);
if (this.log.isDebugEnabled()) {
final String s;
if (duration > 0) {
s = "for " + duration + " " + TimeUnit.MILLISECONDS;
} else {
s = "indefinitely";
}
this.log.debug("Connection can be kept alive " + s);
}
connHolder.setValidFor(duration, TimeUnit.MILLISECONDS);
connHolder.markReusable();
} else {
connHolder.markNonReusable();
}
//......
}
MainClientExec的execute方法会通过reuseStrategy.keepAlive判断连接是否可以复用，是的话则通过keepAliveStrategy.getKeepAliveDuration来获取keepAlive时间，同时设置setValidFor(
keepalive)及markReusable
releaseConnection
org/apache/http/impl/conn/PoolingHttpClientConnectionManager.java
@Override
public void releaseConnection(
final HttpClientConnection managedConn,
final Object state,
final long keepalive, final TimeUnit timeUnit) {
Args.notNull(managedConn, "Managed connection");
synchronized (managedConn) {
final CPoolEntry entry = CPoolProxy.detach(managedConn);
if (entry == null) {
return;
}
final ManagedHttpClientConnection conn = entry.getConnection();
try {
if (conn.isOpen()) {
final TimeUnit effectiveUnit = timeUnit != null ? timeUnit : TimeUnit.MILLISECONDS;
entry.setState(state);
entry.updateExpiry(keepalive, effectiveUnit);
if (this.log.isDebugEnabled()) {
final String s;
if (keepalive > 0) {
s = "for " + (double) effectiveUnit.toMillis(keepalive) / 1000 + " seconds";
} else {
s = "indefinitely";
}
this.log.debug("Connection " + format(entry) + " can be kept alive " + s);
}
conn.setSocketTimeout(0);
}
} finally {
this.pool.release(entry, conn.isOpen() && entry.isRouteComplete());
if (this.log.isDebugEnabled()) {
this.log.debug("Connection released: " + format(entry) + formatStats(entry.getRoute()));
}
}
}
}
PoolingHttpClientConnectionManager的releaseConnection方法在连接是open的时候执行entry.updateExpiry(keepalive, effectiveUnit)
PoolEntry
org/apache/http/pool/PoolEntry.java
public synchronized void updateExpiry(final long time, final TimeUnit timeUnit) {
Args.notNull(timeUnit, "Time unit");
this.updated = System.currentTimeMillis();
final long newExpiry;
if (time > 0) {
newExpiry = this.updated + timeUnit.toMillis(time);
} else {
newExpiry = Long.MAX_VALUE;
}
this.expiry = Math.min(newExpiry, this.validityDeadline);
}
public synchronized boolean isExpired(final long now) {
return now >= this.expiry;
}
它在keepalive大于0的时候更新newExpiry为当前时间+keepalive时间，否则更新newExpiry为Long.MAX_VALUE，最后取newExpiry与validityDeadline的最小值作为entry的expiry；其isExpired方法用当前时间与expiry对比，大于等于的返回true
closeExpired
org/apache/http/pool/AbstractConnPool.java
/**
* Closes expired connections and evicts them from the pool.
*/
public void closeExpired() {
final long now = System.currentTimeMillis();
enumAvailable(new PoolEntryCallback() {
@Override
public void process(final PoolEntry entry) {
if (entry.isExpired(now)) {
entry.close();
}
}
});
}
closeExpired主要是遍历available，挨个判断是否expired，是则执行close
小结
HttpClient的MainClientExec的execute方法会通过reuseStrategy.keepAlive判断连接是否可以复用，是的话则通过keepAliveStrategy.getKeepAliveDuration来获取keepAlive时间，同时设置setValidFor(keepalive)及markReusable；IdleConnectionEvictor线程每隔指定时间会执行closeExpired方法，它是依据当前时间与entry的expiry时间进行比较得出，而expiry时间则取newExpiry与validityDeadline的最小值，其中newExpiry的时间取决于keepAliveStrategy.getKeepAliveDuration，而validityDeadline取决于connTimeToLive值。若connTimeToLive值没有设置则默认为-1，那么validityDeadline的值是Long.MAX_VALUE，那么isExpired方法则取决于keepAliveStrategy.getKeepAliveDuration返回的值。若response的header没有Keep-Alive或者Keep-Alive的header没有timeout参数则keepAliveStrategy.getKeepAliveDuration返回-1(indefinitely)，则newExpiry将是Long.MAX_VALUE。
默认keepalive是开启的，如果走systemProperties，且http.keepAlive设置为false，则ConnectionReuseStrategy会被设置为NoConnectionReuseStrategy(
keepAlive方法返回false)，连接归还的时候会被直接关闭。
doc
- Keep-Alive