-
本文概述
透过源码分析HttpClient的原理。
-
使用回顾
public static void sendRequestWithHttpClient() { List
formparams = new ArrayList (); formparams.add(new BasicNameValuePair("account", "")); formparams.add(new BasicNameValuePair("password", "")); HttpEntity reqEntity = null; try { reqEntity = new UrlEncodedFormEntity(formparams, "utf-8"); } catch (UnsupportedEncodingException e) { e.printStackTrace(); } RequestConfig requestConfig = RequestConfig.custom() .setConnectTimeout(5000)//一、连接超时:connectionTimeout-->指的是连接一个url的连接等待时间 .setSocketTimeout(5000)// 二、读取数据超时:SocketTimeout-->指的是连接上一个url,获取response的返回等待时间 .setConnectionRequestTimeout(5000) .build(); HttpClient client = HttpClientBuilder.create().build(); HttpPost post = new HttpPost("http://xxx.xx.xx/login"); post.setEntity(reqEntity); post.setConfig(requestConfig); HttpResponse response = null; try { response = client.execute(post); } catch (IOException e) { e.printStackTrace(); } if (response.getStatusLine().getStatusCode() == 200) { HttpEntity resEntity = response.getEntity(); String message = null; try { message = EntityUtils.toString(resEntity, "utf-8"); } catch (IOException e) { e.printStackTrace(); } System.out.println(message); } else { System.out.println("请求失败"); } } -
源码分析
NameValuePair是一个存储键/值的接口:
public interface NameValuePair { /** * Gets the name of this pair. * * @return the name of this pair, never {@code null}. */ String getName(); /** * Gets the value of this pair. * * @return the value of this pair, may be {@code null}. */ String getValue();
然后构造一个UrlEncodedFormEntity对象,传入键值信息和编码:
public UrlEncodedFormEntity ( final List extends NameValuePair> parameters, final String charset) throws UnsupportedEncodingException { super(URLEncodedUtils.format(parameters, charset != null ? charset : HTTP.DEF_CONTENT_CHARSET.name()), ContentType.create(URLEncodedUtils.CONTENT_TYPE, charset)); }
super构造方法里,URLEncodedUtils.format将键值信息通过charset转换成编码后的信息字符串:
public static String format( final List extends NameValuePair> parameters, final String charset) { return format(parameters, QP_SEP_A, charset); }
QP_SEP_A是“&”
public static String format( final List extends NameValuePair> parameters, final char parameterSeparator, final String charset) { final StringBuilder result = new StringBuilder(); for (final NameValuePair parameter : parameters) { final String encodedName = encodeFormFields(parameter.getName(), charset); final String encodedValue = encodeFormFields(parameter.getValue(), charset); if (result.length() > 0) { result.append(parameterSeparator); } result.append(encodedName); if (encodedValue != null) { result.append(NAME_VALUE_SEPARATOR); result.append(encodedValue); } } return result.toString(); }
format里把每一对键值都按照传入的charset进行编码后存入result,每一对键值之间用传入的“&”分割,键和值之间用NAME_VALUE_SEPARATOE(“=”)连接。
再来看super的第二个参数ContentType.create(URLEncodedUtils.CONTENT_TYPE, charset),URLEncodedUtils.CONTENT_TYPE的值是application/x-www-form-urlencoded:
public static ContentType create( final String mimeType, final String charset) throws UnsupportedCharsetException { return create(mimeType, !TextUtils.isBlank(charset) ? Charset.forName(charset) : null); }
Charset.forName会把编码格式的字符串名字转换成Charset对象:
public static Charset forName(String charsetName) { Charset cs = lookup(charsetName); if (cs != null) return cs; throw new UnsupportedCharsetException(charsetName); }
private static Charset lookup(String charsetName) { if (charsetName == null) throw new IllegalArgumentException("Null charset name"); //首先判断上一次使用的Charset对象是否还在缓存中 final Map.Entry
cached = cache1; if (cached != null && charsetName.equals(cached.getKey())) return cached.getValue(); //如果没有再从最近使用的一些Charset里面去找 return lookup2(charsetName); } private static Charset lookup2(String charsetName) { Charset cs; synchronized (cache2) { if ((cs = cache2.get(charsetName)) != null) { //如果cache2中有了则直接返回这个对象,并把cache1更新为最新使用的这个Charset对象 cache1 = new AbstractMap.SimpleImmutableEntry<>(charsetName, cs); return cs; } } // Android-changed: Drop support for "standard" and "extended" // providers. //这里通过ServiceLoader加载CharsetProvider,再通过CharsetProvider去加载Jvm中的属性获取Charset对象 if ((cs = NativeConverter.charsetForName(charsetName)) != null || (cs = lookupViaProviders(charsetName)) != null) { //保存到cache1和cache2中 cache(charsetName, cs); return cs; } /* Only need to check the name if we didn't find a charset for it */ checkName(charsetName); return null; } 关于cache1和cache2:
/* The standard set of charsets */ // Android-removed: We use ICU's list of standard charsets. // private static CharsetProvider standardProvider = new StandardCharsets(); // Cache of the most-recently-returned charsets, // along with the names that were used to find them // // cache1/2 usage is explained in the lookup method // private static volatile Map.Entry
cache1 = null; // "Level 1" cache private static final HashMap cache2 = new HashMap<>(); // "Level 2" cache private static void cache(String charsetName, Charset cs) { synchronized(cache2) { String canonicalName = cs.name(); Charset canonicalCharset = cache2.get(canonicalName); if (canonicalCharset != null) { cs = canonicalCharset; } else { cache2.put(canonicalName, cs); for (String alias : cs.aliases()) { cache2.put(alias, cs); } } cache2.put(charsetName, cs); } cache1 = new AbstractMap.SimpleImmutableEntry<>(charsetName, cs); } 回到create方法,最终返回一个ContentType对象。
再进到super里面,是StringEntity的构造方法:
public StringEntity(final String string, final ContentType contentType) throws UnsupportedCharsetException { super(); Args.notNull(string, "Source string"); Charset charset = contentType != null ? contentType.getCharset() : null; if (charset == null) { charset = HTTP.DEF_CONTENT_CHARSET; } this.content = string.getBytes(charset); if (contentType != null) { setContentType(contentType.toString()); } }
如果之前设置的编码不存在则统一按照HTTP.DEF_CONTENT_CHARSET(“ISO-8859-1”)来设置,string就是之前的“key1=value1&key2=value2...”,把它进行编码后存到content中,然后设置ContentType,contentType.toString:
@Override public String toString() { final CharArrayBuffer buf = new CharArrayBuffer(64); buf.append(this.mimeType); //这里的params是null,如果设置这个,可以把charset设置在里面,因为它和charset设置是if-else关系 if (this.params != null) { buf.append("; "); BasicHeaderValueFormatter.INSTANCE.formatParameters(buf, this.params, false); } else if (this.charset != null) { buf.append("; charset="); buf.append(this.charset.name()); } return buf.toString(); }
最终contentType.toString方法会把这些信息设置成Header存进contentType(诸如“application/x-www-form-urlencoded;charset=utf-8”的内容)。
回到HttpClient使用,接下来是构造HttpClient客户端,然后构建请求(这里是POST请求(HttpPost)),给请求设置RequestConfig,参数不深究,直接看client.execute,这里的client是build返回的InternalHttpClient,发现execute是其直接父类CloseableHttpClient的方法,并通过调用重载方法执行doExecute:
@Override public CloseableHttpResponse execute( final HttpUriRequest request, final HttpContext context) throws IOException, ClientProtocolException { Args.notNull(request, "HTTP request"); return doExecute(determineTarget(request), request, context); }
doExecute由InternalHttpClient覆写:
@Override protected CloseableHttpResponse doExecute( final HttpHost target, final HttpRequest request, final HttpContext context) throws IOException, ClientProtocolException { Args.notNull(request, "HTTP request"); HttpExecutionAware execAware = null; if (request instanceof HttpExecutionAware) { execAware = (HttpExecutionAware) request; } try { final HttpRequestWrapper wrapper = HttpRequestWrapper.wrap(request, target); final HttpClientContext localcontext = HttpClientContext.adapt( context != null ? context : new BasicHttpContext()); RequestConfig config = null; if (request instanceof Configurable) { config = ((Configurable) request).getConfig(); } if (config == null) { final HttpParams params = request.getParams(); if (params instanceof HttpParamsNames) { if (!((HttpParamsNames) params).getNames().isEmpty()) { config = HttpClientParamConfig.getRequestConfig(params, this.defaultConfig); } } else { config = HttpClientParamConfig.getRequestConfig(params, this.defaultConfig); } } if (config != null) { localcontext.setRequestConfig(config); } setupContext(localcontext); final HttpRoute route = determineRoute(target, wrapper, localcontext); return this.execChain.execute(route, wrapper, localcontext, execAware); } catch (final HttpException httpException) { throw new ClientProtocolException(httpException); } }
可以看到最后执行了execChain.execute方法,execChain是构造的时候传入的,所以最终顺着找到了MainClientExec:
ClientExecChain execChain = createMainExec( requestExecCopy, connManagerCopy, reuseStrategyCopy, keepAliveStrategyCopy, new ImmutableHttpProcessor(new RequestTargetHost(), new RequestUserAgent(userAgentCopy)), targetAuthStrategyCopy, proxyAuthStrategyCopy, userTokenHandlerCopy); ############# protected ClientExecChain createMainExec( final HttpRequestExecutor requestExec, final HttpClientConnectionManager connManager, final ConnectionReuseStrategy reuseStrategy, final ConnectionKeepAliveStrategy keepAliveStrategy, final HttpProcessor proxyHttpProcessor, final AuthenticationStrategy targetAuthStrategy, final AuthenticationStrategy proxyAuthStrategy, final UserTokenHandler userTokenHandler) { return new MainClientExec( requestExec, connManager, reuseStrategy, keepAliveStrategy, proxyHttpProcessor, targetAuthStrategy, proxyAuthStrategy, userTokenHandler); }
它的execute方法如下:
@Override public CloseableHttpResponse execute( final HttpRoute route, final HttpRequestWrapper request, final HttpClientContext context, final HttpExecutionAware execAware) throws IOException, HttpException { Args.notNull(route, "HTTP route"); Args.notNull(request, "HTTP request"); Args.notNull(context, "HTTP context"); AuthState targetAuthState = context.getTargetAuthState(); if (targetAuthState == null) { targetAuthState = new AuthState(); context.setAttribute(HttpClientContext.TARGET_AUTH_STATE, targetAuthState); } AuthState proxyAuthState = context.getProxyAuthState(); if (proxyAuthState == null) { proxyAuthState = new AuthState(); context.setAttribute(HttpClientContext.PROXY_AUTH_STATE, proxyAuthState); } if (request instanceof HttpEntityEnclosingRequest) { RequestEntityProxy.enhance((HttpEntityEnclosingRequest) request); } Object userToken = context.getUserToken(); final ConnectionRequest connRequest = connManager.requestConnection(route, userToken); if (execAware != null) { if (execAware.isAborted()) { connRequest.cancel(); throw new RequestAbortedException("Request aborted"); } execAware.setCancellable(connRequest); } final RequestConfig config = context.getRequestConfig(); final HttpClientConnection managedConn; try { final int timeout = config.getConnectionRequestTimeout(); managedConn = connRequest.get(timeout > 0 ? timeout : 0, TimeUnit.MILLISECONDS); } catch(final InterruptedException interrupted) { Thread.currentThread().interrupt(); throw new RequestAbortedException("Request aborted", interrupted); } catch(final ExecutionException ex) { Throwable cause = ex.getCause(); if (cause == null) { cause = ex; } throw new RequestAbortedException("Request execution failed", cause); } context.setAttribute(HttpCoreContext.HTTP_CONNECTION, managedConn); if (config.isStaleConnectionCheckEnabled()) { // validate connection if (managedConn.isOpen()) { this.log.debug("Stale connection check"); if (managedConn.isStale()) { this.log.debug("Stale connection detected"); managedConn.close(); } } } final ConnectionHolder connHolder = new ConnectionHolder(this.log, this.connManager, managedConn); try { if (execAware != null) { execAware.setCancellable(connHolder); } HttpResponse response; for (int execCount = 1;; execCount++) { if (execCount > 1 && !RequestEntityProxy.isRepeatable(request)) { throw new NonRepeatableRequestException("Cannot retry request " + "with a non-repeatable request entity."); } if (execAware != null && execAware.isAborted()) { throw new RequestAbortedException("Request aborted"); } if (!managedConn.isOpen()) { this.log.debug("Opening connection " + route); try { establishRoute(proxyAuthState, managedConn, route, request, context); } catch (final TunnelRefusedException ex) { if (this.log.isDebugEnabled()) { this.log.debug(ex.getMessage()); } response = ex.getResponse(); break; } } final int timeout = config.getSocketTimeout(); if (timeout >= 0) { managedConn.setSocketTimeout(timeout); } if (execAware != null && execAware.isAborted()) { throw new RequestAbortedException("Request aborted"); } if (this.log.isDebugEnabled()) { this.log.debug("Executing request " + request.getRequestLine()); } if (!request.containsHeader(AUTH.WWW_AUTH_RESP)) { if (this.log.isDebugEnabled()) { this.log.debug("Target auth state: " + targetAuthState.getState()); } this.authenticator.generateAuthResponse(request, targetAuthState, context); } if (!request.containsHeader(AUTH.PROXY_AUTH_RESP) && !route.isTunnelled()) { if (this.log.isDebugEnabled()) { this.log.debug("Proxy auth state: " + proxyAuthState.getState()); } this.authenticator.generateAuthResponse(request, proxyAuthState, context); } context.setAttribute(HttpCoreContext.HTTP_REQUEST, request); response = requestExecutor.execute(request, managedConn, context); // The connection is in or can be brought to a re-usable state. if (reuseStrategy.keepAlive(response, context)) { // Set the idle duration of this connection final long duration = keepAliveStrategy.getKeepAliveDuration(response, context); if (this.log.isDebugEnabled()) { final String s; if (duration > 0) { s = "for " + duration + " " + TimeUnit.MILLISECONDS; } else { s = "indefinitely"; } this.log.debug("Connection can be kept alive " + s); } connHolder.setValidFor(duration, TimeUnit.MILLISECONDS); connHolder.markReusable(); } else { connHolder.markNonReusable(); } if (needAuthentication( targetAuthState, proxyAuthState, route, response, context)) { // Make sure the response body is fully consumed, if present final HttpEntity entity = response.getEntity(); if (connHolder.isReusable()) { EntityUtils.consume(entity); } else { managedConn.close(); if (proxyAuthState.getState() == AuthProtocolState.SUCCESS && proxyAuthState.isConnectionBased()) { this.log.debug("Resetting proxy auth state"); proxyAuthState.reset(); } if (targetAuthState.getState() == AuthProtocolState.SUCCESS && targetAuthState.isConnectionBased()) { this.log.debug("Resetting target auth state"); targetAuthState.reset(); } } // discard previous auth headers final HttpRequest original = request.getOriginal(); if (!original.containsHeader(AUTH.WWW_AUTH_RESP)) { request.removeHeaders(AUTH.WWW_AUTH_RESP); } if (!original.containsHeader(AUTH.PROXY_AUTH_RESP)) { request.removeHeaders(AUTH.PROXY_AUTH_RESP); } } else { break; } } if (userToken == null) { userToken = userTokenHandler.getUserToken(context); context.setAttribute(HttpClientContext.USER_TOKEN, userToken); } if (userToken != null) { connHolder.setState(userToken); } // check for entity, release connection if possible final HttpEntity entity = response.getEntity(); if (entity == null || !entity.isStreaming()) { // connection not needed and (assumed to be) in re-usable state connHolder.releaseConnection(); return new HttpResponseProxy(response, null); } return new HttpResponseProxy(response, connHolder); } catch (final ConnectionShutdownException ex) { final InterruptedIOException ioex = new InterruptedIOException( "Connection has been shut down"); ioex.initCause(ex); throw ioex; } catch (final HttpException ex) { connHolder.abortConnection(); throw ex; } catch (final IOException ex) { connHolder.abortConnection(); if (proxyAuthState.isConnectionBased()) { proxyAuthState.reset(); } if (targetAuthState.isConnectionBased()) { targetAuthState.reset(); } throw ex; } catch (final RuntimeException ex) { connHolder.abortConnection(); if (proxyAuthState.isConnectionBased()) { proxyAuthState.reset(); } if (targetAuthState.isConnectionBased()) { targetAuthState.reset(); } throw ex; } catch (final Error error) { connManager.shutdown(); throw error; } }
requestExecutor.execute(request, managedConn, context)得到了一个response,所以这句代码就是请求数据的地方,request是前面的HttpPost,managedConn是connRequest.get(timeout > 0 ? timeout : 0, TimeUnit.MILLISECONDS)得到的,connRequest是connManager.requestConnection(route, userToken)得到的,connManager是前面构造MainClientExec时传入的PoolingHttpClientConnectionManager,它的requestConnection方法是:
@Override public ConnectionRequest requestConnection( final HttpRoute route, final Object state) { Args.notNull(route, "HTTP route"); if (this.log.isDebugEnabled()) { this.log.debug("Connection request: " + format(route, state) + formatStats(route)); } Asserts.check(!this.isShutDown.get(), "Connection pool shut down"); final Future
future = this.pool.lease(route, state, null); return new ConnectionRequest() { @Override public boolean cancel() { return future.cancel(true); } @Override public HttpClientConnection get( final long timeout, final TimeUnit timeUnit) throws InterruptedException, ExecutionException, ConnectionPoolTimeoutException { final HttpClientConnection conn = leaseConnection(future, timeout, timeUnit); if (conn.isOpen()) { final HttpHost host; if (route.getProxyHost() != null) { host = route.getProxyHost(); } else { host = route.getTargetHost(); } final SocketConfig socketConfig = resolveSocketConfig(host); conn.setSocketTimeout(socketConfig.getSoTimeout()); } return conn; } }; } 返回的匿名对象ConnectionRequest调用get方法就得到了managedConn,可以看到conn是通过leaseConnection方法得到的:
protected HttpClientConnection leaseConnection( final Future
future, final long timeout, final TimeUnit timeUnit) throws InterruptedException, ExecutionException, ConnectionPoolTimeoutException { final CPoolEntry entry; try { entry = future.get(timeout, timeUnit); if (entry == null || future.isCancelled()) { throw new ExecutionException(new CancellationException("Operation cancelled")); } Asserts.check(entry.getConnection() != null, "Pool entry with no connection"); if (this.log.isDebugEnabled()) { this.log.debug("Connection leased: " + format(entry) + formatStats(entry.getRoute())); } return CPoolProxy.newProxy(entry); } catch (final TimeoutException ex) { throw new ConnectionPoolTimeoutException("Timeout waiting for connection from pool"); } } 所以conn是一个CPoolProxy对象,也就是managedConn。requestExecutor也是前面构造时传入的HttpRequestExecutor,它的execute方法如下:
public HttpResponse execute( final HttpRequest request, final HttpClientConnection conn, final HttpContext context) throws IOException, HttpException { Args.notNull(request, "HTTP request"); Args.notNull(conn, "Client connection"); Args.notNull(context, "HTTP context"); try { HttpResponse response = doSendRequest(request, conn, context); if (response == null) { response = doReceiveResponse(request, conn, context); } return response; } catch (final IOException ex) { closeConnection(conn); throw ex; } catch (final HttpException ex) { closeConnection(conn); throw ex; } catch (final RuntimeException ex) { closeConnection(conn); throw ex; } }
看到了熟悉的doSendRequest方法:
protected HttpResponse doSendRequest( final HttpRequest request, final HttpClientConnection conn, final HttpContext context) throws IOException, HttpException { Args.notNull(request, "HTTP request"); Args.notNull(conn, "Client connection"); Args.notNull(context, "HTTP context"); HttpResponse response = null; context.setAttribute(HttpCoreContext.HTTP_CONNECTION, conn); context.setAttribute(HttpCoreContext.HTTP_REQ_SENT, Boolean.FALSE); conn.sendRequestHeader(request); if (request instanceof HttpEntityEnclosingRequest) { // Check for expect-continue handshake. We have to flush the // headers and wait for an 100-continue response to handle it. // If we get a different response, we must not send the entity. boolean sendentity = true; final ProtocolVersion ver = request.getRequestLine().getProtocolVersion(); if (((HttpEntityEnclosingRequest) request).expectContinue() && !ver.lessEquals(HttpVersion.HTTP_1_0)) { conn.flush(); // As suggested by RFC 2616 section 8.2.3, we don't wait for a // 100-continue response forever. On timeout, send the entity. if (conn.isResponseAvailable(this.waitForContinue)) { response = conn.receiveResponseHeader(); if (canResponseHaveBody(request, response)) { conn.receiveResponseEntity(response); } final int status = response.getStatusLine().getStatusCode(); if (status < 200) { if (status != HttpStatus.SC_CONTINUE) { throw new ProtocolException( "Unexpected response: " + response.getStatusLine()); } // discard 100-continue response = null; } else { sendentity = false; } } } if (sendentity) { conn.sendRequestEntity((HttpEntityEnclosingRequest) request); } } conn.flush(); context.setAttribute(HttpCoreContext.HTTP_REQ_SENT, Boolean.TRUE); return response; }
发现CPoolProxy里面的方法都是通过他持有的对象poolEntry去实现的,所以它叫做Proxy,那肯定是有一个CPool的对象,继续往下找。poolEntry是leaseConnection里的CPoolProxy.newProxy(entry)传入的,entry通过future.get(timeout, timeUnit)创建,future又通过requestConnection里的this.pool.lease(route, state, null)创建,this.pool是构造时创建的:
this.pool = new CPool(new InternalConnectionFactory(this.configData, connFactory), 2, 20, timeToLive, timeUnit);
它里面的ConnFactory是InternalConnectionFactory(this.configData, connFactory),CPool调用lease就得到了future:
@Override public Future
lease(final T route, final Object state, final FutureCallback callback) { Args.notNull(route, "Route"); Asserts.check(!this.isShutDown, "Connection pool shut down"); return new Future () { private final AtomicBoolean cancelled = new AtomicBoolean(false); private final AtomicBoolean done = new AtomicBoolean(false); private final AtomicReference entryRef = new AtomicReference (null); @Override public boolean cancel(final boolean mayInterruptIfRunning) { if (done.compareAndSet(false, true)) { cancelled.set(true); lock.lock(); try { condition.signalAll(); } finally { lock.unlock(); } if (callback != null) { callback.cancelled(); } return true; } return false; } @Override public boolean isCancelled() { return cancelled.get(); } @Override public boolean isDone() { return done.get(); } @Override public E get() throws InterruptedException, ExecutionException { try { return get(0L, TimeUnit.MILLISECONDS); } catch (final TimeoutException ex) { throw new ExecutionException(ex); } } @Override public E get(final long timeout, final TimeUnit timeUnit) throws InterruptedException, ExecutionException, TimeoutException { for (;;) { synchronized (this) { try { final E entry = entryRef.get(); if (entry != null) { return entry; } if (done.get()) { throw new ExecutionException(operationAborted()); } final E leasedEntry = getPoolEntryBlocking(route, state, timeout, timeUnit, this); if (validateAfterInactivity > 0) { if (leasedEntry.getUpdated() + validateAfterInactivity <= System.currentTimeMillis()) { if (!validate(leasedEntry)) { leasedEntry.close(); release(leasedEntry, false); continue; } } } if (done.compareAndSet(false, true)) { entryRef.set(leasedEntry); done.set(true); onLease(leasedEntry); if (callback != null) { callback.completed(leasedEntry); } return leasedEntry; } else { release(leasedEntry, true); throw new ExecutionException(operationAborted()); } } catch (final IOException ex) { if (done.compareAndSet(false, true)) { if (callback != null) { callback.failed(ex); } } throw new ExecutionException(ex); } } } } }; } get方法里通过getPoolEntryBlocking()来得到CPoolEntry(Future
泛型决定是CPoolEntry): private E getPoolEntryBlocking( final T route, final Object state, final long timeout, final TimeUnit timeUnit, final Future
future) throws IOException, InterruptedException, ExecutionException, TimeoutException { Date deadline = null; if (timeout > 0) { deadline = new Date (System.currentTimeMillis() + timeUnit.toMillis(timeout)); } this.lock.lock(); try { final RouteSpecificPool pool = getPool(route); E entry; for (;;) { Asserts.check(!this.isShutDown, "Connection pool shut down"); if (future.isCancelled()) { throw new ExecutionException(operationAborted()); } for (;;) { entry = pool.getFree(state); if (entry == null) { break; } if (entry.isExpired(System.currentTimeMillis())) { entry.close(); } if (entry.isClosed()) { this.available.remove(entry); pool.free(entry, false); } else { break; } } if (entry != null) { this.available.remove(entry); this.leased.add(entry); onReuse(entry); return entry; } // New connection is needed final int maxPerRoute = getMax(route); // Shrink the pool prior to allocating a new connection final int excess = Math.max(0, pool.getAllocatedCount() + 1 - maxPerRoute); if (excess > 0) { for (int i = 0; i < excess; i++) { final E lastUsed = pool.getLastUsed(); if (lastUsed == null) { break; } lastUsed.close(); this.available.remove(lastUsed); pool.remove(lastUsed); } } if (pool.getAllocatedCount() < maxPerRoute) { final int totalUsed = this.leased.size(); final int freeCapacity = Math.max(this.maxTotal - totalUsed, 0); if (freeCapacity > 0) { final int totalAvailable = this.available.size(); if (totalAvailable > freeCapacity - 1) { if (!this.available.isEmpty()) { final E lastUsed = this.available.removeLast(); lastUsed.close(); final RouteSpecificPool otherpool = getPool(lastUsed.getRoute()); otherpool.remove(lastUsed); } } final C conn = this.connFactory.create(route); entry = pool.add(conn); this.leased.add(entry); return entry; } } boolean success = false; try { pool.queue(future); this.pending.add(future); if (deadline != null) { success = this.condition.awaitUntil(deadline); } else { this.condition.await(); success = true; } if (future.isCancelled()) { throw new ExecutionException(operationAborted()); } } finally { // In case of 'success', we were woken up by the // connection pool and should now have a connection // waiting for us, or else we're shutting down. // Just continue in the loop, both cases are checked. pool.unqueue(future); this.pending.remove(future); } // check for spurious wakeup vs. timeout if (!success && (deadline != null && deadline.getTime() <= System.currentTimeMillis())) { break; } } throw new TimeoutException("Timeout waiting for connection"); } finally { this.lock.unlock(); } } 首先要通过getPool拿到pool:
private RouteSpecificPool
getPool(final T route) { RouteSpecificPool pool = this.routeToPool.get(route); if (pool == null) { pool = new RouteSpecificPool (route) { @Override protected E createEntry(final C conn) { return AbstractConnPool.this.createEntry(route, conn); } }; this.routeToPool.put(route, pool); } return pool; } 然后通过pool.getFree(state)得到entry,RouteSpecificPool中:
public E getFree(final Object state) { if (!this.available.isEmpty()) { if (state != null) { final Iterator
it = this.available.iterator(); while (it.hasNext()) { final E entry = it.next(); if (state.equals(entry.getState())) { it.remove(); this.leased.add(entry); return entry; } } } final Iterator it = this.available.iterator(); while (it.hasNext()) { final E entry = it.next(); if (entry.getState() == null) { it.remove(); this.leased.add(entry); return entry; } } } return null; } 这里的available暂时是空的,所以getPoolEntryBlocking会走到;
if (pool.getAllocatedCount() < maxPerRoute) { final int totalUsed = this.leased.size(); final int freeCapacity = Math.max(this.maxTotal - totalUsed, 0); if (freeCapacity > 0) { final int totalAvailable = this.available.size(); if (totalAvailable > freeCapacity - 1) { if (!this.available.isEmpty()) { final E lastUsed = this.available.removeLast(); lastUsed.close(); final RouteSpecificPool
otherpool = getPool(lastUsed.getRoute()); otherpool.remove(lastUsed); } } final C conn = this.connFactory.create(route); entry = pool.add(conn); this.leased.add(entry); return entry; } } entry通过pool.add方法创建了:
//RouteSpecificPool里的add方法 public E add(final C conn) { final E entry = createEntry(conn); this.leased.add(entry); return entry; } ...... //add方法调用的createEntry就是匿名对象(也就是entry对象)的createEntry pool = new RouteSpecificPool
(route) { @Override protected E createEntry(final C conn) { return AbstractConnPool.this.createEntry(route, conn); } }; //AbstractConnPool.this也就是当前上下文对象CPool,所以这里的AbstractConnPool.this.createEntry就是CPool里面覆写的 @Override protected CPoolEntry createEntry(final HttpRoute route, final ManagedHttpClientConnection conn) { final String id = Long.toString(COUNTER.getAndIncrement()); return new CPoolEntry(this.log, id, route, conn, this.timeToLive, this.timeUnit); } 终于找到了CPoolEntry对象。
再回到doSendRequest,conn.flush方法调用过程如下:
@Override public void flush() throws IOException { getValidConnection().flush(); } ..... ManagedHttpClientConnection getValidConnection() { final ManagedHttpClientConnection conn = getConnection(); if (conn == null) { throw new ConnectionShutdownException(); } return conn; } ...... ManagedHttpClientConnection getConnection() { final CPoolEntry local = this.poolEntry; if (local == null) { return null; } return local.getConnection(); }
那这里获取到的local就是上面的new CPoolEntry(this.log, id, route, conn, this.timeToLive, this.timeUnit),最后是通过它的conn对象来执行flush的,一步步往上找,最终找到conn是通过this.connFactory.create(route)创建的,通过查找,connFactory就是前面构造CPool的时候传入的InternalConnectionFactory,看一下它的create方法:
@Override public ManagedHttpClientConnection create(final HttpRoute route) throws IOException { ConnectionConfig config = null; if (route.getProxyHost() != null) { config = this.configData.getConnectionConfig(route.getProxyHost()); } if (config == null) { config = this.configData.getConnectionConfig(route.getTargetHost()); } if (config == null) { config = this.configData.getDefaultConnectionConfig(); } if (config == null) { config = ConnectionConfig.DEFAULT; } return this.connFactory.create(route, config); }
里面还有个connFactory,发现是构造的时候设置的,构造的时候传入的是null,所以根据:
InternalConnectionFactory( final ConfigData configData, final HttpConnectionFactory
connFactory) { super(); this.configData = configData != null ? configData : new ConfigData(); this.connFactory = connFactory != null ? connFactory : ManagedHttpClientConnectionFactory.INSTANCE; } 可知这里的connFactory就是ManagedHttpClientConnectionFactory.INSTANCE,它的create方法如下:
@Override public ManagedHttpClientConnection create(final HttpRoute route, final ConnectionConfig config) { final ConnectionConfig cconfig = config != null ? config : ConnectionConfig.DEFAULT; CharsetDecoder charDecoder = null; CharsetEncoder charEncoder = null; final Charset charset = cconfig.getCharset(); final CodingErrorAction malformedInputAction = cconfig.getMalformedInputAction() != null ? cconfig.getMalformedInputAction() : CodingErrorAction.REPORT; final CodingErrorAction unmappableInputAction = cconfig.getUnmappableInputAction() != null ? cconfig.getUnmappableInputAction() : CodingErrorAction.REPORT; if (charset != null) { charDecoder = charset.newDecoder(); charDecoder.onMalformedInput(malformedInputAction); charDecoder.onUnmappableCharacter(unmappableInputAction); charEncoder = charset.newEncoder(); charEncoder.onMalformedInput(malformedInputAction); charEncoder.onUnmappableCharacter(unmappableInputAction); } final String id = "http-outgoing-" + Long.toString(COUNTER.getAndIncrement()); return new LoggingManagedHttpClientConnection( id, log, headerLog, wireLog, cconfig.getBufferSize(), cconfig.getFragmentSizeHint(), charDecoder, charEncoder, cconfig.getMessageConstraints(), incomingContentStrategy, outgoingContentStrategy, requestWriterFactory, responseParserFactory); }
所以我们找一下LoggingManagedHttpClientConnection的flush方法,没找到,但在它的父类的父类DefaultBHttpClientConnection中找到了:
@Override public void flush() throws IOException { ensureOpen(); doFlush(); }
ensureOpen在其父类中:
protected void ensureOpen() throws IOException { final Socket socket = this.socketHolder.get(); if (socket == null) { throw new ConnectionClosedException(); } if (!this.inBuffer.isBound()) { this.inBuffer.bind(getSocketInputStream(socket)); } if (!this.outbuffer.isBound()) { this.outbuffer.bind(getSocketOutputStream(socket)); } } ...... protected InputStream getSocketInputStream(final Socket socket) throws IOException { return socket.getInputStream(); } protected OutputStream getSocketOutputStream(final Socket socket) throws IOException { return socket.getOutputStream(); }
到这里就调用到了socket层的API了,ensureOpen就相当于前面文章说的HttpURLConnection里面的connect。
还有flush:
protected void doFlush() throws IOException { this.outbuffer.flush(); } ...... //SessionOutputBufferImpl中 @Override public void flush() throws IOException { //flushBuffer是把缓冲区的内容添加到stream flushBuffer(); //flushStream是把stream中的数据通过网路发送出去 flushStream(); } ...... private void flushBuffer() throws IOException { final int len = this.buffer.length(); if (len > 0) { streamWrite(this.buffer.buffer(), 0, len); this.buffer.clear(); this.metrics.incrementBytesTransferred(len); } } private void streamWrite(final byte[] b, final int off, final int len) throws IOException { Asserts.notNull(outStream, "Output stream"); //outStream是前面ensureOpen里面通过bind方法赋值的 this.outStream.write(b, off, len); } private void flushStream() throws IOException { if (this.outStream != null) { this.outStream.flush(); } }
至此,HttpClient的请求结束了,那response是如何拿到的呢?
因为doSendRequest中response是通过receiveResponseHeader去获取的,所以以同样的方式找到了DefaultBHttpClientConnection中的receiveResponseHeader:
@Override public HttpResponse receiveResponseHeader() throws HttpException, IOException { ensureOpen(); final HttpResponse response = this.responseParser.parse(); onResponseReceived(response); if (response.getStatusLine().getStatusCode() >= HttpStatus.SC_OK) { incrementResponseCount(); } return response; }
this.responseParser.parse()最终在AbstractMessageParser中找到:
@Override public T parse() throws IOException, HttpException { final int st = this.state; switch (st) { case HEAD_LINE: try { this.message = parseHead(this.sessionBuffer); } catch (final ParseException px) { throw new ProtocolException(px.getMessage(), px); } this.state = HEADERS; //$FALL-THROUGH$ case HEADERS: final Header[] headers = AbstractMessageParser.parseHeaders( this.sessionBuffer, this.messageConstraints.getMaxHeaderCount(), this.messageConstraints.getMaxLineLength(), this.lineParser, this.headerLines); this.message.setHeaders(headers); final T result = this.message; this.message = null; this.headerLines.clear(); this.state = HEAD_LINE; return result; default: throw new IllegalStateException("Inconsistent parser state"); } }
这里只会解析header部分的数据,因为doSendRequest中还有一个receiveResponseHeader之后还会执行一个receiveResponseEntity方法(canResponseHaveBody(request, response)判断如果有body的话)用来解析body:
@Override public void receiveResponseEntity( final HttpResponse response) throws HttpException, IOException { Args.notNull(response, "HTTP response"); ensureOpen(); final HttpEntity entity = prepareInput(response); response.setEntity(entity); } ...... //BHttpConnectionBase中 protected HttpEntity prepareInput(final HttpMessage message) throws HttpException { final BasicHttpEntity entity = new BasicHttpEntity(); final long len = this.incomingContentStrategy.determineLength(message); final InputStream inStream = createInputStream(len, this.inBuffer); if (len == ContentLengthStrategy.CHUNKED) { entity.setChunked(true); entity.setContentLength(-1); entity.setContent(inStream); } else if (len == ContentLengthStrategy.IDENTITY) { entity.setChunked(false); entity.setContentLength(-1); entity.setContent(inStream); } else { entity.setChunked(false); entity.setContentLength(len); entity.setContent(inStream); } final Header contentTypeHeader = message.getFirstHeader(HTTP.CONTENT_TYPE); if (contentTypeHeader != null) { entity.setContentType(contentTypeHeader); } final Header contentEncodingHeader = message.getFirstHeader(HTTP.CONTENT_ENCODING); if (contentEncodingHeader != null) { entity.setContentEncoding(contentEncodingHeader); } return entity; }
最后返回到doSendRequest,再返回到调用client.execute的地方。至此,HttpClient的调用过程就结束了。
-
总结
HttpClient的调用过程相较于HttpURLConnection又做了一层封装,这使得使用起来更加简洁,但是分析的过程中也能感觉到HttpClient框架对此做了很多工作,通过源码分析知道了底层也是调用socket进行网络通信。