本文章基于:
org.apache.httpcomponents
httpclient
4.5.6
在项目中,我们经常使用到HttpClient,但是很少用到连接池。偶然在项目中看到一位大佬把我们疯狂创建的HttpClient给注入进去:
@Autowired
private HttpClient httpClient;
看到这个代码,我表示一脸懵逼,还能这样操作?
先看下声明这个Bean的地方:
@Bean
public HttpClient httpClient() {
return HttpClientBuilder.create()
.setConnectionManager(connectionManager())
.setDefaultRequestConfig(requestConfig())
.build();
}
好像和平时的创建的差不多啊。不对,好像多设置了几个地方,设置了ConnectionManager和DefaultRequestConfig。再看一下connectionManager()和requestConfig()方法。
@Bean
public RequestConfig requestConfig() {
RequestConfig.Builder requestConfigBuilder = RequestConfig.custom();
//指从连接池获取连接的超时时间
requestConfigBuilder.setConnectionRequestTimeout(httpClientProperties.getRequestConfig().getConnectionRequestTimeout());
//指客户端和服务器建立连接的超时时间
requestConfigBuilder.setConnectTimeout(httpClientProperties.getRequestConfig().getConnectTimeout());
//指客户端从服务器下载数据(或页面)时,两个数据包之间的最大时间间隔
requestConfigBuilder.setSocketTimeout(httpClientProperties.getRequestConfig().getSocketTimeout());
return requestConfigBuilder.build();
}
@Bean
public HttpClientConnectionManager connectionManager() {
PoolingHttpClientConnectionManager httpClientConnectionManager = new PoolingHttpClientConnectionManager();
//连接池最大容量(即全部路由的总连接数)
httpClientConnectionManager.setMaxTotal(httpClientProperties.getConnectionManager().getMaxTotal());
//每个路由的默认最大连接数
httpClientConnectionManager.setDefaultMaxPerRoute(httpClientProperties.getConnectionManager().getDefaultMaxPerRoute());
//如果对某个站点的连接数需求比较大,可以单独设置其最大连接数
//httpClientConnectionManager.setMaxPerRoute();
return httpClientConnectionManager;
}
看着好像很复杂的样子。
平时创建一个HttpClient我们一般都这么写
HttpClient httpClient = HttpClients.createDefault();
或者这么写:
HttpClient httpClient = HttpClientBuilder.create().build();
其实这两种写法都没错,HttpClients.createDefault()调用的就是HttpClientBuilder.create().build()。让我们看一下HttpClientBuilder.create().build()方法:
public CloseableHttpClient build() {
//省略部分代码......
//如果没有连接池,则创建默认的连接池
HttpClientConnectionManager connManagerCopy = this.connManager;
if (connManagerCopy == null) {
//省略部分代码......
final PoolingHttpClientConnectionManager poolingmgr = new PoolingHttpClientConnectionManager(
RegistryBuilder.create()
.register("http", PlainConnectionSocketFactory.getSocketFactory())
.register("https", sslSocketFactoryCopy)
.build(),
null,
null,
dnsResolver,
connTimeToLive,
connTimeToLiveTimeUnit != null ? connTimeToLiveTimeUnit : TimeUnit.MILLISECONDS);
//省略部分代码......
connManagerCopy = poolingmgr;
}
//其实就是创建MainClientExec实例的execChain
ClientExecChain execChain = createMainExec(
requestExecCopy,
connManagerCopy,
reuseStrategyCopy,
keepAliveStrategyCopy,
new ImmutableHttpProcessor(new RequestTargetHost(), new RequestUserAgent(userAgentCopy)),
targetAuthStrategyCopy,
proxyAuthStrategyCopy,
userTokenHandlerCopy);
//省略部分代码......
HttpRoutePlanner routePlannerCopy = this.routePlanner;
if (routePlannerCopy == null) {
SchemePortResolver schemePortResolverCopy = this.schemePortResolver;
if (schemePortResolverCopy == null) {
schemePortResolverCopy = DefaultSchemePortResolver.INSTANCE;
}
if (proxy != null) {
routePlannerCopy = new DefaultProxyRoutePlanner(proxy, schemePortResolverCopy);
} else if (systemProperties) {
routePlannerCopy = new SystemDefaultRoutePlanner(
schemePortResolverCopy, ProxySelector.getDefault());
} else {
routePlannerCopy = new DefaultRoutePlanner(schemePortResolverCopy);
}
}
//省略部分代码......
return new InternalHttpClient(
execChain,
connManagerCopy,
routePlannerCopy,
cookieSpecRegistryCopy,
authSchemeRegistryCopy,
defaultCookieStore,
defaultCredentialsProvider,
defaultRequestConfig != null ? defaultRequestConfig : RequestConfig.DEFAULT,
closeablesCopy);
}
它创建了一个InternalHttpClient对象,它有何不同呢?
让我们看一下InternalHttpClient产生请求的实际执行方法:
protected CloseableHttpResponse doExecute(
final HttpHost target,
final HttpRequest request,
final HttpContext context) throws IOException, ClientProtocolException {
Args.notNull(request, "HTTP request");
HttpExecutionAware execAware = null;
if (request instanceof HttpExecutionAware) {
execAware = (HttpExecutionAware) request;
}
try {
//省略部分代码......
final HttpRoute route = determineRoute(target, wrapper, localcontext);
return this.execChain.execute(route, wrapper, localcontext, execAware);
} catch (final HttpException httpException) {
throw new ClientProtocolException(httpException);
}
}
在真正执行前,它执行了determineRoute()方法,根据的请求url得到一个HttpRoute:
private HttpRoute determineRoute(
final HttpHost target,
final HttpRequest request,
final HttpContext context) throws HttpException {
HttpHost host = target;
if (host == null) {
host = (HttpHost) request.getParams().getParameter(ClientPNames.DEFAULT_HOST);
}
return this.routePlanner.determineRoute(host, request, context);
}
再看下真正执行的方法MainClientExec的execute()方法(在前面说了获取到的execChain是MainClientExec)
public CloseableHttpResponse execute(
final HttpRoute route,
final HttpRequestWrapper request,
final HttpClientContext context,
final HttpExecutionAware execAware) throws IOException, HttpException {
//省略部分代码......
//根据HttpRoute从连接池中获取连接
final ConnectionRequest connRequest = connManager.requestConnection(route, userToken);
if (execAware != null) {
if (execAware.isAborted()) {
connRequest.cancel();
throw new RequestAbortedException("Request aborted");
} else {
execAware.setCancellable(connRequest);
}
}
final RequestConfig config = context.getRequestConfig();
final HttpClientConnection managedConn;
try {
final int timeout = config.getConnectionRequestTimeout();
managedConn = connRequest.get(timeout > 0 ? timeout : 0, TimeUnit.MILLISECONDS);
} catch(final InterruptedException interrupted) {
Thread.currentThread().interrupt();
throw new RequestAbortedException("Request aborted", interrupted);
} catch(final ExecutionException ex) {
Throwable cause = ex.getCause();
if (cause == null) {
cause = ex;
}
throw new RequestAbortedException("Request execution failed", cause);
}
//省略部分代码......
final ConnectionHolder connHolder = new ConnectionHolder(this.log, this.connManager, managedConn);
try {
//省略部分代码......
HttpResponse response;
for (int execCount = 1;; execCount++) {
//省略部分代码......
response = requestExecutor.execute(request, managedConn, context);
//省略部分代码......
}
//省略部分代码......
}//省略部分代码......
}
看了以上源码,我得知PoolingHttpClientConnectionManager连接池的大概原理就是,它里面有多个路由,每个路由又有多个连接。我们每次访问都是现根据HttpHost找到HttpRoute,再根据HttpRoute获取连接。
现在我明白了为啥可以使用单例的HttpClient(实现类InternalHttpClient),它内部已经有连接池了,并不需要担心并发的时候,会出现等待或者其它问题。
如果我们平时总是有多个HttpClient,在build一个InternalHttpClient时,如果你没有设置连接池,它也会给你构建一个默认的连接池。这样就会多创建很多连接池,这就造成了浪费。