根据业务量级决定使用同步调用或异步调用：异步回调方式的并发性非常高，缺点是代码可读性一般，在开发中，我会首先选择同步实现，在遇到性能问题后再考虑优化为异步回调方式。在Spring项目中使用HttpClient时，可以借用FactoryBean的概念，编写自己的HttpClientFactoryBean，我在LeanJava中写了一个例子：link
一、同步HttpClient
首先编写HttpClientFactoryBean，代码和其中关键的几个参数的解释如下：
package org.java.learn.httpclient;
import org.apache.commons.codec.Charsets;
import org.apache.http.NoHttpResponseException;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.config.ConnectionConfig;
import org.apache.http.conn.ConnectTimeoutException;
import org.apache.http.impl.client.HttpClients;
import org.springframework.beans.factory.FactoryBean;
import java.net.SocketTimeoutException;
/**
* Created by IntelliJ IDEA.
* User: duqi
* Date: 2017/2/9
* Time: 13:54
*/
public class HttpClientFactoryBean implements FactoryBean {
// 知识点1：路由(MAX_PER_ROUTE)是对最大连接数（MAX_TOTAL）的细分，整个连接池的限制数量实际使用DefaultMaxPerRoute并非MaxTotal。
// 设置过小无法支持大并发(ConnectionPoolTimeoutException: Timeout waiting for connection from pool)，
private static final int DEFAULT_MAX_TOTAL = 512; //最大支持的连接数
private static final int DEFAULT_MAX_PER_ROUTE = 64; //针对某个域名的最大连接数
private static final int DEFAULT_CONNECTION_TIMEOUT = 5000; //知识点2：跟目标服务建立连接超时时间，根据自己的业务调整
private static final int DEFAULT_SOCKET_TIMEOUT = 3000; //知识点3：请求的超时时间（建联后，获取response的返回等待时间）
private static final int DEFAULT_TIMEOUT = 1000; //知识点4：从连接池中获取连接的超时时间
@Override
public HttpClient getObject() throws Exception {
ConnectionConfig config = ConnectionConfig.custom()
.setCharset(Charsets.UTF_8)
.build();
RequestConfig defaultRequestConfig = RequestConfig.custom()
.setConnectTimeout(DEFAULT_CONNECTION_TIMEOUT)
.setSocketTimeout(DEFAULT_SOCKET_TIMEOUT)
.setConnectionRequestTimeout(DEFAULT_TIMEOUT)
.build();
return HttpClients.custom()
.setMaxConnPerRoute(DEFAULT_MAX_PER_ROUTE)
.setMaxConnTotal(DEFAULT_MAX_TOTAL)
.setRetryHandler((exception, executionCount, context) -> executionCount <= 3 && (exception instanceof NoHttpResponseException
|| exception instanceof ClientProtocolException
|| exception instanceof SocketTimeoutException
|| exception instanceof ConnectTimeoutException))
.setDefaultConnectionConfig(config)
.setDefaultRequestConfig(defaultRequestConfig)
.build();
}
@Override
public Class getObjectType() {
return HttpClient.class;
}
@Override
public boolean isSingleton() {
return true;
}
}
第二，在xml文件中进行如下配置，配置完这一步后，就可以在其他spring bean中编入httpclient使用了。
第三，编写单元测试，检查是否可用
import org.apache.http.client.HttpClient;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import javax.annotation.Resource;
/**
* Created by IntelliJ IDEA.
* User: duqi
* Date: 2017/2/9
* Time: 14:18
*/
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = "classpath:applicationContext.xml")
public class HttpClientFactoryBeanXmlTest {
@Resource
HttpClient httpClient;
@Test
public void httpClientAutoWired() throws Exception {
Assert.assertNotNull(httpClient);
}
}
二、异步HttpClient
首先编写AsyncHttpClientFactoryBean，几个关于超时时间的参数和之前相同。这里需要简单理解ioReactor的含义——Async HttpClient使用了Reactor模式，该模式又有别名Dispatcher或Notifier。
从Netty源码解读（四）Netty与Reactor模式一文可以看到，在Reactor模式中，有一个不断循环的线程监听一个队列，每个异步请求发出去以后，就会在这个队列里注册一个handler（call back对象），当某个请求响应回来后，由中间人负责调用对应的handler，这个中间人的名字就是Reactor。
AsyncHttpClientFactoryBean的代码如下：
package org.java.learn.httpclient;
import org.apache.commons.lang3.concurrent.BasicThreadFactory;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.impl.nio.client.CloseableHttpAsyncClient;
import org.apache.http.impl.nio.client.HttpAsyncClients;
import org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager;
import org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor;
import org.apache.http.impl.nio.reactor.IOReactorConfig;
import org.springframework.beans.factory.FactoryBean;
/**
* Created by IntelliJ IDEA.
* User: duqi
* Date: 2017/2/9
* Time: 15:06
*/
public class AsyncHttpClientFactoryBean implements FactoryBean {
private static final int DEFAULT_MAX_TOTAL = 512;
private static final int DEFAULT_MAX_PER_ROUTE = 64;
private static final int DEFAULT_CONNECTION_TIMEOUT = 5000;
private static final int DEFAULT_SOCKET_TIMEOUT = 3000;
private static final int DEFAULT_TIMEOUT = 1000;
@Override
public CloseableHttpAsyncClient getObject() throws Exception {
DefaultConnectingIOReactor ioReactor = new DefaultConnectingIOReactor(IOReactorConfig.custom()
.setSoKeepAlive(true).build());
PoolingNHttpClientConnectionManager pcm = new PoolingNHttpClientConnectionManager(ioReactor);
pcm.setMaxTotal(DEFAULT_MAX_TOTAL);
pcm.setDefaultMaxPerRoute(DEFAULT_MAX_PER_ROUTE);
RequestConfig defaultRequestConfig = RequestConfig.custom()
.setConnectTimeout(DEFAULT_CONNECTION_TIMEOUT)
.setSocketTimeout(DEFAULT_SOCKET_TIMEOUT)
.setConnectionRequestTimeout(DEFAULT_TIMEOUT)
.build();
return HttpAsyncClients.custom()
.setThreadFactory(new BasicThreadFactory.Builder().namingPattern("AysncHttpThread-%d").build())
.setConnectionManager(pcm)
.setDefaultRequestConfig(defaultRequestConfig)
.build();
}
@Override
public Class getObjectType() {
return CloseableHttpAsyncClient.class;
}
@Override
public boolean isSingleton() {
return true;
}
}
和之前一样，我们在单元测试中测试了，该FactoryBean已经可以正常为我们生产CloseableHttpAsyncClient对象，现在需要看下如何使用该对象：
private static final Semaphore concurrency = new Semaphore(1024);
@Test
public void asyncClientTest() throws Exception {
Assert.assertNotNull(asyncClient);
//step1 获取信号量控制并发数（防止内存溢出）
concurrency.acquireUninterruptibly();
try {
//step2 设置HttpUrlRequest
final HttpUriRequest httpUriRequest = RequestBuilder.get()
.setUri("http://www.baidu.com")
.build();
//step3 执行异步调用
asyncClient.execute(httpUriRequest, new FutureCallback() {
@Override
public void completed(HttpResponse httpResponse) {
//处理Http响应
}
@Override
public void failed(Exception e) {
//根据情况进行重试
}
@Override
public void cancelled() {
//记录失败日志
}
});
} finally {
//step4 释放信号量
concurrency.release();
}
}
上面四步就是我们使用异步httpclient的常规模式，这里需要使用信号量控制并发，原因是：中间人（Reactor）维护的handler队列是一个无界队列，如果目标服务挂了，这边的请求并发量又很高，就会造成队列无限增长，从而造成OOM。
三、参考文章
- 使用httpclient必须知道的参数设置及代码写法、存在的风险
- ConnectionTimeout, SocketTimeout values set are not effective](http://stackoverflow.com/questions/9725492/java-httpclient-4-1-2-connectiontimeout-sockettimeout-values-set-are-not-ef)
本号专注于后端技术、JVM问题排查和优化、Java面试题、个人成长和自我管理等主题，为读者提供一线开发者的工作和成长经验，期待你能在这里有所收获。
javaadu