httpclient总结
1、httpclient总结：
一、基本知识准备
(1)构建URI工具类，URIBuilder
(2)HttpResponse类，可以添加Header信息
获取所有Header信息的方法，调用HeaderIterator接口
HeaderIterator it = response.headerIterator("Set-Cookie");
while (it.hasNext()) {
System.out.println(it.next());
}
(3)HttpEntity内容实体 可以被装入Request和Response中..
只有独立的entity才可以被重复调用.
当请求需要entity HttpEntity.writeTo(OutputStream)
从响应中解析entity HttpEntity.getContent()
HttpEntity.getContentType()
HttpEntity.getContentLength()
HttpEntity.getContentEncoding()
对entity进行解析可采用流的方式或者调用EntityUtils，但后者有长度的限制2048
利用 BufferedEntity可以将entity缓存到本地磁盘，用来进行多次读取.
创建entity信息时需要指定meta信息，包括contentType
(4)可以调用ResponseHandler写入响应统一处理
二、常用策略
keep-Alieve策略:自定义ConnectionKeepAliveStrategy
重定向策略:LaxRedirectStrategy
三、资源分配
当CloseableHttpClient不再需要，并且不再连接管理的范围，需要调用CloseableHttpClient.close()方法将其关闭..
四、HttpClient状态管理
1、在HTTP上下文中，很多有逻辑关系的请求都可以放入到同一个session中..
HttpClient本身线程HttpContext 包含任意的键值对，因此线程不安全..通常建议每个线程拥有自己的上下文
2、自动恢复机制---->HttpRequestRetryHandler
HttpRequestRetryHandler myRetryHandler = new HttpRequestRetryHandler() { public boolean retryRequest(IOException exception,int executionCount,HttpContext context) { if (executionCount >= 5) { // Do not retry if over max retry count return false; } if (exception instanceof InterruptedIOException) { // Timeout return false; } if (exception instanceof UnknownHostException) { // Unknown host return false; } if (exception instanceof ConnectTimeoutException) { // Connection refused return false; } if (exception instanceof SSLException) { // SSL handshake exception return false; } HttpClientContext clientContext = HttpClientContext.adapt(context); HttpRequest request = clientContext.getRequest(); boolean idempotent = !(request instanceof HttpEntityEnclosingRequest); if (idempotent) { // Retry if the request is considered idempotent return true; } return false; } }; CloseableHttpClient httpclient = HttpClients.custom() .setRetryHandler(myRetryHandler) .build();
4、多线程中的应用AtomicInteger，，，，待研究...
从连接管理器中获取连接 (1)通过HttpClientConnectionManager来管理一个连接 HttpClientContext context = HttpClientContext.create(); HttpClientConnectionManager connMrg = new BasicHttpClientConnectionManager(); HttpRoute route = new HttpRoute(new HttpHost("www.yeetrack.com", 80)); // 获取新的连接. 这里可能耗费很多时间 ConnectionRequest connRequest = connMrg.requestConnection(route, null); // 10秒超时 HttpClientConnection conn = connRequest.get(10, TimeUnit.SECONDS); try { // 如果创建连接失败 if (!conn.isOpen()) { // establish connection based on its route info connMrg.connect(conn, route, 1000, context); // and mark it as route complete connMrg.routeComplete(conn, route, context); } // 进行自己的操作. } finally { connMrg.releaseConnection(conn, null, 1, TimeUnit.MINUTES); } 通过更复杂的PoolingHttpClientConnectionManager来管理多个连接，适合多线程中的请求 PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(); // 将最大连接数增加到200 cm.setMaxTotal(200); // 将每个路由基础的连接增加到20 cm.setDefaultMaxPerRoute(20); //将目标主机的最大连接数增加到50 HttpHost localhost = new HttpHost("www.yeetrack.com", 80); cm.setMaxPerRoute(new HttpRoute(localhost), 50); CloseableHttpClient httpClient = HttpClients.custom() .setConnectionManager(cm) .build(); 示例1--------------------------------- PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(); CloseableHttpClient httpClient = HttpClients.custom() .setConnectionManager(cm) .build(); // URL列表数组 String[] urisToGet = { "http://www.domain1.com/", "http://www.domain2.com/", "http://www.domain3.com/", "http://www.domain4.com/" }; // 为每个url创建一个线程，GetThread是自定义的类 GetThread[] threads = new GetThread[urisToGet.length]; for (int i = 0; i < threads.length; i++) { HttpGet httpget = new HttpGet(urisToGet[i]); threads[i] = new GetThread(httpClient, httpget); } // 启动线程 for (int j = 0; j < threads.length; j++) { threads[j].start(); } // join the threads for (int j = 0; j < threads.length; j++) { threads[j].join(); } 自定义类GetThread static class GetThread extends Thread { private final CloseableHttpClient httpClient; private final HttpContext context; private final HttpGet httpget; public GetThread(CloseableHttpClient httpClient, HttpGet httpget) { this.httpClient = httpClient; this.context = HttpClientContext.create(); this.httpget = httpget; } @Override public void run() { try { CloseableHttpResponse response = httpClient.execute( httpget, context); try { HttpEntity entity = response.getEntity(); } finally { response.close(); } } catch (ClientProtocolException ex) { // Handle protocol errors } catch (IOException ex) { // Handle I/O errors } } }
注意：即使httpclient可以被多线程访问，仍建议每个httpclient采用自己的context
5、 public static class IdleConnectionMonitorThread extends Thread { private final HttpClientConnectionManager connMgr; private volatile boolean shutdown; public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) { super(); this.connMgr = connMgr; } @Override public void run() { try { while (!shutdown) { synchronized (this) { wait(5000); // 关闭失效的连接 connMgr.closeExpiredConnections(); // 可选的, 关闭30秒内不活动的连接 connMgr.closeIdleConnections(30, TimeUnit.SECONDS); } } } catch (InterruptedException ex) { // terminate } } public void shutdown() { shutdown = true; synchronized (this) { notifyAll(); } } }