4、Httpclient源码解析之HTTP协议
初始化CloseableHttpClient过程中涉及ExecChainHandler & DefaultHttpProcessor，即典型客户端责任链中的请求执行处理器。
责任链中各节点涉及请求处理器【ExecChainHandler】顺序如下：RedirectExec、ContentCompressionExec、HttpRequestRetryExec、ProtocolExec、ConnectExec、MainClientExec。
DefaultHttpProcessor中HttpRequestInterceptor类型的数组【requestInterceptors[]】包含RequestDefaultHeaders、RequestContent、RequestTargetHost、RequestClientConnControl、RequestUserAgent、RequestExpectContinue、RequestAddCookies、RequestAuthCache、ResponseProcessCookies。
1、InternalHttpClient
protected CloseableHttpResponse doExecute(HttpHost target,ClassicHttpRequest request,HttpContext context) {
...
final HttpClientContext localcontext = HttpClientContext.adapt(
context != null ? context : new BasicHttpContext());
RequestConfig config = null;
if (request instanceof Configurable) {
config = ((Configurable) request).getConfig();
}
if (config != null) {
localcontext.setRequestConfig(config);
}
final HttpRoute route = determineRoute(target, request, localcontext);
...
final ExecRuntime execRuntime = new InternalExecRuntime(log, connManager, requestExecutor,
request instanceof CancellableDependency ? (CancellableDependency) request : null);
final ExecChain.Scope scope = new ExecChain.Scope(exchangeId, route, request, execRuntime, localcontext);
final ClassicHttpResponse response = this.execChain.execute(ClassicRequestCopier.INSTANCE.copy(request), scope);
return CloseableHttpResponse.adapt(response);
}
HttpClientContext：生命周期为单次请求。
execChain：ExecChainElement。按照开头责任链中节点顺序执行。
2、ExecChainElement
class ExecChainElement {
private final ExecChainHandler handler;
private final ExecChainElement next;
ExecChainElement(final ExecChainHandler handler, final ExecChainElement next) {
this.handler = handler;
this.next = next;
}
public ClassicHttpResponse execute(
final ClassicHttpRequest request,
final ExecChain.Scope scope) throws IOException, HttpException {
return handler.execute(request, scope, new ExecChain() {//首个节点为RedirectExec
@Override
public ClassicHttpResponse proceed(
final ClassicHttpRequest request,
final Scope scope) throws IOException, HttpException {
return next.execute(request, scope);
}
});
}
2.1、RedirectExec
public ClassicHttpResponse execute(ClassicHttpRequest request,ExecChain.Scope scope,ExecChain chain) {
RedirectLocations redirectLocations = context.getRedirectLocations();
...
RequestConfig config = context.getRequestConfig();
for (int redirectCount = 0;;) {
ClassicHttpResponse response = chain.proceed(currentRequest, currentScope);
...
URI redirectUri = this.redirectStrategy.getLocationURI(currentRequest, response, context);
...
}
}
2.2、ContentCompressionExec
public ClassicHttpResponse execute(ClassicHttpRequest request,ExecChain.Scope scope,ExecChain chain) {
/* Signal support for Accept-Encoding transfer encodings. */
if (!request.containsHeader(HttpHeaders.ACCEPT_ENCODING) && requestConfig.isContentCompressionEnabled()) {
request.addHeader(acceptEncoding);
}
ClassicHttpResponse response = chain.proceed(request, scope);
...
}
2.3、HttpRequestRetryExec
public ClassicHttpResponse execute(ClassicHttpRequest request,ExecChain.Scope scope,ExecChain chain) {
for (int execCount = 1;; execCount++) {
response = chain.proceed(currentRequest, scope);
...
final HttpEntity entity = request.getEntity();
if (entity != null && !entity.isRepeatable()) {
return response;
}
if (retryStrategy.retryRequest(response, execCount, context)) {
response.close();
final TimeValue nextInterval = retryStrategy.getRetryInterval(response, execCount, context);
if (TimeValue.isPositive(nextInterval)) {
nextInterval.sleep();
}
currentRequest = ClassicRequestCopier.INSTANCE.copy(scope.originalRequest);
} else {
return response;
}
}
}
如果请求存在重试策略，则通过for循环实现请求多次请求服务端。
2.4、ProtocolExec
public ClassicHttpResponse execute(ClassicHttpRequest request,ExecChain.Scope scope,ExecChain chain) {
for (;;) {
//DefaultHttpProcessor
httpProcessor.process(request, request.getEntity(), context);
...
ClassicHttpResponse response = chain.proceed(request, scope);
}
}
2.4.1、DefaultHttpProcessor
for (final HttpRequestInterceptor requestInterceptor : this.requestInterceptors) {
requestInterceptor.process(request, entity, context);
}
2.5、ConnectExec
public ClassicHttpResponse execute(ClassicHttpRequest request,ExecChain.Scope scope,ExecChain chain) {
ExecRuntime execRuntime = scope.execRuntime;//InternalExecRuntime
...
execRuntime.acquireEndpoint(exchangeId, route, userToken, context);//#1
do {
final HttpRoute fact = tracker.toRoute();
step = this.routeDirector.nextStep(route, fact);
switch (step) {
case HttpRouteDirector.CONNECT_TARGET:
execRuntime.connectEndpoint(context);
tracker.connectTarget(route.isSecure());
break;
case HttpRouteDirector.CONNECT_PROXY:
execRuntime.connectEndpoint(context);
final HttpHost proxy = route.getProxyHost();
tracker.connectProxy(proxy, route.isSecure() && !route.isTunnelled());
break;
...
}
} while (step > HttpRouteDirector.COMPLETE);
}
return chain.proceed(request, scope);
}
- 步骤1：涉及获取连接池中的连接。
2.6、MainClientExec
public ClassicHttpResponse execute(ClassicHttpRequest request,ExecChain.Scope scope,ExecChain chain) {
ClassicHttpResponse response = execRuntime.execute(exchangeId, request, context);//InternalExecRuntime #1
...
if (reuseStrategy.keepAlive(request, response, context)) {
// Set the idle duration of this connection
final TimeValue duration = keepAliveStrategy.getKeepAliveDuration(response, context);
if (this.log.isDebugEnabled()) {
final String s;
if (duration != null) {
s = "for " + duration;
} else {
s = "indefinitely";
}
}
execRuntime.markConnectionReusable(userToken, duration);
} else {
execRuntime.markConnectionNonReusable();
}
ResponseEntityProxy.enhance(response, execRuntime);//#2
return new CloseableHttpResponse(response, execRuntime);
}
- 步骤1：参考2.6.1之HttpRequestExecutor。
- 最终返回响应体ResponseEntityProxy。
2.6.1、HttpRequestExecutor
public ClassicHttpResponse execute(ClassicHttpRequest request,HttpClientConnection conn,
HttpResponseInformationCallback informationCallback,HttpContext context) {
...
ClassicHttpResponse response = null;
while (response == null) {
if (expectContinue) {
...
} else {
//#1 状态行 & 响应头 DefaultManagedHttpClientConnection#receiveResponseHeader
response = conn.receiveResponseHeader();
if (streamListener != null) {
streamListener.onResponseHead(conn, response);
}
final int status = response.getCode();
...
if (status < HttpStatus.SC_SUCCESS) {
if (informationCallback != null && status != HttpStatus.SC_CONTINUE) {
informationCallback.execute(response, conn, context);
}
response = null;
}
}
}
if (MessageSupport.canResponseHaveBody(request.getMethod(), response)) {
//conn：DefaultBHttpClientConnection
conn.receiveResponseEntity(response);//#2 响应体IncomingHttpEntity
}
return response;
}
- 步骤1：从原始输入流
socketHolder.getInputStream()获取默认8192个字节于SessionInputBufferImpl。最终从SessionInputBufferImpl中获取当前响应的 状态行 & 响应头。 - 步骤2：解析响应体IncomingHttpEntity，响应体中InputStream类型的属性content其取值为ContentLengthInputStream。
@Override
public void receiveResponseEntity( final ClassicHttpResponse response) throws HttpException, IOException {
...
response.setEntity(createIncomingEntity(response, this.inBuffer, socketHolder.getInputStream(), len));
}
HttpEntity createIncomingEntity(HttpMessage message,SessionInputBuffer inBuffer,InputStream inputStream,long len) {
return new IncomingHttpEntity(
createContentInputStream(len, inBuffer, inputStream),
len >= 0 ? len : -1, len == ContentLengthStrategy.CHUNKED,
message.getFirstHeader(HttpHeaders.CONTENT_TYPE),
message.getFirstHeader(HttpHeaders.CONTENT_ENCODING));
}
protected InputStream createContentInputStream(long len,SessionInputBuffer buffer,InputStream inputStream) {
if (len > 0) {
return new ContentLengthInputStream(buffer, inputStream, len);
} else if (len == 0) {
return EmptyInputStream.INSTANCE;
} else if (len == ContentLengthStrategy.CHUNKED) {
return new ChunkedInputStream(buffer, inputStream, this.http1Config);
} else {
return new IdentityInputStream(buffer, inputStream);
}
}
2、SessionInputBufferImpl & SessionOutputBufferImpl
DefaultBHttpClientConnection实例化过程中触发SessionInputBufferImpl、SessionOutputBufferImpl的实例化。
SessionInputBufferImpl：Abstract base class for session input buffers that stream data from an arbitrary {@link InputStream}. This class buffers input data in an internal byte array for optimal input performance。
public int read(final byte[] b, final int off, final int len, final InputStream inputStream) throws IOException {
...
if (hasBufferedData()) {//#1
final int chunk = Math.min(len, this.bufferLen - this.bufferPos);
System.arraycopy(this.buffer, this.bufferPos, b, off, chunk);
this.bufferPos += chunk;
return chunk;
}
if (len > this.minChunkLimit) {
final int read = inputStream.read(b, off, len);//#2
if (read > 0) {
this.metrics.incrementBytesTransferred(read);
}
return read;
}
while (!hasBufferedData()) {//#3
final int readLen = fillBuffer(inputStream);
if (readLen == -1) {
return -1;
}
}
final int chunk = Math.min(len, this.bufferLen - this.bufferPos);
System.arraycopy(this.buffer, this.bufferPos, b, off, chunk);
this.bufferPos += chunk;
return chunk;
}
步骤1：将章节2.6.1中读取到部分响应字节复制到字节数组b，最终追加到CharArrayBuffer。
步骤2：剩下的响应体字节直接从原始的输入流socketHolder.getInputStream()直接读取。
步骤3：章节2.6.1中首次解析状态行 & 响应头时，将默认8192个字节数从原始输入流读取到SessionInputBufferImpl中。
3、SocketHolder
Utility class that holds a {@link Socket} along with copies of its {@link InputStream} and {@link OutputStream}。
4、ContentLengthInputStream
Input stream that cuts off after a defined number of bytes【被定义好的字节，其实就是指状态行、响应头】。This class is used to receive content of HTTP messages where the end of the content entity is determined by the value of the {@code Content-Length header}【响应体的字节长度是通过响应头中Content-Length其value决定的】。Entities transferred using this stream can be maximum {@link Long#MAX_VALUE} long【使用此流传输的实体可以是long最大值】。
5、响应体IncomingHttpEntity
5.1、ChunkedInputStream
该实体类是抽象类InputStream的子类。真正实现流读写的是内部属性之原始输入流inputStream【网络流SocketInputStream】。
public int read (final byte[] b, final int off, final int len) throws IOException {
if (eof) {
return -1;
}
if (state != State.CHUNK_DATA) {
nextChunk();
if (eof) {
return -1;
}
}
final int bytesRead = buffer.read(b, off, (int) Math.min(len, chunkSize - pos), inputStream);//#1
if (bytesRead != -1) {
pos += bytesRead;
if (pos >= chunkSize) {
state = State.CHUNK_CRLF;
}
return bytesRead;
}
eof = true;
}
步骤1：SessionInputBufferImpl#read。
5.1、流关闭
public void close() throws IOException {
if (!closed) {
try {
if (!eof && state != State.CHUNK_INVALID) {//此处正常不会执行
final byte[] buff = new byte[BUFFER_SIZE];
while (read(buff) >= 0) {
}
}
} finally {
eof = true;//表示输入流中字节读取完毕 True if we've reached the end of stream
closed = true;
}
}
}
6、ResponseEntityProxy
ResponseEntityProxy(final HttpEntity entity, final ExecRuntime execRuntime) {
super(entity);
this.execRuntime = execRuntime;
}
entity：IncomingHttpEntity。
execRuntime：InternalExecRuntime。
public boolean eofDetected(final InputStream wrapped) throws IOException {
if (wrapped != null) {
wrapped.close();//ChunkedInputStream#close
}
releaseConnection();
return false;
}
6.1、EofSensorInputStream
public InputStream getContent() throws IOException {
return new EofSensorInputStream(super.getContent(), this);
}
super.getContent()：通过IncomingHttpEntity获取到ChunkedInputStream。
public int read(final byte[] b, final int off, final int len) throws IOException {
int readLen = -1;
if (isReadAllowed()) {
readLen = wrappedStream.read(b, off, len);//ChunkedInputStream#read
checkEOF(readLen);
}
return readLen;
}
7.1、EntityUtils.toString
private static String toString(final HttpEntity entity, final ContentType contentType, final int maxResultLength){
final int contentLength = toContentLength((int) Args.checkContentLength(entity));
...
InputStream inStream = entity.getContent();//ResponseEntityProxy#getContent
return toCharArrayBuffer(inStream, contentLength, charset, maxResultLength).toString();
}
private static CharArrayBuffer toCharArrayBuffer(final InputStream inStream, final long contentLength,
final Charset charset, final int maxResultLength){
final Charset actualCharset = charset == null ? DEFAULT_CHARSET : charset;
final CharArrayBuffer buf = new CharArrayBuffer(
Math.min(maxResultLength, contentLength > 0 ? (int) contentLength : DEFAULT_CHAR_BUFFER_SIZE));
final Reader reader = new InputStreamReader(inStream, actualCharset);
final char[] tmp = new char[DEFAULT_CHAR_BUFFER_SIZE];
int chReadCount;
while ((chReadCount = reader.read(tmp)) != -1) {//#1
buf.append(tmp, 0, chReadCount);
}
return buf;
}
- 步骤1：EofSensorInputStream#read。
7.2、CharArrayBuffer
该实体就是最终返回到用户的实体类。
8、涉及流的关闭之EofSensorInputStream
private void checkEOF(final int eof) throws IOException {
final InputStream toCheckStream = wrappedStream;//ChunkedInputStream
if ((toCheckStream != null) && (eof < 0)) {
try {
boolean scws = true;
if (eofWatcher != null) {
scws = eofWatcher.eofDetected(toCheckStream);//ResponseEntityProxy
}
if (scws) {
toCheckStream.close();
}
} finally {
wrappedStream = null;
}
}
}
整个HTTP请求过程中只涉及ChunkedInputStream、原始输入流SocketInputStream。但是在触发流关闭的过程中并未涉及SocketInputStream#close方法，不知道为啥？