上一篇通过分析Retrofit2的源码,我们了解了整个请求流程的底层实现,其最终是通过OKHttp3中的OkHttpClient对象创建一个RealCall来完成实际请求的。本篇我们继续上一篇的内容,还是通过源码来分析这个流程的底层实现。
我们在请求网络时,要么用同步execute(),要么用异步enqueue(Callback responseCallback),在这里,我们只分析异步,同步方法很简单,异步是我们经常在开发中用到的。
我们先来看RealCall的enqueue这个方法:
@Override
public void enqueue(Callback responseCallback) {
synchronized (this) {
if (executed) throw new IllegalStateException("Already Executed");
executed = true;
}
client.dispatcher().enqueue(new AsyncCall(responseCallback));
}
首先我们需要明白,一个RealCall对象就是一个请求,我们在前一篇文章就讲了Retrofit2在create方法的动态代理中返回的就是一个RealCall对象,所以当我们进行异步请求的时候,调用的就是RealCall的这个enqueue方法。从源码可以看出,先判断该任务是否已经执行,如果已经执行了,则抛出异常,否则去执行任务,并改变状态。这里我们看到它是通过调用client.dispatcher()的enqueue方法去执行的,client就是OkHttpClient对象,我们看一下他的dispatcher()方法:
public Dispatcher dispatcher() {
return dispatcher;
}
public Builder() {
dispatcher = new Dispatcher();
...
}
得知返回的是一个Dispatcher类的对象,在分析它的enqueue方法之前我们先来看一下Dispatcher类的几个成员变量:
/** Executes calls. Created lazily. */
private ExecutorService executorService;
/** Ready async calls in the order they'll be run. */
private final Deque readyAsyncCalls = new ArrayDeque<>();
/** Running asynchronous calls. Includes canceled calls that haven't finished yet. */
private final Deque runningAsyncCalls = new ArrayDeque<>();
/** Running synchronous calls. Includes canceled calls that haven't finished yet. */
private final Deque runningSyncCalls = new ArrayDeque<>();
public synchronized ExecutorService executorService() {
if (executorService == null) {
executorService = new ThreadPoolExecutor(0, Integer.MAX_VALUE, 60, TimeUnit.SECONDS,
new SynchronousQueue(), Util.threadFactory("OkHttp Dispatcher", false));
}
return executorService;
}
executorService是一个线程池执行器,专门执行call任务的;readyAsyncCalls是一个存放待执行的异步任务队列,runningAsyncCalls是一个正在执行的异步任务队列,从注释可以看出来,它包括还没执行完成但是被取消的任务;runningSyncCalls则是正在执行的同步任务队列。现在我们再来分析enqueue方法;
synchronized void enqueue(AsyncCall call) {
if (runningAsyncCalls.size() < maxRequests && runningCallsForHost(call) < maxRequestsPerHost) {
runningAsyncCalls.add(call);
executorService().execute(call);
} else {
readyAsyncCalls.add(call);
}
}
首先,判断如果正在执行的任务数量 小于 最大执行数 并且 对一个主机的同时请求数还未超过最大值时,就把该任务加入正在执行的队列中,并将它交给线程池executorService去立即执行。否则就将该任务加入待执行的队列中,等待有任务被执行完成后再到该队列中拿取去执行。
现在我们来看这个AsyncCall,它是在RealCall类中定义的内部类,我们来看一下它的具体实现。
public abstract class NamedRunnable implements Runnable {
protected final String name;
public NamedRunnable(String format, Object... args) {
this.name = Util.format(format, args);
}
@Override public final void run() {
String oldName = Thread.currentThread().getName();
Thread.currentThread().setName(name);
try {
execute();
} finally {
Thread.currentThread().setName(oldName);
}
}
protected abstract void execute();
}
final class AsyncCall extends NamedRunnable {
private final Callback responseCallback;
private AsyncCall(Callback responseCallback) {
super("OkHttp %s", redactedUrl().toString());
this.responseCallback = responseCallback;
}
String host() {
return originalRequest.url().host();
}
Request request() {
return originalRequest;
}
RealCall get() {
return RealCall.this;
}
@Override protected void execute() {
boolean signalledCallback = false;
try {
Response response = getResponseWithInterceptorChain();
if (retryAndFollowUpInterceptor.isCanceled()) {
signalledCallback = true;
responseCallback.onFailure(RealCall.this, new IOException("Canceled"));
} else {
signalledCallback = true;
responseCallback.onResponse(RealCall.this, response);
}
} catch (IOException e) {
if (signalledCallback) {
// Do not signal the callback twice!
Platform.get().log(INFO, "Callback failure for " + toLoggableString(), e);
} else {
responseCallback.onFailure(RealCall.this, e);
}
} finally {
client.dispatcher().finished(this);
}
}
}
可以看到AsyncCall是继承了NamedRunnable,而NamedRunnable继承了Runnable,所以该AsyncCall是可以被线程池执行器直接执行的对象。我们看到NamedRunnable覆写了run方法,加入了线程跟踪标记,然后采用模板方法,调用子类的
execute()方法区执行任务,执行完成后还原线程信息。我们再来看一下AsyncCall的execute()方法,现在,重量级别的方法终于登场了:getResponseWithInterceptorChain(),我们稍后再做讲解,这里只要知道它做了两件事,发送请求,返回响应信息。然后我们接着往下看,如果etryAndFollowUpInterceptor拦截器取消了,就回调onFailure,否则回调onResponse方法,并将返回的响应信息传递过去,signalledCallback是标记是否进行了回调。如果上面过程出现了异常,则打印日志,没有回调就回调。最后,调用分发器Dispatcher的finished方法结束本次任务。我们来看一下这个finished方法:
/** Used by {@code AsyncCall#run} to signal completion. */
void finished(AsyncCall call) {
finished(runningAsyncCalls, call, true);
}
private void finished(Deque calls, T call, boolean promoteCalls) {
int runningCallsCount;
Runnable idleCallback;
synchronized (this) {
if (!calls.remove(call)) throw new AssertionError("Call wasn't in-flight!");
if (promoteCalls) promoteCalls();
runningCallsCount = runningCallsCount();
idleCallback = this.idleCallback;
}
if (runningCallsCount == 0 && idleCallback != null) {
idleCallback.run();
}
}
private void promoteCalls() {
if (runningAsyncCalls.size() >= maxRequests) return; // Already running max capacity.
if (readyAsyncCalls.isEmpty()) return; // No ready calls to promote.
for (Iterator i = readyAsyncCalls.iterator(); i.hasNext(); ) {
AsyncCall call = i.next();
if (runningCallsForHost(call) < maxRequestsPerHost) {
i.remove();
runningAsyncCalls.add(call);
executorService().execute(call);
}
if (runningAsyncCalls.size() >= maxRequests) return; // Reached max capacity.
}
}
可以看到,将完成的任务从runningAsyncCalls这个队列中移除,然后,如果runningAsyncCalls中的任务数量小于最大请求数并且readyAsyncCalls中有待被执行的任务的时候,就遍历readyAsyncCalls这个集合拿取任务,添加到runningAsyncCalls队列中,并将任务交给线程池执行器去执行,如此循环。
总结一下异步任务的流程:
1.添加一个新异步任务时,如果正在执行的任务队列容量还未达到最大值,并且该任务请求的主机正同时处理的任务还未达到最大值时,就将该任务添加到正在执行的任务队列中,并立即执行该任务。
2.如果正在执行的任务队列已满,则将其添加到待执行的队列中
3.任务执行完成后,将该任务从正在执行的任务队列移除,并从待执行的队列中拿取下一个任务,如此循环。
但从上面的这些来看,OKHttp3相比于其他的网络框架还不具备特别的优势,它牛就牛在getResponseWithInterceptorChain这个方法中,现在我们就来看看这个方法到底做了什么:
private Response getResponseWithInterceptorChain() throws IOException {
// Build a full stack of interceptors.
List interceptors = new ArrayList<>();
interceptors.addAll(client.interceptors());
interceptors.add(retryAndFollowUpInterceptor);
interceptors.add(new BridgeInterceptor(client.cookieJar()));
interceptors.add(new CacheInterceptor(client.internalCache()));
interceptors.add(new ConnectInterceptor(client));
if (!retryAndFollowUpInterceptor.isForWebSocket()) {
interceptors.addAll(client.networkInterceptors());
}
interceptors.add(new CallServerInterceptor(
retryAndFollowUpInterceptor.isForWebSocket()));
Interceptor.Chain chain = new RealInterceptorChain(
interceptors, null, null, null, 0, originalRequest);
return chain.proceed(originalRequest);
}
首先,创建了一个ArrayList,然后将我们自定义的拦截器添加进去,client.interceptors()就是我们在初始化OkHttpClient的时候自定义的拦截器。然后依次添加自带的五个拦截器,注意他们的顺序,在添加最后一个拦截器之前会添加我们自定义的网络拦截器。最后将这个拦截器集合与原始请求信息封装到RealInterceptorChain对象中,调用该对象的proceed方法让整条链开始工作起来。这5个拦截器稍后讲解,我们先来看看RealInterceptorChain这个类:
public final class RealInterceptorChain implements Interceptor.Chain {
public RealInterceptorChain(List interceptors, StreamAllocation streamAllocation,
HttpStream httpStream, Connection connection, int index, Request request) {
this.interceptors = interceptors;
this.connection = connection;
this.streamAllocation = streamAllocation;
this.httpStream = httpStream;
this.index = index;
this.request = request;
}
@Override
public Response proceed(Request request) throws IOException {
return proceed(request, streamAllocation, httpStream, connection);
}
public Response proceed(Request request, StreamAllocation streamAllocation, HttpStream httpStream,Connection connection) throws IOException {
if (index >= interceptors.size()) throw new AssertionError();
calls++;
// If we already have a stream, confirm that the incoming request will use it.
if (this.httpStream != null && !sameConnection(request.url())) {
throw new IllegalStateException("network interceptor " + interceptors.get(index - 1)
+ " must retain the same host and port");
}
// If we already have a stream, confirm that this is the only call to chain.proceed().
if (this.httpStream != null && calls > 1) {
throw new IllegalStateException("network interceptor " + interceptors.get(index - 1)
+ " must call proceed() exactly once");
}
// Call the next interceptor in the chain.
RealInterceptorChain next = new RealInterceptorChain(
interceptors, streamAllocation, httpStream, connection, index + 1, request);
Interceptor interceptor = interceptors.get(index);
Response response = interceptor.intercept(next);
// Confirm that the next interceptor made its required call to chain.proceed().
if (httpStream != null && index + 1 < interceptors.size() && next.calls != 1) {
throw new IllegalStateException("network interceptor " + interceptor
+ " must call proceed() exactly once");
}
// Confirm that the intercepted response isn't null.
if (response == null) {
throw new NullPointerException("interceptor " + interceptor + " returned null");
}
return response;
}
}
前面在构造RealInterceptorChain对象的时候connection,streamAllocation,httpStream这三个传的都是null,index = 0,所以,我们重点看proceed方法中的这三行代码:
// Call the next interceptor in the chain.
RealInterceptorChain next = new RealInterceptorChain(interceptors, streamAllocation, httpStream, connection, index + 1, request);
Interceptor interceptor = interceptors.get(index);
Response response = interceptor.intercept(next);
首先,又创建了一个RealInterceptorChain对象,和之前那个对象一样,区别在于此处的index变成了index+1,然后从拦截器集合中拿取第一个拦截器,执行它的intercept方法,并将这个新的RealInterceptorChain对象传递进去。这里有小伙伴可能会有疑问,为什么需要创建新对象next,而不是直接复用上一层的RealInterceptorChain对象?因为这里是递归调用,在调用下一层拦截器的interupter()方法的时候,本层的 response阶段还没有执行完成,如果复用RealInterceptorChain对象,必然导致下一层修改该RealInterceptorChain对象,所以需要重新创建RealInterceptorChain对象。
所有的拦截器都实现了Interceptor接口,并覆写了intercept方法,在这个方法中,都会调用chain.proceed(request)用来将请求传递下去,直到最后一个拦截器CallServerInterceptor被调用,并在接收到服务器返回后读取响应信息,再一层一层往回传递,等所有的拦截器都各自处理完成响应信息后,整个getResponseWithInterceptorChain方法才调用完成。下面简要罗列一下这五个拦截器的intercept方法主要代码:
/**
* 主要作用:
* ①在网络请求失败后进行重试;
* ②当服务器返回当前请求需要进行重定向时直接发起新的请求,并在条件允许情况下复用当前连接
*/
public final class RetryAndFollowUpInterceptor implements Interceptor {
@Override
public Response intercept(Chain chain) throws IOException {
Request request = chain.request();
streamAllocation = new StreamAllocation(client.connectionPool(), createAddress(request.url()));
...
while (true) {
if (canceled) {
streamAllocation.release();
throw new IOException("Canceled");
}
response = ((RealInterceptorChain) chain).proceed(request, streamAllocation, null, null);
...
}
}
}
/**
* 这个拦截器是在v3.4.0之后由HttpEngine拆分出来的。
* 主要作用:
* ①设置内容长度,内容编码
* ②设置gzip压缩,并在接收到内容后进行解压。省去了应用层处理数据解压的麻烦
* 说到这里有一个小坑需要注意:
* 源码中注释也有说明,
* If we add an "Accept-Encoding: gzip" header field,
* we're responsible for also decompressing the transfer stream.
* 就是说OkHttp是自动支持gzip压缩的,也会自动添加header,这* 时候如果你自己添加了"Accept-Encoding: gzip",它就不管你了,就需要你自己解压缩。
* ③添加cookie
* ④设置其他报头,如User-Agent,Host,Keep-alive等。其中Keep-Alive是实现多路复用的必要步骤
*
*/
public final class BridgeInterceptor implements Interceptor {
@Override
public Response intercept(Chain chain) throws IOException {
Request userRequest = chain.request();
Request.Builder requestBuilder = userRequest.newBuilder();
...
Response networkResponse = chain.proceed(requestBuilder.build());
Response.Builder responseBuilder = networkResponse.newBuilder().request(userRequest);
...
return responseBuilder.build();
}
/**
* 这个拦截器是在v3.4.0之后由HttpEngine拆分出来的。
* 主要作用:
* ①当网络请求有符合要求的Cache时直接返回Cache
* ②当服务器返回内容有改变时更新当前cache
* ③如果当前cache失效,删除
*/
public final class CacheInterceptor implements Interceptor {
@Override public Response intercept(Chain chain) throws IOException {
...
CacheStrategy strategy = new CacheStrategy.Factory(now, chain.request(), cacheCandidate).get();
Request networkRequest = strategy.networkRequest;
Response cacheResponse = strategy.cacheResponse;
...
networkResponse = chain.proceed(networkRequest);
...
Response response = networkResponse.newBuilder()
.cacheResponse(stripBody(cacheResponse))
.networkResponse(stripBody(networkResponse))
.build();
...
return response;
}
}
/**
* 这个拦截器是在v3.4.0之后由HttpEngine拆分出来的。
* 主要作用:
* 为当前请求找到合适的连接,可能复用已有连接也可能是重新创建的连接,返回的连接由连接池负责决定
*
*/
public final class ConnectInterceptor implements Interceptor {
@Override
public Response intercept(Chain chain) throws IOException {
RealInterceptorChain realChain = (RealInterceptorChain) chain;
Request request = realChain.request();
StreamAllocation streamAllocation = realChain.streamAllocation();
// We need the network to satisfy this request. Possibly for validating a conditional GET.
boolean doExtensiveHealthChecks = !request.method().equals("GET");
HttpStream httpStream = streamAllocation.newStream(client, doExtensiveHealthChecks);
RealConnection connection = streamAllocation.connection();
return realChain.proceed(request, streamAllocation, httpStream, connection);
}
}
/**
* 主要作用:
* 负责向服务器发起真正的访问请求,并在接收到服务器返回后读取响应返回。
*/
public final class CallServerInterceptor implements Interceptor {
@Override
public Response intercept(Chain chain) throws IOException {
HttpStream httpStream = ((RealInterceptorChain) chain).httpStream();
...
httpStream.finishRequest();
Response response = httpStream.readResponseHeaders()
.request(request)
.handshake(streamAllocation.connection().handshake())
.sentRequestAtMillis(sentRequestMillis)
.receivedResponseAtMillis(System.currentTimeMillis())
.build();
...
return response;
}
}
/**
* 自定义参数拦截器 ,统一去掉所有的空格,制表符、换页符等空白字符
*/
public class ParamIntercept implements Interceptor {
@Override
public Response intercept(Chain chain) throws IOException {
Request request = chain.request();//拿到了request对象
HttpUrl requestUrlString = request.url();
String method = request.method();
if (method.equals("GET")) {
HashMap map = new HashMap<>();
Set names = requestUrlString.queryParameterNames();
ArrayList parameterNames = new ArrayList<>(names);
parameterNames.remove(0);//第一个是函数名非参数,要去掉
for (String key : parameterNames) { //循环参数列表
//去掉参数值中所有的空格,制表符、换页符等空白字符
if(TextUtils.isEmpty(requestUrlString.queryParameter(key))){
map.put(key, requestUrlString.queryParameter(key));
}else{
map.put(key, requestUrlString.queryParameter(key).replaceAll("\\s*", ""));
}
}
String url = requestUrlString.toString();
int index = url.indexOf("&");
if (index > 0) {
url = url.substring(0, index);
}
url = url + formatMap(map); //拼接新的url
request = request.newBuilder().url(url).build(); //重新构建请求
}
return chain.proceed(request);
}
private String formatMap(HashMap params) {
StringBuilder sb = new StringBuilder();
for (Map.Entry entry : params.entrySet()) {
sb.append("&")
.append(entry.getKey())
.append("=")
.append(entry.getValue());
}
return sb.substring(0, sb.length());
}
}
总结这整个链的过程就是:在对服务器发起请求之前,每个拦截器做完请求前的工作后调用 chain.proceed(request)递归将请求一层一层传递下去,直到最后发送请求到服务器;在服务器返回响应之后,读取响应信息,封装成Response对象,并将此对象又一层一层的往回传递到每一个拦截器中,拦截器拿到这个响应信息后又可以做自己的事,完成之后将此Response返回。这样一层一层的传递过去,每层都只专注于自己的工作,不用管其它层的影响,非常利于扩展,这种责任链模式在计算机行业用的很多,我们熟知的TCP/IP协议的五层模型和OSI七层模型都是典型的这种思想,用过爬虫框架Scrapy的小伙伴可能也知道它的中间件也是如此,这是值得我们学习的地方。