Volley -- 网络请求源码分析

简介

本篇文章分析Volley的网络请求的过程，以及获取缓存数据时是如何判断缓存是否过期，是否需要刷新。

RequestQueue再分析

从之前的文章Volley – 基本用法 中知道，每一个请求都添加到RequestQueue中，有其分配管理，那么它是怎么管理的呢？？
查看其成员变量可以发现其有4个集合对象，现在先来看看分别是什么

 /** * Staging area for requests that already have a duplicate request in flight. * * <ul> * <li>containsKey(cacheKey) indicates that there is a request in flight for the given cache * key.</li> * <li>get(cacheKey) returns waiting requests for the given cache key. The in flight request * is <em>not</em> contained in that list. Is null if no requests are staged.</li> * </ul> */
     // #由注释可知，该集合用于保存正在执行中的请求，并且处理重复添加的请求，等到第一个请求执行完毕，在ExecutorDelivery中传递到主线程接口并调用mRequest.finish("done");释放对象。
    private final Map<String, Queue<Request<?>>> mWaitingRequests =
            new HashMap<String, Queue<Request<?>>>();

    /** * The set of all requests currently being processed by this RequestQueue. A Request * will be in this set if it is waiting in any queue or currently being processed by * any dispatcher. */
     // #该集合用于保存最近添加的请求，在cancelAll()方法中释放对象，因此应该在Activity的onDestroy()方法中调用释放对象。
    private final Set<Request<?>> mCurrentRequests = new HashSet<Request<?>>();

    /** The cache triage queue. */
    // #该集合存储等待从缓存中获取数据的请求
    private final PriorityBlockingQueue<Request<?>> mCacheQueue =
        new PriorityBlockingQueue<Request<?>>();

    /** The queue of requests that are actually going out to the network. */
    // #该集合存储等待从服务器获取数据的请求
    private final PriorityBlockingQueue<Request<?>> mNetworkQueue =
        new PriorityBlockingQueue<Request<?>>();

回到原来问题，RequestQueue是如何管理Request对象，查看add(Request request)代码：

/** * Adds a Request to the dispatch queue. * @param request The request to service * @return The passed-in request */
    public <T> Request<T> add(Request<T> request) {
        // Tag the request as belonging to this queue and add it to the set of current requests.
        request.setRequestQueue(this);
    // #先添加到最近请求的集合中
        synchronized (mCurrentRequests) {
            mCurrentRequests.add(request);
        }

        // Process requests in the order they are added.
        request.setSequence(getSequenceNumber());
        request.addMarker("add-to-queue");

        // If the request is uncacheable, skip the cache queue and go straight to the network.
        // #判断该请求是否需要缓存，默认为true，如果不需要则添加到等待从服务器获取数据的请求的集合中
        if (!request.shouldCache()) {
            mNetworkQueue.add(request);
            return request;
        }

        // Insert request into stage if there's already a request with the same cache key in flight.
        synchronized (mWaitingRequests) {
            String cacheKey = request.getCacheKey();
            // #如果mWaitingRequests包含该请求的cacheKey，则说明，该请求正在加载中，并且将该请求保存起来
            if (mWaitingRequests.containsKey(cacheKey)) {
                // There is already a request in flight. Queue up.
                Queue<Request<?>> stagedRequests = mWaitingRequests.get(cacheKey);
                if (stagedRequests == null) {
                    stagedRequests = new LinkedList<Request<?>>();
                }
                // #保存该请求
                stagedRequests.add(request);
                mWaitingRequests.put(cacheKey, stagedRequests);
                if (VolleyLog.DEBUG) {
                    VolleyLog.v("Request for cacheKey=%s is in flight, putting on hold.", cacheKey);
                }
            } else {
                // Insert 'null' queue for this cacheKey, indicating there is now a request in
                // flight.
                //# 该请求没有正在加载中，则添加null标记为正在加载中
                mWaitingRequests.put(cacheKey, null);
                // #添加到mCacheQueue集合中，由CacheDispatecher处理，如果没有缓存再添加到mNetworkQueue中等待NetworkDispatecher处理。
                mCacheQueue.add(request);
            }
            return request;
        }
    }

从上面可以看出：

• mWaitingRequests：用于处理重复请求，并且将重复请求保存起来，等到第一个请求执行完毕后，在后调到结果接口时通过。Request..finish(“done”)方法调用所在RequestQueue的finish方法释放请求对象。
• mCurrentRequests：用于保存最近添加的请求，在Acitivty的onStop()和onDestroy()方法中可通过调用RequestQueue的cancelAll()方法释放请求对象。
• mCacheQueue：用于保存待处理请求，等待从缓存中获取缓存，如果没有缓存或者缓存过期，则添加到mNetworkQueue中。
• mNetworkQueue：用于存储等待从服务器获取数据的请求。

CacheDispatcher再讨论

从 Volley – 源码分析 这篇文章的问题三中可以知道，CacheDispatcher在处理请求时会发生不存在缓存文件和缓存文件过期以及缓存文件是否需要刷新的情况，那么缓存文件是根据什么来判断以上这些请求的发生呢？

通过Cache接口的内部类Entry的isExpired()方法判断缓存是否过期，refreshNeeded()方法判断缓存是否需要刷新
追溯到Entry类

    /** * Data and metadata for an entry returned by the cache. */
    public static class Entry {
        /** The data returned from cache. */
        // #所请求的数据
        public byte[] data;

        /** ETag for cache coherency. */
        // # Http 响应字段：用于判断所请求数据的一致性
        public String etag;

        /** Date of this response as reported by the server. */
        // #向服务器发送请求的时间
        public long serverDate;

        /** The last modified date for the requested object. */
        // #请求对象上一次修改的时间
        public long lastModified;

        /** TTL for this record. */
        // #该缓存存活的时间
        public long ttl;

        /** Soft TTL for this record. */
        public long softTtl;

        /** Immutable response headers as received from server; must be non-null. */
        public Map<String, String> responseHeaders = Collections.emptyMap();

        /** True if the entry is expired. */
        // # 判断改缓存是否过期
        public boolean isExpired() {
            return this.ttl < System.currentTimeMillis();
        }

        /** True if a refresh is needed from the original data source. */
        // # 判断该缓存是否需要刷新
        public boolean refreshNeeded() {
            return this.softTtl < System.currentTimeMillis();
        }
    }

那么Entry类的这些成员变量从何而来？？让我们看一下NetworkDispatcher先

NetworkDispatcher再讨论

从 Volley – 源码分析 这篇文章的问题二中可以知道，从Request到Response的大致过程，接下来就详细的分析一下。先看一下Network的类图

这里分析一下BasicNetwork是怎么得到NetworkResponse的。代码比较多，只截取performRequest(Request

@Override
    public NetworkResponse performRequest(Request<?> request) throws VolleyError {
        long requestStart = SystemClock.elapsedRealtime();
        while (true) {
            HttpResponse httpResponse = null;
            byte[] responseContents = null;
            Map<String, String> responseHeaders = Collections.emptyMap();
            try {
                // Gather headers.
                Map<String, String> headers = new HashMap<String, String>();
                // #添加缓存信息到请求头 -- HTTP报头 ，后面再讲解
                addCacheHeaders(headers, request.getCacheEntry());
                httpResponse = mHttpStack.performRequest(request, headers);
                StatusLine statusLine = httpResponse.getStatusLine();
                int statusCode = statusLine.getStatusCode();
// # 获取响应头信息
                responseHeaders = convertHeaders(httpResponse.getAllHeaders());
                // Handle cache validation.
                // # 结果码为304 -- 表示资源已经找到，但是未满足条件请求
                if (statusCode == HttpStatus.SC_NOT_MODIFIED) {

                    Entry entry = request.getCacheEntry();
                    if (entry == null) {
                        return new NetworkResponse(HttpStatus.SC_NOT_MODIFIED, null,
                                responseHeaders, true,
                                SystemClock.elapsedRealtime() - requestStart);
                    }

                    // A HTTP 304 response does not have all header fields. We
                    // have to use the header fields from the cache entry plus
                    // the new ones from the response.
                    // http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5
                    entry.responseHeaders.putAll(responseHeaders);
                    return new NetworkResponse(HttpStatus.SC_NOT_MODIFIED, entry.data,
                            entry.responseHeaders, true,
                            SystemClock.elapsedRealtime() - requestStart);
                }

                // Handle moved resources
                // #结果码为301 -- 永久性重定向，表示资源的URL已经更新，这时根据Location首部字段获取新的URL
                if (statusCode == HttpStatus.SC_MOVED_PERMANENTLY || statusCode == HttpStatus.SC_MOVED_TEMPORARILY) {
                    String newUrl = responseHeaders.get("Location");
                    request.setRedirectUrl(newUrl);
                }

                // Some responses such as 204s do not have content. We must check.
                if (httpResponse.getEntity() != null) {
                  responseContents = entityToBytes(httpResponse.getEntity());
                } else {
                  // Add 0 byte response as a way of honestly representing a
                  // no-content request.
                  responseContents = new byte[0];
                }

                // if the request is slow, log it.
                long requestLifetime = SystemClock.elapsedRealtime() - requestStar

                if (statusCode < 200 || statusCode > 299) {
                    throw new IOException();
                }
                return new NetworkResponse(statusCode, responseContents, responseHeaders, false,
                        SystemClock.elapsedRealtime() - requestStart);
            }
            // # catch部分
            ....
    }

private void addCacheHeaders(Map<String, String> headers, Cache.Entry entry) {
        // If there's no cache entry, we're done.
        if (entry == null) {
            return;
        }

        // # 这里涉及到请求首部字段"If-None-Match"，
        // # 在解析之前，要知道响应首部字段"ETag"的作用：用于告知客户端实体标识，服务器会为每份资源分配对应的ETag值，另外，当资源更新时，ETag也需要更新.
        // # 而请求首部字段"If-None-Match"的作用是：用于指定If-None-Match字段值的实体标记(ETag)不一致时，就告知服务器处理该请求。在GET和HEAD方法中使用首部字段If-None-Match可获取最新资源
        if (entry.etag != null) {
            headers.put("If-None-Match", entry.etag);
        }

        // # 如果在If-Modified-Since字段指定的时间之后，资源发生了更新，服务器会接受请求。
        // # 如果在If-Modified-Since字段指定的时间之后，如果请求的资源都没有更新过，则返回状态码304 Not Modified的响应
        if (entry.lastModified > 0) {
            // # 获取entry上一次修改的时间
            Date refTime = new Date(entry.lastModified);
            headers.put("If-Modified-Since", DateUtils.formatDate(refTime));
        }
    }

通过上面代码可以知道，通过设置添加If-Modified-Since和If-None-Match的请求头字段来获取NetworkResponse，现在来分析一下NetworkResponse的成员变量：

/** * Data and headers returned from {@link Network#performRequest(Request)}. */
public class NetworkResponse implements Serializable{

    // # 构造方法...

    /** The HTTP status code. */
    // # Http结果码
    public final int statusCode;

    /** Raw data from this response. */
    // # 结果数据
    public final byte[] data;

    /** Response headers. */
    // # 响应头
    public final Map<String, String> headers;

    /** True if the server returned a 304 (Not Modified). */
    // # 是否返回304的结果码
    public final boolean notModified;

    /** Network roundtrip time in milliseconds. */
    // # 从服务器获取数据花费的时间
    public final long networkTimeMs;
}

接下来分析NetworkDispatcher处理Request的主要流程：

@Override
    public void run() {
        Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);
        Request<?> request;
        while (true) {
            long startTimeMs = SystemClock.elapsedRealtime();
           // # 先获取request对象

            try {
                request.addMarker("network-queue-take");

                // If the request was cancelled already, do not perform the
                // network request.
                // # 判断请求是否取消
                if (request.isCanceled()) {
                    request.finish("network-discard-cancelled");
                    continue;
                }

                addTrafficStatsTag(request);

                // Perform the network request.
                // # 执行获取NetworkResponse结果
                NetworkResponse networkResponse = mNetwork.performRequest(request);
                request.addMarker("network-http-complete");

                // If the server returned 304 AND we delivered a response already,
                // we're done -- don't deliver a second identical response.
                // # 判断是否为304结果并且结果已经传送完成
                if (networkResponse.notModified && request.hasHadResponseDelivered()) {
                    request.finish("not-modified");
                    continue;
                }

                // Parse the response here on the worker thread.
                // # 获取Response对象
                Response<?> response = request.parseNetworkResponse(networkResponse);
                request.addMarker("network-parse-complete");

                // Write to cache if applicable.
                // TODO: Only update cache metadata instead of entire record for 304s.
                // # 判断是否需要缓存
                if (request.shouldCache() && response.cacheEntry != null) {
                    mCache.put(request.getCacheKey(), response.cacheEntry);
                    request.addMarker("network-cache-written");
                }

                // Post the response back.
                // # 传递结果
                request.markDelivered();
                mDelivery.postResponse(request, response);
            } catch (VolleyError volleyError) {
                volleyError.setNetworkTimeMs(SystemClock.elapsedRealtime() - startTimeMs);
                parseAndDeliverNetworkError(request, volleyError);
            } catch (Exception e) {
                // ...
            }
        }
    }

看到这里你会发现，结果都已经被传送了，却没有发现关于Entry类中的存活时间等数据。其实不然，看一下StringRequest的parseNetworkResponse(NetworkResponse response)方法：

@Override
    protected Response<String> parseNetworkResponse(NetworkResponse response) {
        String parsed;
        try {
            parsed = new String(response.data, HttpHeaderParser.parseCharset(response.headers));
        } catch (UnsupportedEncodingException e) {
            parsed = new String(response.data);
        }
        // # 通过HttpHeaderParser将NetworkResponse 结果解析成Entry对象
        return Response.success(parsed, HttpHeaderParser.parseCacheHeaders(response));
    }

看一下parseCacheHeaders(NetworkResponse response) 方法：

/** * Extracts a {@link Cache.Entry} from a {@link NetworkResponse}. * * @param response The network response to parse headers from * @return a cache entry for the given response, or null if the response is not cacheable. */
    public static Cache.Entry parseCacheHeaders(NetworkResponse response) {
        long now = System.currentTimeMillis();

        Map<String, String> headers = response.headers;

        // ...

        // # 首部字段Date -- 表明创建HTTP报文日期和时间 
        // # HTTP/1.1协议使用RFC1123中规定的时间格式，具体解析看parseDateAsEpoch()方法
        headerValue = headers.get("Date");
        if (headerValue != null) {
            serverDate = parseDateAsEpoch(headerValue);
        }

        // # 首部字段Cache-Control
        headerValue = headers.get("Cache-Control");
        if (headerValue != null) {
            hasCacheControl = true;
            String[] tokens = headerValue.split(",");
            for (int i = 0; i < tokens.length; i++) {
                String token = tokens[i].trim();
                // # no-cache -- 缓存前必须先确定其有效性
                // # no-store -- 不缓存请求或响应的任何内容
                // # 在这里表示无缓存
                if (token.equals("no-cache") || token.equals("no-store")) {
                    return null;
                } else if (token.startsWith("max-age=")) {
                    // # max-age -- 资源文件的存活时间
                    try {
                        maxAge = Long.parseLong(token.substring(8));
                    } catch (Exception e) {
                    }
                } else if (token.startsWith("stale-while-revalidate=")) {
                   // # stale-while-revalidate : 这个指令的解释相对比较少，根据一篇论文的介绍，大概可能解释为缓存超过存活时间，重新请求或者资源更新过程的延迟时间（论文地址：http://tools.ietf.org/html/draft-nottingham-http-stale-while-revalidate-01#section-1）
                    try {
                        staleWhileRevalidate = Long.parseLong(token.substring(23));
                    } catch (Exception e) {
                    }
                } else if (token.equals("must-revalidate") || token.equals("proxy-revalidate")) {
                // # must-revalidate -- 表示可缓存但必须再向源服务器进行确认
                // # proxy-revalidate -- 表示中间缓存服务器对缓存响应的有效性再进行确认
                    mustRevalidate = true;
                }
            }
        }

        // # Expires -- 资源失效日期
        // # 当Cache-Control有指令max-age时，优先处理max-age指令
        headerValue = headers.get("Expires");
        if (headerValue != null) {
            serverExpires = parseDateAsEpoch(headerValue);
        }

        // # Last-Modified -- 资源最终修改的时间
        headerValue = headers.get("Last-Modified");
        if (headerValue != null) {
            lastModified = parseDateAsEpoch(headerValue);
        }
        // # ETag -- 资源标识
        serverEtag = headers.get("ETag");

        // Cache-Control takes precedence over an Expires header, even if both exist and Expires
        // is more restrictive.
        if (hasCacheControl) {
            softExpire = now + maxAge * 1000;
            finalExpire = mustRevalidate
                    ? softExpire
                    : softExpire + staleWhileRevalidate * 1000;
        } else if (serverDate > 0 && serverExpires >= serverDate) {
            // Default semantic for Expire header in HTTP specification is softExpire.
            softExpire = now + (serverExpires - serverDate);
            finalExpire = softExpire;
        }

        Cache.Entry entry = new Cache.Entry();
        // ...

        return entry;
    }

到了这里，才发现Entry的相关信息是在这里被赋值初始化。

总结

• 知道RequestQueue是如何管理Request
• 知道缓存数据Entry是如何判断是否过期是否需要更新
• 知道HTTP的部分首部字段的使用，有
  
  • ETag
  • If-None-Match
  • If-Modified-Since
  • Cache-control
  • Data
  • Expires
  • Location

最后，如果有哪个地方不足或者错误，希望能够指出