Android视频点播-边播边缓存-方案
简述
一些知名的视频app客户端（优酷，爱奇艺）播放视频的时候都有一些缓存进度(二级进度缓存)，qq，微信有关的小视频，还有一些短视频app，都有边播边缓的处理。还有就是当文件缓存完毕了再次播放的话就不再请求网络了直接播放本地文件了。既节省了流程又提高了加载速度。
今天我们就是来研究讨论实现这个边播边缓存的框架，因为它不和任何的业务逻辑耦合。
开源的项目
目前比较好的开源项目是：
https://github.com/danikula/AndroidVideoCache
代码的架构写的也很不错，网络用的httpurlconnect，文件缓存处理，文件最大限度策略，回调监听处理，断点续传,代理服务等。很值得研究阅读.
个人觉得项目中有几个需要优化的点，今天就来处理这几个并简要分析下原理
优化点比如：
- 文件的缓存超过限制后没有按照lru算法删除，
- 处理返回给播放器的http响应头消息，响应头消息的获取处理改为head请求（需服务器支持）
- 替换网络库为okhttp（因为大部分的项目都是以okhttp为网络请求库的）
该开源项目的原理分析-本地代理
原始的方式是直接塞播放地址给播放器，它就可以直接播放。现在我们要在中间加一层本地代理，播放器播放的时候（获取数据）是通过我们的本地代理的地址来播放的，这样我们就可以很好的在中间层（本地代理层）做一些处理，比如：文件缓存，预缓存（秒开处理），监控等。
视频点播框架图.jpg
- 采用了本地代理服务的方式，通过原始url给播放器返回一个本地代理的一个url ，代理URL类似：http://127.0.0.1:57430/真实url；（57430端口为系统随机分配的有效端口，真实url是为了真正的下载，当然也可以是一定的加密规则），然后播放器播放的时候请求到了你本地的代理上了。
- 读取客户端就是socket来读取数据（http协议请求）解析http协议。
- 根据url检查视频文件是否存在，读取文件数据给播放器，也就是往socket里写入数据（socket通信）。同时如果没有下载完成会进行断点下载,当然弱网的话数据需要生产消费同步处理。
优化点
1. 文件的缓存超过限制后没有按照lru算法删除.
Files类。
由于在移动设备上file.setLastModified（） 方法不支持毫秒级的时间处理，导致超出限制大小后本应该删除老的，却没有删除抛出了异常。注释掉主动抛出的异常即可。因为文件的修改时间就是对的。
static void setLastModifiedNow(File file) throws IOException {
if (file.exists()) {
long now = System.currentTimeMillis();
boolean modified = file.setLastModified(now/1000*1000); // on some devices (e.g. Nexus 5) doesn't work
if (!modified) {
modify(file);
// if (file.lastModified() < now) {
// VideoCacheLog.debug("LruDiskUsage", "modified not ok ");
// throw new IOException("Error set last modified date to " + file);
// }else{
// VideoCacheLog.debug("LruDiskUsage", "modified ok ");
// }
}
}
}
2. 处理返回给播放器的http响应头消息，响应头消息的获取处理改为head请求(需要服务器支持)
HttpUrlSource类。fetchContentInfo方法是获取视频文件的Content-Type，Content-Length信息，是为了播放器播放的时候给播放器组装http响应头信息用的。所以这一块需要用数据库保存，这样播放器每次播放的时候不要在此获取了，减少了请求的次数，节省了流量。既然是只需要头信息，不需要响应体，所以我们在获取的时候可以直接采用HEAD方法。所以代码增加了一个方法openConnectionForHeader如下：
private void fetchContentInfo() throws ProxyCacheException {
VideoCacheLog.debug(TAG,"Read content info from " + sourceInfo.url);
HttpURLConnection urlConnection = null;
InputStream inputStream = null;
try {
urlConnection = openConnectionForHeader(20000);
long length = getContentLength(urlConnection);
String mime = urlConnection.getContentType();
inputStream = urlConnection.getInputStream();
this.sourceInfo = new SourceInfo(sourceInfo.url, length, mime);
this.sourceInfoStorage.put(sourceInfo.url, sourceInfo);
VideoCacheLog.debug(TAG,"Source info fetched: " + sourceInfo);
} catch (IOException e) {
VideoCacheLog.error(TAG,"Error fetching info from " + sourceInfo.url ,e);
} finally {
ProxyCacheUtils.close(inputStream);
if (urlConnection != null) {
urlConnection.disconnect();
}
}
}
// for HEAD
private HttpURLConnection openConnectionForHeader(int timeout) throws IOException, ProxyCacheException {
HttpURLConnection connection;
boolean redirected;
int redirectCount = 0;
String url = this.sourceInfo.url;
do {
VideoCacheLog.debug(TAG, "Open connection for header to " + url);
connection = (HttpURLConnection) new URL(url).openConnection();
if (timeout > 0) {
connection.setConnectTimeout(timeout);
connection.setReadTimeout(timeout);
}
//只返回头部，不需要BODY，既可以提高响应速度也可以减少网络流量
connection.setRequestMethod("HEAD");
int code = connection.getResponseCode();
redirected = code == HTTP_MOVED_PERM || code == HTTP_MOVED_TEMP || code == HTTP_SEE_OTHER;
if (redirected) {
url = connection.getHeaderField("Location");
VideoCacheLog.debug(TAG,"Redirect to:" + url);
redirectCount++;
connection.disconnect();
VideoCacheLog.debug(TAG,"Redirect closed:" + url);
}
if (redirectCount > MAX_REDIRECTS) {
throw new ProxyCacheException("Too many redirects: " + redirectCount);
}
} while (redirected);
return connection;
}
3.替换网络库为okhttp（因为大部分的项目都是以okhttp为网络请求库的）
为什么我们要换呢？！一是OKHttp是一款高效的HTTP客户端，支持连接同一地址的链接共享同一个socket，通过连接池来减小响应延迟，
还有透明的GZIP压缩，请求缓存等优势，其核心主要有路由、连接协议、拦截器、代理、安全性认证、连接池以及网络适配，拦截器主要是指添加，移除或者转换请求或者回应的头部信息。得到了android开发的认可。二是大部分的app都是采用OKHttp，而且google会将其纳入android 源码中。
三是该作者代码中用的httpurlconnet在HttpUrlSource有这么一段：
@Override
public void close() throws ProxyCacheException {
if (connection != null) {
try {
connection.disconnect();
} catch (NullPointerException | IllegalArgumentException e) {
String message = "Wait... but why? WTF!? " +
"Really shouldn't happen any more after fixing https://github.com/danikula/AndroidVideoCache/issues/43. " +
"If you read it on your device log, please, notify me [email protected] or create issue here " +
"https://github.com/danikula/AndroidVideoCache/issues.";
throw new RuntimeException(message, e);
} catch (ArrayIndexOutOfBoundsException e) {
VideoCacheLog.error(TAG,"Error closing connection correctly. Should happen only on Android L. " +
"If anybody know how to fix it, please visit https://github.com/danikula/AndroidVideoCache/issues/88. " +
"Until good solution is not know, just ignore this issue :(", e);
}
}
}
在没有像okhttp这些优秀的网络开源项目之前，android开发都是采用httpurlconnet或者httpclient，部分手机可能会遇到这个问题哈。
这里采用的 compile 'com.squareup.okhttp:okhttp:2.7.5' 版本的来实现该类的功能。在原作者的架构思路上我们只需要增加实现Source接口的类OkHttpUrlSource即可，可见作者的代码架构还是不错的，当然我们同样需要处理上文中提高的优化点2中的问题。将项目中所有用到HttpUrlSource的地方改为OkHttpUrlSource即可。
源码如下：
/**
* ================================================
* 作 者：顾修忠[email protected]/[email protected]
* 版 本：
* 创建日期：2017/4/13-上午12:03
* 描 述：在一些Android手机上HttpURLConnection.disconnect()方法仍然耗时太久，
* 进行导致MediaPlayer要等待很久才会开始播放，因此决定使用okhttp替换HttpURLConnection
* 修订历史：
* ================================================
*/
public class OkHttpUrlSource implements Source {
private static final String TAG = OkHttpUrlSource.class.getSimpleName();
private static final int MAX_REDIRECTS = 5;
private final SourceInfoStorage sourceInfoStorage;
private SourceInfo sourceInfo;
private OkHttpClient okHttpClient = new OkHttpClient();
private Call requestCall = null;
private InputStream inputStream;
public OkHttpUrlSource(String url) {
this(url, SourceInfoStorageFactory.newEmptySourceInfoStorage());
}
public OkHttpUrlSource(String url, SourceInfoStorage sourceInfoStorage) {
this.sourceInfoStorage = checkNotNull(sourceInfoStorage);
SourceInfo sourceInfo = sourceInfoStorage.get(url);
this.sourceInfo = sourceInfo != null ? sourceInfo :
new SourceInfo(url, Integer.MIN_VALUE, ProxyCacheUtils.getSupposablyMime(url));
}
public OkHttpUrlSource(OkHttpUrlSource source) {
this.sourceInfo = source.sourceInfo;
this.sourceInfoStorage = source.sourceInfoStorage;
}
@Override
public synchronized long length() throws ProxyCacheException {
if (sourceInfo.length == Integer.MIN_VALUE) {
fetchContentInfo();
}
return sourceInfo.length;
}
@Override
public void open(long offset) throws ProxyCacheException {
try {
Response response = openConnection(offset, -1);
String mime = response.header("Content-Type");
this.inputStream = new BufferedInputStream(response.body().byteStream(), DEFAULT_BUFFER_SIZE);
long length = readSourceAvailableBytes(response, offset, response.code());
this.sourceInfo = new SourceInfo(sourceInfo.url, length, mime);
this.sourceInfoStorage.put(sourceInfo.url, sourceInfo);
} catch (IOException e) {
throw new ProxyCacheException("Error opening okHttpClient for " + sourceInfo.url + " with offset " + offset, e);
}
}
private long readSourceAvailableBytes(Response response, long offset, int responseCode) throws IOException {
long contentLength = getContentLength(response);
return responseCode == HTTP_OK ? contentLength
: responseCode == HTTP_PARTIAL ? contentLength + offset : sourceInfo.length;
}
private long getContentLength(Response response) {
String contentLengthValue = response.header("Content-Length");
return contentLengthValue == null ? -1 : Long.parseLong(contentLengthValue);
}
@Override
public void close() throws ProxyCacheException {
if (okHttpClient != null && inputStream != null && requestCall != null) {
try {
inputStream.close();
requestCall.cancel();
} catch (IOException e) {
e.printStackTrace();
throw new RuntimeException(e.getMessage(), e);
}
}
}
@Override
public int read(byte[] buffer) throws ProxyCacheException {
if (inputStream == null) {
throw new ProxyCacheException("Error reading data from " + sourceInfo.url + ": okHttpClient is absent!");
}
try {
return inputStream.read(buffer, 0, buffer.length);
} catch (InterruptedIOException e) {
throw new InterruptedProxyCacheException("Reading source " + sourceInfo.url + " is interrupted", e);
} catch (IOException e) {
throw new ProxyCacheException("Error reading data from " + sourceInfo.url, e);
}
}
private void fetchContentInfo() throws ProxyCacheException {
VideoCacheLog.debug(TAG, "Read content info from " + sourceInfo.url);
Response response = null;
InputStream inputStream = null;
try {
response = openConnectionForHeader(20000);
if (response == null || !response.isSuccessful()) {
throw new ProxyCacheException("Fail to fetchContentInfo: " + sourceInfo.url);
}
long length = getContentLength(response);
String mime = response.header("Content-Type", "application/mp4");
inputStream = response.body().byteStream();
this.sourceInfo = new SourceInfo(sourceInfo.url, length, mime);
this.sourceInfoStorage.put(sourceInfo.url, sourceInfo);
VideoCacheLog.info(TAG, "Content info for `" + sourceInfo.url + "`: mime: " + mime + ", content-length: " + length);
} catch (IOException e) {
VideoCacheLog.error(TAG, "Error fetching info from " + sourceInfo.url, e);
} finally {
ProxyCacheUtils.close(inputStream);
if (response != null && requestCall != null) {
requestCall.cancel();
}
}
}
// for HEAD
private Response openConnectionForHeader(int timeout) throws IOException, ProxyCacheException {
if (timeout > 0) {
// okHttpClient.setConnectTimeout(timeout, TimeUnit.MILLISECONDS);
// okHttpClient.setReadTimeout(timeout, TimeUnit.MILLISECONDS);
// okHttpClient.setWriteTimeout(timeout, TimeUnit.MILLISECONDS);
}
Response response;
boolean isRedirect = false;
String newUrl = this.sourceInfo.url;
int redirectCount = 0;
do {
//只返回头部，不需要BODY，既可以提高响应速度也可以减少网络流量
Request request = new Request.Builder()
.head()
.url(newUrl)
.build();
requestCall = okHttpClient.newCall(request);
response = requestCall.execute();
if (response.isRedirect()) {
newUrl = response.header("Location");
VideoCacheLog.debug(TAG, "Redirect to:" + newUrl);
isRedirect = response.isRedirect();
redirectCount++;
requestCall.cancel();
VideoCacheLog.debug(TAG, "Redirect closed:" + newUrl);
}
if (redirectCount > MAX_REDIRECTS) {
throw new ProxyCacheException("Too many redirects: " + redirectCount);
}
} while (isRedirect);
return response;
}
private Response openConnection(long offset, int timeout) throws IOException, ProxyCacheException {
if (timeout > 0) {
// okHttpClient.setConnectTimeout(timeout, TimeUnit.MILLISECONDS);
// okHttpClient.setReadTimeout(timeout, TimeUnit.MILLISECONDS);
// okHttpClient.setWriteTimeout(timeout, TimeUnit.MILLISECONDS);
}
Response response;
boolean isRedirect = false;
String newUrl = this.sourceInfo.url;
int redirectCount = 0;
do {
VideoCacheLog.debug(TAG, "Open connection" + (offset > 0 ? " with offset " + offset : "") + " to " + sourceInfo.url);
Request.Builder requestBuilder = new Request.Builder()
.get()
.url(newUrl);
if (offset > 0) {
requestBuilder.addHeader("Range", "bytes=" + offset + "-");
}
requestCall = okHttpClient.newCall(requestBuilder.build());
response = requestCall.execute();
if (response.isRedirect()) {
newUrl = response.header("Location");
isRedirect = response.isRedirect();
redirectCount++;
}
if (redirectCount > MAX_REDIRECTS) {
throw new ProxyCacheException("Too many redirects: " + redirectCount);
}
} while (isRedirect);
return response;
}
public synchronized String getMime() throws ProxyCacheException {
if (TextUtils.isEmpty(sourceInfo.mime)) {
fetchContentInfo();
}
return sourceInfo.mime;
}
public String getUrl() {
return sourceInfo.url;
}
@Override
public String toString() {
return "OkHttpUrlSource{sourceInfo='" + sourceInfo + "}";
}
}