下文中所用到的依赖:implementation 'com.squareup.retrofit2:retrofit:2.1.0',虽然用的retrofit的包,但是只用了里面的okhttp部分,因为是demo项目,原先就已经加了依赖了,不方便换了。
现在我们开始多线程分段下载一个大文件,用的是服务上一个apk文件(23M,假设是大文件)
开工,ps:网络权限,文件读写权限,别忘了
1. 既然是多线程下载,先建立一个线程池
ExecutorService executorService = Executors.newFixedThreadPool(5);
2. 确认下载文件的大小
OkHttpClient client = createOkHttpClient();
final Request request = new Request.Builder()
.url(url)
.head()//这里注意请求方式为head
.build();
try {
Response response = client.newCall(request).execute();
httpResponseCall.onResponseSuccess(response);
} catch (IOException e) {
e.printStackTrace();
}
因为本身是线程调用的,这里网络请求就用同步了,request请求方式为head,只要获取文件大小就好了,然后在请求返回的head中获取文件大小
long length = Long.parseLong(response.header("content-length"));//获取文件长度
long count = length / M + 1;//下载次数 M = 1024 * 1024 * 5,M为一次下载多少长度,这里为5mb
for (int i = 0; i < count; i++) {
long start, end;//每次一次下载的初始位置和结束位置
if (i == count - 1) {
end = length - 1;//因为从0开始的,所以要-1,比如10个字节,就是0-9
} else {
end = (i + 1) * M - 1;
}
start = i * M;
getFile(start, end);//开启下载线程
}
3. 文件内容下载
OkHttpClient client = createOkHttpClient();
final Request request = new Request.Builder()
//由于只下载文件部分内容,所以如要添加head 格式为 Range,bytes=0-199
.addHeader("Range", String.format("bytes=%d-%d", start, end)
.url(url)
.build();
try {
Response response = client.newCall(request).execute();
httpResponseCall.onResponseSuccess(response);
} catch (IOException e) {
e.printStackTrace();
}
4. 下载之后处理下载数据
InputStream inputStream = response.body().byteStream();//获取流
FileUtil.appendFileWithInstram(FILE_NAME, inputStream, start);//写入文件
response.close();
response.notify();
5. 读取数据写入文件
public static void appendFileWithInstram(String fileName, InputStream inputStream, long start) {
File file = new File(fileName);//获取文件,我在下载之前就删除了原文件
synchronized (TAG) {//设置线程锁
try {
while (true) {//通过线程锁,对线程进行排序
long fileLength = PreferenceUtil.getLong(fileName, 0l);//记录文件长度
JLog.d("FileUtil", "fileLength = " + fileLength +"\n start = "+ start);
if (fileLength == start) {//当文件长度,与下载文件的初始值一样,则开始写入文件,否则等待
break;
}
TAG.wait();
}
} catch (InterruptedException e) {
e.printStackTrace();
}
RandomAccessFile randomAccessFile = null;
try {
if (!file.exists()) {
file.createNewFile();
}
randomAccessFile = new RandomAccessFile(file, "rw");
final long M = 1024;
long total = PreferenceUtil.getLong(fileName, 0l);
for (; ; ) {//循环读取写入文件
byte[] bytes = new byte[(int) M];
int readCount = inputStream.read(bytes);
if (readCount == 0){//因为请求是一部分,一部分的,不是一次性的,所以会存在0的情况,注意0!=-1
continue;
}
if (readCount == -1) {//读取完成,结束循环
JLog.d(TAG, "readCount = " + readCount + "\n start = " + start +"\ntotal = " + total);
PreferenceUtil.setLongValue(fileName, total);//将写入的总长度记录下来
inputStream.close();
randomAccessFile.close();
break;
}
randomAccessFile.seek(total);//设置偏移量
total += readCount;
//写入读取的总长度,不能只写randomAccessFile.write(bytes)会导致文件长度大于读取的长度
randomAccessFile.write(bytes, 0, readCount);
JLog.d(TAG, "length = " + randomAccessFile.length());
JLog.d(TAG, "total = " + total);
}
TAG.notifyAll();//结束循环之后,唤醒其他线程进行写入
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
6. 过程中遇到的问题
02-22 13:37:43.007 7378-7411/com.nmssdmf.testmodule W/System.err: java.net.ProtocolException: unexpected end of stream
02-22 13:37:43.007 7378-7411/com.nmssdmf.testmodule W/System.err: at okhttp3.internal.http.Http1xStream$FixedLengthSource.read(Http1xStream.java:384)
02-22 13:37:43.007 7378-7411/com.nmssdmf.testmodule W/System.err: at okio.RealBufferedSource$1.read(RealBufferedSource.java:386)
02-22 13:37:43.007 7378-7411/com.nmssdmf.testmodule W/System.err: at java.io.InputStream.read(InputStream.java:101)
感觉这个bug是okHttp的,一开始我用直接用response.body.bytes的方式读取,没有遇到这个问题,因为直接获取流的byte数组,所需要的内存占用比较大,后改成从流中获取的方式,但是不进行线程阻塞,则会在最后一个线程读取中出现这个问题,经过各种资料查找,在OkHttpClient.Builder
builder.connectTimeout(60, TimeUnit.SECONDS);
builder.readTimeout(60, TimeUnit.SECONDS);
builder.writeTimeout(60, TimeUnit.SECONDS);
大概是因为响应时间的问题,该问题得到解决,再后来把读写文件改成现在一样线程阻塞的方法,在公司网络下,依旧出现问题(必现),在4g热点下,没有该问题。目前还没有比较好的解决方法。
7. 读写文件可以用下面的方法,更方便
/**
* 文件分批写入
*
* @param fileName 文件路径
* @param inputStream 需要写入的内容
* @param start 写入的初始位置
*/
public synchronized static void appendFileWithInstram(String fileName, InputStream inputStream, long start) {
File file = new File(fileName);
RandomAccessFile randomAccessFile = null;
try {
if (!file.exists()) {
file.createNewFile();
}
randomAccessFile = new RandomAccessFile(file, "rw");
final long M = 1024;
byte[] bytes = new byte[(int) M];
long seek = start;
long total = 0;
for (; ; ) {
int readCount = inputStream.read(bytes);
if (readCount == 0) {
continue;
}
if (readCount == -1) {
JLog.d(TAG, "readCount = " + readCount + "\n start = " + start + "\ntotal = " + total);
JLog.d(TAG, "fileLength = " + randomAccessFile.length()+ "\n seek = " + seek );
inputStream.close();
break;
}
total += readCount;
randomAccessFile.seek(seek );
randomAccessFile.write(bytes, 0, readCount);
seek += readCount;
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}