用 Java 实现断点续传 (HTTP)

断点续传的原理

其实断点续传的原理很简单，就是在 Http 的请求上和一般的下载有所不同而已。 
打个比方，浏览器请求服务器上的一个文时，所发出的请求如下： 
假设服务器域名为 wwww.sjtu.edu.cn，文件名为 down.zip。 
GET /down.zip HTTP/1.1 
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms- 
excel, application/msword, application/vnd.ms-powerpoint, */* 
Accept-Language: zh-cn 
Accept-Encoding: gzip, deflate 
User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0) 
Connection: Keep-Alive

服务器收到请求后，按要求寻找请求的文件，提取文件的信息，然后返回给浏览器，返回信息如下：

200 
Content-Length=106786028 
Accept-Ranges=bytes 
Date=Mon, 30 Apr 2001 12:56:11 GMT 
ETag=W/"02ca57e173c11:95b" 
Content-Type=application/octet-stream 
Server=Microsoft-IIS/5.0 
Last-Modified=Mon, 30 Apr 2001 12:56:11 GMT

所谓断点续传，也就是要从文件已经下载的地方开始继续下载。所以在客户端浏览器传给 Web 服务器的时候要多加一条信息 -- 从哪里开始。 
下面是用自己编的一个"浏览器"来传递请求信息给 Web 服务器，要求从 2000070 字节开始。 
GET /down.zip HTTP/1.0 
User-Agent: NetFox 
RANGE: bytes=2000070- 
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2

仔细看一下就会发现多了一行 RANGE: bytes=2000070- 
这一行的意思就是告诉服务器 down.zip 这个文件从 2000070 字节开始传，前面的字节不用传了。 
服务器收到这个请求以后，返回的信息如下： 
206 
Content-Length=106786028 
Content-Range=bytes 2000070-106786027/106786028 
Date=Mon, 30 Apr 2001 12:55:20 GMT 
ETag=W/"02ca57e173c11:95b" 
Content-Type=application/octet-stream 
Server=Microsoft-IIS/5.0 
Last-Modified=Mon, 30 Apr 2001 12:55:20 GMT

和前面服务器返回的信息比较一下，就会发现增加了一行： 
Content-Range=bytes 2000070-106786027/106786028 
返回的代码也改为 206 了，而不再是 200 了。

知道了以上原理，就可以进行断点续传的编程了。

---

Java 实现断点续传的关键几点

1. (1) 用什么方法实现提交 RANGE: bytes=2000070-。 
   当然用最原始的 Socket 是肯定能完成的，不过那样太费事了，其实 Java 的 net 包中提供了这种功能。代码如下： 
   
   URL url = new URL("http://www.sjtu.edu.cn/down.zip"); 
   HttpURLConnection httpConnection = (HttpURLConnection)url.openConnection(); 
   
   // 设置 User-Agent 
   httpConnection.setRequestProperty("User-Agent","NetFox"); 
   // 设置断点续传的开始位置 
   httpConnection.setRequestProperty("RANGE","bytes=2000070"); 
   // 获得输入流 
   InputStream input = httpConnection.getInputStream(); 
   

   从输入流中取出的字节流就是 down.zip 文件从 2000070 开始的字节流。 大家看，其实断点续传用 Java 实现起来还是很简单的吧。 接下来要做的事就是怎么保存获得的流到文件中去了。

2. 保存文件采用的方法。 
   我采用的是 IO 包中的 RandAccessFile 类。 
   操作相当简单，假设从 2000070 处开始保存文件，代码如下： 
   RandomAccess oSavedFile = new RandomAccessFile("down.zip","rw"); 
   long nPos = 2000070; 
   // 定位文件指针到 nPos 位置 
   oSavedFile.seek(nPos); 
   byte[] b = new byte[1024]; 
   int nRead; 
   // 从输入流中读入字节流，然后写到文件中 
   while((nRead=input.read(b,0,1024)) > 0) 
   { 
   oSavedFile.write(b,0,nRead); 
   }

怎么样，也很简单吧。 接下来要做的就是整合成一个完整的程序了。包括一系列的线程控制等等。

---

 

断点续传内核的实现

主要用了 6 个类，包括一个测试类。 
SiteFileFetch.java 负责整个文件的抓取，控制内部线程 (FileSplitterFetch 类 )。 
FileSplitterFetch.java 负责部分文件的抓取。 
FileAccess.java 负责文件的存储。 
SiteInfoBean.java 要抓取的文件的信息，如文件保存的目录，名字，抓取文件的 URL 等。 
Utility.java 工具类，放一些简单的方法。 
TestMethod.java 测试类。

下面是源程序：

  1 /* 

  2  /*

  3  * SiteFileFetch.java 

  4  */ 

  5  package NetFox; 

  6  import java.io.*; 

  7  import java.net.*; 

  8  public class SiteFileFetch extends Thread { 

  9  SiteInfoBean siteInfoBean = null; // 文件信息 Bean 

 10  long[] nStartPos; // 开始位置

 11  long[] nEndPos; // 结束位置

 12  FileSplitterFetch[] fileSplitterFetch; // 子线程对象

 13  long nFileLength; // 文件长度

 14  boolean bFirst = true; // 是否第一次取文件

 15  boolean bStop = false; // 停止标志

 16  File tmpFile; // 文件下载的临时信息

 17  DataOutputStream output; // 输出到文件的输出流

 18  public SiteFileFetch(SiteInfoBean bean) throws IOException 

 19  { 

 20  siteInfoBean = bean; 

 21  //tmpFile = File.createTempFile ("zhong","1111",new File(bean.getSFilePath())); 

 22  tmpFile = new File(bean.getSFilePath()+File.separator + bean.getSFileName()+".info");

 23  if(tmpFile.exists ()) 

 24  { 

 25  bFirst = false; 

 26  read_nPos(); 

 27  } 

 28  else 

 29  { 

 30  nStartPos = new long[bean.getNSplitter()]; 

 31  nEndPos = new long[bean.getNSplitter()]; 

 32  } 

 33  } 

 34  public void run() 

 35  { 

 36  // 获得文件长度

 37  // 分割文件

 38  // 实例 FileSplitterFetch 

 39  // 启动 FileSplitterFetch 线程

 40  // 等待子线程返回

 41  try{ 

 42  if(bFirst) 

 43  { 

 44  nFileLength = getFileSize(); 

 45  if(nFileLength == -1) 

 46  { 

 47  System.err.println("File Length is not known!"); 

 48  } 

 49  else if(nFileLength == -2) 

 50  { 

 51  System.err.println("File is not access!"); 

 52  } 

 53  else 

 54  { 

 55  for(int i=0;i<nStartPos.length;i++) 

 56  { 

 57  nStartPos[i] = (long)(i*(nFileLength/nStartPos.length)); 

 58  } 

 59  for(int i=0;i<nEndPos.length-1;i++) 

 60  { 

 61  nEndPos[i] = nStartPos[i+1]; 

 62  } 

 63  nEndPos[nEndPos.length-1] = nFileLength; 

 64  } 

 65  } 

 66  // 启动子线程

 67  fileSplitterFetch = new FileSplitterFetch[nStartPos.length]; 

 68  for(int i=0;i<nStartPos.length;i++) 

 69  { 

 70  fileSplitterFetch[i] = new FileSplitterFetch(siteInfoBean.getSSiteURL(), 

 71  siteInfoBean.getSFilePath() + File.separator + siteInfoBean.getSFileName(), 

 72  nStartPos[i],nEndPos[i],i); 

 73  Utility.log("Thread " + i + " , nStartPos = " + nStartPos[i] + ", nEndPos = " 

 74  + nEndPos[i]); 

 75  fileSplitterFetch[i].start(); 

 76  } 

 77  // fileSplitterFetch[nPos.length-1] = new FileSplitterFetch(siteInfoBean.getSSiteURL(),

 78  siteInfoBean.getSFilePath() + File.separator 

 79  + siteInfoBean.getSFileName(),nPos[nPos.length-1],nFileLength,nPos.length-1); 

 80  // Utility.log("Thread " +(nPos.length-1) + ",nStartPos = "+nPos[nPos.length-1]+",

 81  nEndPos = " + nFileLength); 

 82  // fileSplitterFetch[nPos.length-1].start(); 

 83  // 等待子线程结束

 84  //int count = 0; 

 85  // 是否结束 while 循环

 86  boolean breakWhile = false; 

 87  while(!bStop) 

 88  { 

 89  write_nPos(); 

 90  Utility.sleep(500); 

 91  breakWhile = true; 

 92  for(int i=0;i<nStartPos.length;i++) 

 93  { 

 94  if(!fileSplitterFetch[i].bDownOver) 

 95  { 

 96  breakWhile = false; 

 97  break; 

 98  } 

 99  } 

100  if(breakWhile) 

101  break; 

102  //count++; 

103  //if(count>4) 

104  // siteStop(); 

105  } 

106  System.err.println("文件下载结束！"); 

107  } 

108  catch(Exception e){e.printStackTrace ();} 

109  } 

110  // 获得文件长度

111  public long getFileSize() 

112  { 

113  int nFileLength = -1; 

114  try{ 

115  URL url = new URL(siteInfoBean.getSSiteURL()); 

116  HttpURLConnection httpConnection = (HttpURLConnection)url.openConnection ();

117  httpConnection.setRequestProperty("User-Agent","NetFox"); 

118  int responseCode=httpConnection.getResponseCode(); 

119  if(responseCode>=400) 

120  { 

121  processErrorCode(responseCode); 

122  return -2; //-2 represent access is error 

123  } 

124  String sHeader; 

125  for(int i=1;;i++) 

126  { 

127  //DataInputStream in = new DataInputStream(httpConnection.getInputStream ()); 

128  //Utility.log(in.readLine()); 

129  sHeader=httpConnection.getHeaderFieldKey(i); 

130  if(sHeader!=null) 

131  { 

132  if(sHeader.equals("Content-Length")) 

133  { 

134  nFileLength = Integer.parseInt(httpConnection.getHeaderField(sHeader)); 

135  break; 

136  } 

137  } 

138  else 

139  break; 

140  } 

141  } 

142  catch(IOException e){e.printStackTrace ();} 

143  catch(Exception e){e.printStackTrace ();} 

144  Utility.log(nFileLength); 

145  return nFileLength; 

146  } 

147  // 保存下载信息（文件指针位置）

148  private void write_nPos() 

149  { 

150  try{ 

151  output = new DataOutputStream(new FileOutputStream(tmpFile)); 

152  output.writeInt(nStartPos.length); 

153  for(int i=0;i<nStartPos.length;i++) 

154  { 

155  // output.writeLong(nPos[i]); 

156  output.writeLong(fileSplitterFetch[i].nStartPos); 

157  output.writeLong(fileSplitterFetch[i].nEndPos); 

158  } 

159  output.close(); 

160  } 

161  catch(IOException e){e.printStackTrace ();} 

162  catch(Exception e){e.printStackTrace ();} 

163  } 

164  // 读取保存的下载信息（文件指针位置）

165  private void read_nPos() 

166  { 

167  try{ 

168  DataInputStream input = new DataInputStream(new FileInputStream(tmpFile)); 

169  int nCount = input.readInt(); 

170  nStartPos = new long[nCount]; 

171  nEndPos = new long[nCount]; 

172  for(int i=0;i<nStartPos.length;i++) 

173  { 

174  nStartPos[i] = input.readLong(); 

175  nEndPos[i] = input.readLong(); 

176  } 

177  input.close(); 

178  } 

179  catch(IOException e){e.printStackTrace ();} 

180  catch(Exception e){e.printStackTrace ();} 

181  } 

182  private void processErrorCode(int nErrorCode) 

183  { 

184  System.err.println("Error Code : " + nErrorCode); 

185  } 

186  // 停止文件下载

187  public void siteStop() 

188  { 

189  bStop = true; 

190  for(int i=0;i<nStartPos.length;i++) 

191  fileSplitterFetch[i].splitterStop(); 

192  } 

193  }

View Code

  1 /* 

  2  **FileSplitterFetch.java 

  3  */ 

  4  package NetFox; 

  5  import java.io.*; 

  6  import java.net.*; 

  7  public class FileSplitterFetch extends Thread { 

  8  String sURL; //File URL 

  9  long nStartPos; //File Snippet Start Position 

 10  long nEndPos; //File Snippet End Position 

 11  int nThreadID; //Thread's ID 

 12  boolean bDownOver = false; //Downing is over 

 13  boolean bStop = false; //Stop identical 

 14  FileAccessI fileAccessI = null; //File Access interface 

 15  public FileSplitterFetch(String sURL,String sName,long nStart,long nEnd,int id)

 16  throws IOException 

 17  { 

 18  this.sURL = sURL; 

 19  this.nStartPos = nStart; 

 20  this.nEndPos = nEnd; 

 21  nThreadID = id; 

 22  fileAccessI = new FileAccessI(sName,nStartPos); 

 23  } 

 24  public void run() 

 25  { 

 26  while(nStartPos < nEndPos && !bStop) 

 27  { 

 28  try{ 

 29  URL url = new URL(sURL); 

 30  HttpURLConnection httpConnection = (HttpURLConnection)url.openConnection (); 

 31  httpConnection.setRequestProperty("User-Agent","NetFox"); 

 32  String sProperty = "bytes="+nStartPos+"-"; 

 33  httpConnection.setRequestProperty("RANGE",sProperty); 

 34  Utility.log(sProperty); 

 35  InputStream input = httpConnection.getInputStream(); 

 36  //logResponseHead(httpConnection); 

 37  byte[] b = new byte[1024]; 

 38  int nRead; 

 39  while((nRead=input.read(b,0,1024)) > 0 && nStartPos < nEndPos 

 40  && !bStop) 

 41  { 

 42  nStartPos += fileAccessI.write(b,0,nRead); 

 43  //if(nThreadID == 1) 

 44  // Utility.log("nStartPos = " + nStartPos + ", nEndPos = " + nEndPos); 

 45  } 

 46  Utility.log("Thread " + nThreadID + " is over!"); 

 47  bDownOver = true; 

 48  //nPos = fileAccessI.write (b,0,nRead); 

 49  } 

 50  catch(Exception e){e.printStackTrace ();} 

 51  } 

 52  } 

 53  // 打印回应的头信息

 54  public void logResponseHead(HttpURLConnection con) 

 55  { 

 56  for(int i=1;;i++) 

 57  { 

 58  String header=con.getHeaderFieldKey(i); 

 59  if(header!=null) 

 60  //responseHeaders.put(header,httpConnection.getHeaderField(header)); 

 61  Utility.log(header+" : "+con.getHeaderField(header)); 

 62  else 

 63  break; 

 64  } 

 65  } 

 66  public void splitterStop() 

 67  { 

 68  bStop = true; 

 69  } 

 70  } 

 71  

 72  /* 

 73  **FileAccess.java 

 74  */ 

 75  package NetFox; 

 76  import java.io.*; 

 77  public class FileAccessI implements Serializable{ 

 78  RandomAccessFile oSavedFile; 

 79  long nPos; 

 80  public FileAccessI() throws IOException 

 81  { 

 82  this("",0); 

 83  } 

 84  public FileAccessI(String sName,long nPos) throws IOException 

 85  { 

 86  oSavedFile = new RandomAccessFile(sName,"rw"); 

 87  this.nPos = nPos; 

 88  oSavedFile.seek(nPos); 

 89  } 

 90  public synchronized int write(byte[] b,int nStart,int nLen) 

 91  { 

 92  int n = -1; 

 93  try{ 

 94  oSavedFile.write(b,nStart,nLen); 

 95  n = nLen; 

 96  } 

 97  catch(IOException e) 

 98  { 

 99  e.printStackTrace (); 

100  } 

101  return n; 

102  } 

103  } 

104  

105  /* 

106  **SiteInfoBean.java 

107  */ 

108  package NetFox; 

109  public class SiteInfoBean { 

110  private String sSiteURL; //Site's URL 

111  private String sFilePath; //Saved File's Path 

112  private String sFileName; //Saved File's Name 

113  private int nSplitter; //Count of Splited Downloading File 

114  public SiteInfoBean() 

115  { 

116  //default value of nSplitter is 5 

117  this("","","",5); 

118  } 

119  public SiteInfoBean(String sURL,String sPath,String sName,int nSpiltter)

120  { 

121  sSiteURL= sURL; 

122  sFilePath = sPath; 

123  sFileName = sName; 

124  this.nSplitter = nSpiltter; 

125  } 

126  public String getSSiteURL() 

127  { 

128  return sSiteURL; 

129  } 

130  public void setSSiteURL(String value) 

131  { 

132  sSiteURL = value; 

133  } 

134  public String getSFilePath() 

135  { 

136  return sFilePath; 

137  } 

138  public void setSFilePath(String value) 

139  { 

140  sFilePath = value; 

141  } 

142  public String getSFileName() 

143  { 

144  return sFileName; 

145  } 

146  public void setSFileName(String value) 

147  { 

148  sFileName = value; 

149  } 

150  public int getNSplitter() 

151  { 

152  return nSplitter; 

153  } 

154  public void setNSplitter(int nCount) 

155  { 

156  nSplitter = nCount; 

157  } 

158  } 

159  

160  /* 

161  **Utility.java 

162  */ 

163  package NetFox; 

164  public class Utility { 

165  public Utility() 

166  { 

167  } 

168  public static void sleep(int nSecond) 

169  { 

170  try{ 

171  Thread.sleep(nSecond); 

172  } 

173  catch(Exception e) 

174  { 

175  e.printStackTrace (); 

176  } 

177  } 

178  public static void log(String sMsg) 

179  { 

180  System.err.println(sMsg); 

181  } 

182  public static void log(int sMsg) 

183  { 

184  System.err.println(sMsg); 

185  } 

186  } 

187  

188  /* 

189  **TestMethod.java 

190  */ 

191  package NetFox; 

192  public class TestMethod { 

193  public TestMethod() 

194  { ///xx/weblogic60b2_win.exe 

195  try{ 

196  SiteInfoBean bean = new SiteInfoBean("http://localhost/xx/weblogic60b2_win.exe",

197      "L:\\temp","weblogic60b2_win.exe",5); 

198  //SiteInfoBean bean = new SiteInfoBean("http://localhost:8080/down.zip","L:\\temp",

199      "weblogic60b2_win.exe",5); 

200  SiteFileFetch fileFetch = new SiteFileFetch(bean); 

201  fileFetch.start(); 

202  } 

203  catch(Exception e){e.printStackTrace ();} 

204  } 

205  public static void main(String[] args) 

206  { 

207  new TestMethod(); 

208  } 

209  }

View Code

 

原文链接：http://www.ibm.com/developerworks/cn/java/joy-down/