代理 下载网页，挖掘数据

代理 下载网页，挖掘数据

URL url = new URL("http://blog.csdn.net/mywait_00/article/details/1698627");

//设置代理
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("openproxy.fsfd.com", 8080));
//打开代理
URLConnection coon = url.openConnection(proxy);
//访问的时候需要设置 user-agent
coon.setRequestProperty("User-Agent","Mozila/4.0(compatible;MSIE 5.0;Windows XP;DigExt");

BufferedReader in = new BufferedReader(new InputStreamReader(coon.getInputStream()));

String inputLine; StringBuffer html = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
    html.append(inputLine);
}

 

已有 0 人发表留言，猛击->>这里<<-参与讨论


ITeye推荐

• —软件人才免语言低担保 赴美带薪读研！—