python3.5 爬取mzitu网站图片

之前的文章,都是基于python2.X来做的,现在我们很多都是使用python3.x,导致很多的代码不能用 ,因为python2和python3 的差别还是很大的,所以根据修改后可以执行的代码如下:

模块使用的是:requests和bs4模块,安装也是很简单,pip install XXX 即可

编辑器:Vscode

版本:python3.5

import requests
from bs4 import BeautifulSoup
import os

Hostreferer={
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36',
    'Referer':'http://www.mzitu.com'
} 
Picreferer={
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36',
    'Referer':'http://i.meizitu.net'
}

def get_html(url):
    req=requests.get(url,headers=Hostreferer)
    html=req.text
    return html

def get_page_name(url):
    html=get_html(url)
    soup=BeautifulSoup(html,'lxml')
    span=soup.findAll('span')
    title=soup.find('h2',class_='main-title')
    return span[10].text,title.text

def get_img_url(url,name):
    html=get_html(url)
    soup=BeautifulSoup(html,'lxml')
    img_url=soup.find('img',alt=name)
    return img_url['src']

def save_img(img_url,count,name):
    req=requests.get(img_url,headers=Picreferer)
    with open('name+'/'+str(count)+'.jpg','wb')as f:
        f.write(req.content)


if '__name__'=='__main__':

     old_url='http://www.mzitu.com/123114'
     page,name=get_page_name(old_url)
     os.mkdir(name)
     for i in range(1,int(page)+1):
         url=old_url+'/'+str(i)
         img_url=get_img_url(url,name)
         save_img(img_url,i,name)
         print('保存第'+str(i)+'张图片成功')


代码是可以运行的,结果图就不放了,你懂的!!!

 

 

你可能感兴趣的:(Spider,Python爬虫专栏)