python爬虫-批量下载qq音乐

目标

这次要爬取的是qq音乐网站【排行榜】中所有的歌曲（共100首）
网址：https://y.qq.com/n/yqq/toplist/4.html#stat=y_new.top.pop.logout

废话不多说，直接开始吧！

接口分析

点击排行榜页面中的任意一首歌，进入到播放界面。

打开开发者工具，重新刷新网页，寻找歌曲下载接口
找呀找，找呀找
发现许多media类型的，选择size最大的那条，点击进去。
果然，找到了歌曲下载的url：
http://124.232.155.149/amobile.music.tc.qq.com/C400101A9o5D1EWzyd.m4a?guid=5300073565&vkey=30F12E95C394B1C0550AAAC8E946191E757BB8190B983FA1E13E7C0EEF6CE49AC1C051CB2922C371F7EBA77B8C36DA7F0BB3D85819525C75&uin=0&fromtag=66

然后，分析下这个url的构造：
和其他的歌曲对比，只有m4a文件名和vkey这两个参数不一致
所以，我们要找的就是这两个参数。
那么到底从何而来呢？

又找啊找，找啊找
终于发现了！找到了vkey！

接着找m4a文件名
找啊找，找啊找
咦！好像有新发现！
purl中好像已经包含了m4a文件名和vkey！
没错，就是purl了！
实际上，歌曲下载的url就是 http://124.232.155.149/amobile.music.tc.qq.com/+purl

接下来，就是要分析purl是怎么来的了
分析请求的url：

还是老办法，和其他的歌曲对比。（此处省略复杂繁琐对比。。。）
发现只有songmid这个参数不一致
所以，要分析的就是songmid是从哪个接口过来的。
再次回到排行榜页面
找啊找 找了很久 终于发现了！

最后，再分析下songmid的请求url：
https://c.y.qq.com/v8/fcg-bin/fcg_v8_toplist_cp.fcg?tpl=3&page=detail&date=2018-12-14&topid=4&type=top&song_begin=0&song_num=30&g_tk=5381&jsonpCallback=MusicJsonCallbacktoplist&loginUin=0&hostUin=0&format=jsonp&inCharset=utf8&outCharset=utf-8¬ice=0&platform=yqq&needNewCode=0
song_num是指定请求多少条数据。
由于我们要下载所有的歌曲（共100首）,所以要将song_num改成100

完整代码

#目标：下载qq音乐排行榜前100首
#思路：
#1.歌曲下载链接url1：http://124.232.144.154/amobile.music.tc.qq.com/C400101A9o5D1EWzyd.m4a?guid=5300073565&vkey=B1A12B2E4190CB21134F652893BEEF69AE405D39E060DEEACF358A6F729541E160F926F5EADD391C68047B2ED0156690A97CA50AC99DA40D&uin=0&fromtag=66
#2.寻找下载链接中的参数filname:C400101A9o5D1EWzyd.m4a    vkey
#3.filname与vkey的请求链接url2：https://u.y.qq.com/cgi-bin/musicu.fcg?callback=getplaysongvkey2904725697971273&g_tk=5381&jsonpCallback=getplaysongvkey2904725697971273&loginUin=0&hostUin=0&format=jsonp&inCharset=utf8&outCharset=utf-8¬ice=0&platform=yqq&needNewCode=0&data=%7B%22req%22%3A%7B%22module%22%3A%22CDN.SrfCdnDispatchServer%22%2C%22method%22%3A%22GetCdnDispatch%22%2C%22param%22%3A%7B%22guid%22%3A%225300073565%22%2C%22calltype%22%3A0%2C%22userip%22%3A%22%22%7D%7D%2C%22req_0%22%3A%7B%22module%22%3A%22vkey.GetVkeyServer%22%2C%22method%22%3A%22CgiGetVkey%22%2C%22param%22%3A%7B%22guid%22%3A%225300073565%22%2C%22songmid%22%3A%5B%22001A9o5D1EWzyd%22%5D%2C%22songtype%22%3A%5B0%5D%2C%22uin%22%3A%220%22%2C%22loginflag%22%3A1%2C%22platform%22%3A%2220%22%7D%7D%2C%22comm%22%3A%7B%22uin%22%3A0%2C%22format%22%3A%22json%22%2C%22ct%22%3A20%2C%22cv%22%3A0%7D%7D
#4.url2的构造：songmid
#5.寻找songmid:
# songmid请求的链接url3:https://c.y.qq.com/v8/fcg-bin/fcg_v8_toplist_cp.fcg?tpl=3&page=detail&date=2018-12-14&topid=4&type=top&song_begin=0&song_num=30&g_tk=5381&jsonpCallback=MusicJsonCallbacktoplist&loginUin=0&hostUin=0&format=jsonp&inCharset=utf8&outCharset=utf-8¬ice=0&platform=yqq&needNewCode=0


import requests
import re
import json
from multiprocessing.dummy import Pool as ThreadPool


#返回songmid和songname
def get_param():
    url = 'https://c.y.qq.com/v8/fcg-bin/fcg_v8_toplist_cp.fcg?tpl=3&page=detail&date=2018-12-14&topid=4&type=top&song_begin=0&song_num=100&g_tk=5381&jsonpCallback=MusicJsonCallbacktoplist&loginUin=0&hostUin=0&format=jsonp&inCharset=utf8&outCharset=utf-8¬ice=0&platform=yqq&needNewCode=0'
    response = requests.get(url).content.decode('utf-8')
    purl_data = re.findall(r'\w+\((.*)',response)[0]
    data_json = json.loads(purl_data)
    songlist = data_json['songlist']
    param_list = []
    for item in songlist:
        param_dict = {}
        songmid = item['data']['songmid']
        songname = item['data']['songname']
        param_dict['songmid'] = songmid
        param_dict['songname'] = songname
        param_list.append(param_dict)
    return param_list

#下载音乐
def download_music(param_dict):
    songmid = param_dict['songmid']
    songname = param_dict['songname']
    data =json.dumps({"req": {"module": "CDN.SrfCdnDispatchServer","method": "GetCdnDispatch","param": {"guid": "5300073565","calltype": 0,"userip": ""}},"req_0": {"module": "vkey.GetVkeyServer","method": "CgiGetVkey","param": {"guid": "5300073565","songmid": [songmid],"songtype": [0],"uin": "0","loginflag": 1,"platform": "20"}},"comm": {"uin": 0,"format": "json","ct": 20,"cv": 0}})
    url = 'https://u.y.qq.com/cgi-bin/musicu.fcg?callback=getplaysongvkey2904725697971273&g_tk=5381&jsonpCallback=getplaysongvkey2904725697971273&loginUin=0&hostUin=0&format=jsonp&inCharset=utf8&outCharset=utf-8¬ice=0&platform=yqq&needNewCode=0&data={}'.format(data)
    response = requests.get(url).content.decode('utf-8')
    purl_data = re.findall(r'\w+\((.*)\)',response)[0]
    data_json = json.loads(purl_data)
    purl = data_json['req_0']['data']['midurlinfo'][0]['purl']
    download_url = 'http://124.232.144.154/amobile.music.tc.qq.com/{}'.format(purl)
    print("=======正在下载:{}=======".format(songname))
    music = requests.get(download_url)
    with open("F:\BaiduNetdiskDownload\QQmusic\music\{}.m4a".format(re.sub(r'[\s+|@<>:\\"/]','',songname)),"wb") as m:
         m.write(music.content)

if __name__ == '__main__':
    param_list = get_param()
    pool = ThreadPool(5)
    pool.map(download_music,param_list)
    pool.close()
    pool.join()

运行效果

下载完的歌曲：

现在终于可以听歌了！
听着听着。。。发现有个别歌曲文件大小居然是0KB
原来，VIP音乐没下载下来。