python3爬虫,通过cookies来跳过验证码直接爬取友盟数据

最近在做游戏的运营,每天都要上友盟的数据统计平台拉每天的活跃和新增,正好自己会python,就做了一个自动化的爬虫。

目前是单一的json处理,后续直接加一个循环的链接嵌套就行了。

import requests
import xlwt

cookie = '''abc'''#放入自己的账号cookies
header = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36',
'Connection': 'keep-alive',
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
'cookie':cookie}
url = 'https://mobile.umeng.com/apps/650000651047c2f767d74c95/game_reports/load_table_data?page=1&per_page=30&stats=channel_stats_details&daytime=yesterday'#对于的游戏的友盟接口

jichu = [] #空列表,做总的列表汇总。
wbdate = requests.get(url,headers =header).json()['stats'] #直接读取返回的json数据串
zhongzhuan = wbdate
jichu = jichu+zhongzhuan

#初始化一个Excel
workbook = xlwt.Workbook(encoding = 'utf-8')
sheet1 = workbook.add_sheet('shuju')

#列表写入Excel
ll = list(jichu[0].keys())
for i in range(0,len(ll)):
    sheet1.write(0,i,ll[i])
for j in range(0,len(jichu)):
        m = 0
        ls = list(jichu[j].values())
        for k in ls:
                sheet1.write(j+1,m,k)
                m+=1    


workbook.save('D:\\work\\ceshi.xls')

你可能感兴趣的:(python)