爬虫之Boss直聘 招聘数据采集（zp_stoken参数逆向分析JSRPC）

前言

大家早好、午好、晚好吖 ❤ ~欢迎光临本文章

如果有什么疑惑/资料需要的可以点击文章末尾名片领取源码

课程亮点:

1、动态数据抓包

2、JSON数据解析方法

环境介绍:

• python 3.8 解释器: 执行代码的

• pycharm 专业版: 写代码的 帮助我们 快速开发代码的

• requests >>> pip install requests

• sekiro >>> RPC框架

抓包分析

确定需要的数据 然后找该数据的来源

https://www.zhipin.com/wapi/zpgeek/search/joblist.json?scene=1&query=python%E7%88%AC%E8%99%AB&city=100010000&experience=&payType=&partTime=°ree=&industry=&scale=&stage=&position=&jobType=&salary=&multiBusinessDistrict=&multiSubway=&page=1&pageSize=30

代码实现

1. 发送请求

2. 获取数据

3. 解析数据

代码展示

'''
python资料获取看这里噢！！ 小编 V：qian97378，即可获取：
文章源码/教程/资料/解答等福利，还有不错的视频学习教程和PDF电子书！
'''
import requests
import csv


with open('boss.csv', mode='w', newline='', encoding='utf-8') as f:
    csv.writer(f).writerow(['jobName', 'cityName', 'areaDistrict', 'businessDistrict', 'salaryDesc', 'jobExperience', 'jobDegree', 'skills', 'brandName', 'brandIndustry', 'brandStageName', 'brandScaleName', 'welfareList'])
# 请求头
headers = {
    'Referer': 'https://www.zhipin.com/web/geek/job?query=python%E7%88%AC%E8%99%AB&city=100010000&page=1',
    #          wd_guid=74818318-ca4c-4c55-8023-515796a1af9a; historyState=state; _bl_uid=gLl3kn8OimXutaow0kgvk3sq8qbh; __g=-; Hm_lvt_194df3105ad7148dcf2b98a91b5e727a=1699018011,1699707457,1699962314,1699965523; Hm_lpvt_194df3105ad7148dcf2b98a91b5e727a=1699965558; __zp_stoken__=3f6ceWFc7eFF0Skx2DUZ5fiBSb0RZDWpRc3ciERc6aTt3IB5UeRZzNyVTO2s9VRI5D24SQDxkDzVdfi9bSVQWXQZtQUcefiNsFCk5FHMuZHlKCVVkOnIELDdGfhZeASBxTVcgPzgbDT9tbCE%3D; __c=1699963318; __l=l=%2Fwww.zhipin.com%2Fweb%2Fgeek%2Fjob%3Fquery%3Dpython%25E7%2588%25AC%25E8%2599%25AB%26city%3D100010000%26page%3D1&r=&g=&s=3&friend_source=0&s=3&friend_source=0; __a=40359856.1696853365.1699962313.1699963318.75.11.11.75
    'Cookie': 'wd_guid=74818318-ca4c-4c55-8023-515796a1af9a; historyState=state; _bl_uid=gLl3kn8OimXutaow0kgvk3sq8qbh; __g=-; Hm_lvt_194df3105ad7148dcf2b98a91b5e727a=1699018011,1699707457,1699962314,1699965523; Hm_lpvt_194df3105ad7148dcf2b98a91b5e727a=1699965558; __zp_stoken__=3f6ceWFc7eFF0SnAKXyRmfiBSb0Q1dz8TTzUiERc6aRxqORo%2BeRZzNyVTeiwdMmkRD24SQDxkDzVzfntbRwlYVU9KZxESMRhUGQ0DFHMuZHlKCX0fXVJDbTdGfhZeASBxTVcgPzgbDT9tbCE%3D; __c=1699963318; __l=l=%2Fwww.zhipin.com%2Fweb%2Fgeek%2Fjob%3Fquery%3Dpython%25E7%2588%25AC%25E8%2599%25AB%26city%3D100010000%26page%3D1&r=&g=&s=3&friend_source=0&s=3&friend_source=0; __a=40359856.1696853365.1699962313.1699963318.75.11.11.75',
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36'
}
for page in range(1, 31):
    url = f'https://www.zhipin.com/wapi/zpgeek/search/joblist.json?scene=1&query=python%E7%88%AC%E8%99%AB&city=100010000&experience=&payType=&partTime=°ree=&industry=&scale=&stage=&position=&jobType=&salary=&multiBusinessDistrict=&multiSubway=&page={page}&pageSize=30'
    # 1. 发送请求
    response = requests.get(url=url, headers=headers)
    # 2. 获取数据
    json_data = response.json()
    # 3. 提取数据
    jobList = json_data['zpData']['jobList']
    for job in jobList:
        jobName = job['jobName']
        cityName = job['cityName']
        areaDistrict = job['areaDistrict']
        businessDistrict = job['businessDistrict']
        salaryDesc = job['salaryDesc']
        jobExperience = job['jobExperience']
        jobDegree = job['jobDegree']
        skills = ' '.join(job['skills'])
        brandName = job['brandName']
        brandIndustry = job['brandIndustry']
        brandStageName = job['brandStageName']
        brandScaleName = job['brandScaleName']
        welfareList = ' '.join(job['welfareList'])
        print(jobName, cityName, areaDistrict, businessDistrict, salaryDesc, jobExperience, jobDegree, skills, brandName, brandIndustry, brandStageName, brandScaleName, welfareList)
        with open('boss.csv', mode='a', newline='', encoding='utf-8') as f:
            csv.writer(f).writerow([jobName, cityName, areaDistrict, businessDistrict, salaryDesc, jobExperience, jobDegree, skills, brandName, brandIndustry, brandStageName, brandScaleName, welfareList])

尾语

好了，今天的分享就差不多到这里了！

对下一篇大家想看什么，可在评论区留言哦！看到我会更新哒(ง •_•)ง

喜欢就关注一下博主，或点赞收藏评论一下我的文章叭！！！

最后，宣传一下呀~ 更多源码、资料、素材、解答、交流 皆点击下方名片获取呀