淘宝店铺商品全量爬取

场景

1.获取淘宝某个店铺下商品价格和销量

逆向思路分析

1.在店铺首页点击宝贝栏目,打开调试,发现

 https://h5api.m.taobao.com/h5/mtop.taobao.wsearch.appsearch/1.0/

这个接口.返回json数据格式,里面包含了商品价格和销量.
2.再仔细看接口,发现做了加密

jsv: 2.5.1
appKey: 12574478
t: 1585103036332
sign: e394cf6719ba893a80e11ec622277439
api: mtop.taobao.wsearch.appSearch
v: 1.0
H5Request: true
AntiCreep: true

3.debug一下url,发现加密位置.

        if (d.H5Request === !0) {
            var f = "//" + (d.prefix ? d.prefix + "." : "") + (d.subDomain ? d.subDomain + "." : "") + d.mainDomain + "/h5/" + c.api.toLowerCase() + "/" + c.v.toLowerCase() + "/"
              , g = c.appKey || ("waptest" === d.subDomain ? "4272" : "12574478")
              , i = (new Date).getTime()
              , j = h(d.token + "&" + i + "&" + g + "&" + c.data)
              , k = {
                jsv: w,
                appKey: g,
                t: i,
                sign: j
            }

4.最后就很简单拉.Python使用pyexecjs执行js代码块.输出:

def run():
    js = '''找到的js代码'''
    param = '53a4d7c47f841429bf186e9a7a9c16fd&1585100918934&12574478&{"m":"shopitemsearch","vm":"nw","sversion":"4.6","shopId":"575905230","sellerId":"2200723654612","style":"wf","page":"1","sort":"_coefp","catmap":"","wirelessShopCategoryList":""}'
    ej = execjs.compile(js)
    result = ej.call('h', param)
    print(result)
输出:    
0f3dcda1219fe95d7f0944a53c5131f6