Python+selenium使用cookie登录淘宝

众所周知，使用常规方法爬取淘宝的难度是很高的，所以使用selenium+浏览器几乎成了爬取淘宝最理想的方法。然而现在淘宝搜索需要用户登录才可以，据说是为了收集匹配用户信息。

taobao.gif

而每次使用selenium时（+chrome），chromedriver会新打开一个浏览器窗口，所以添加cookie便显得很必要了。
这里看一下selenium的add_cookie方法

20190105143721122.png

与普通的在headers里添加{'Cookies':' '}不一样的是，此方法需要按照cookie的name,value,path,domain格式逐个cookie添加，如下图

淘宝cookies格式

淘宝的cookie很多，所以需要逐项添加
而手动复制的话很不方便，所以这里使用到一个chrome插件
EditThisCookie

EditThisCookie

它有个导出功能，当你登录完淘宝后点击导出便会得到一个list格式的字符串

cookies=
[
{
    "domain": ".baidu.com",
    "expirationDate": 1563853177.468827,
    "hostOnly": false,
    "httpOnly": false,
    "name": "BAIDUID",
    "path": "/",
    "sameSite": "no_restriction",
    "secure": false,
    "session": false,
    "storeId": "0",
    "value": "C0E2BCF5DCEB680B2453F78AC4CD0956:FG=1",
    "id": 1
},
{
    "domain": ".baidu.com",
    "expirationDate": 1546756058.11283,
    "hostOnly": false,
    "httpOnly": false,
    "name": "BDORZ",
    "path": "/",
    "sameSite": "no_restriction",
    "secure": false,
    "session": false,
    "storeId": "0",
    "value": "FFFB88E999055A3F8A630C64834BD6D0",
    "id": 2
},
{
    "domain": ".baidu.com",
    "expirationDate": 3648534764.317173,
    "hostOnly": false,
    "httpOnly": false,
    "name": "BIDUPSID",
    "path": "/",
    "sameSite": "no_restriction",
    "secure": false,
    "session": false,
    "storeId": "0",
    "value": "CC0841B75FADCE2D56D45D685391FED0",
    "id": 3
},
{
    "domain": ".baidu.com",
    "hostOnly": false,
    "httpOnly": false,
    "name": "H_PS_PSSID",
    "path": "/",
    "sameSite": "no_restriction",
    "secure": false,
    "session": true,
    "storeId": "0",
    "value": "",
    "id": 4
},
{
    "domain": ".baidu.com",
    "expirationDate": 2493441539.633233,
    "hostOnly": false,
    "httpOnly": false,
    "name": "MCITY",
    "path": "/",
    "sameSite": "no_restriction",
    "secure": false,
    "session": false,
    "storeId": "0",
    "value": "-%3A",
    "id": 5
},
{
    "domain": ".baidu.com",
    "expirationDate": 3648534764.317246,
    "hostOnly": false,
    "httpOnly": false,
    "name": "PSTM",
    "path": "/",
    "sameSite": "no_restriction",
    "secure": false,
    "session": false,
    "storeId": "0",
    "value": "1501051117",
    "id": 6
},
{
    "domain": ".www.baidu.com",
    "expirationDate": 2492662056,
    "hostOnly": false,
    "httpOnly": false,
    "name": "bdime",
    "path": "/",
    "sameSite": "no_restriction",
    "secure": false,
    "session": false,
    "storeId": "0",
    "value": "0",
    "id": 7
}
]

稍加修改就可以作为python的list来导入cookie了

for item in cookies:
    driver.add_cookie(item)

之后使用selenium就可以正常搜索了

淘宝搜索