Python爬虫之模拟登陆知乎

在chrome浏览器下抓取登陆过程的包（注意把Preserve log勾上）：

表单的结构主要包括_xsrf, password, phone_num

我们要找到_xsrf的值，重新载入zhihu.com之后我们可以发现Response里面有_xsrf

我们就可以把_xsrf的值读取出来

然后set一下cookies，就可以模拟登陆知乎了。

import requests
from bs4 import BeautifulSoup

zhihu_url = 'http://www.zhihu.com'
headers = {
        'Referer': 'http://www.zhihu.com/',
        'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/55.0.2883.87 Chrome/55.0.2883.87 Safari/537.36'}
#get xsrf
xsrf = BeautifulSoup(requests.get(zhihu_url, headers = headers).content, 'html.parser').find('input')['value']

#get cookies
data = {
        '_xsrf': xsrf,
        'phone_num': '***********',
        'password': '**********'}

loginurl = 'https://www.zhihu.com/login/phone_num'
cookies = requests.post(loginurl, data = data, headers = headers).cookies

#login
url = 'https://www.zhihu.com/question/55940910'
html = requests.get(url, headers = headers, cookies = cookies)
soup = BeautifulSoup(html.content, 'html.parser')
titles = soup.select(r'.QuestionHeader-title')
print('title: ', titles[0].text)