爬虫响应cookie阿里系案例:某财经

声明:
该文章为学习使用,严禁用于商业用途和非法用途,违者后果自负,由此产生的一切后果均与作者无关

一、响应cookie阿里系特点
  1. cookie中一定有acw_sc__v2
  2. 清除所有cookie刷新页面时,会自动debugger到设置cookie的文件
  3. 同一个请求会发两次
  4. 第一次请求,响应状态码200,拿到响应cookie:acw_tc,并生成acw_sc__v2,
  5. 第二次请求,响应状态码200,使用acw_tc、acw_sc__v2重新发送请求,并拿到响应cookie:xq_a_token
二、找出需要加密的参数
  1. js运行 atob(‘aHR0cHM6Ly94dWVxaXUuY29tL3RvZGF5’) 拿到网址,F12打开调试工具,点击雪球热帖,找到 hot/listV2.json 请求,鼠标右击请求找到Copy>Copy as cUrl(cmd)
  2. 打开网站:https://spidertools.cn/#/curl2Request,把拷贝好的curl转成python代码,新建 xueqiu.py,把代码复制到该文件
    爬虫响应cookie阿里系案例:某财经_第1张图片
  3. 然后把代码中的header全部注释,再运行文件,会发现数据依然可以请求成功,再把cookie全部注释,会发现请求失败,说明cookie中有加密字段,再逐个注释会发现,除了xqat、xq_a_token其他字段注释都能请求成功,说明xqat、xq_a_token是加密字段,而xq_a_token、xqat值是一样的,所以只要找到一个加密方式就行
    爬虫响应cookie阿里系案例:某财经_第2张图片
  4. 新建xueqiu.js,用于放扣下的代码
三、定位cookie生成
  1. 关键字xq_a_token搜索,会发现xq_a_token是由响应头set-cookie设置的cookie,Set-Cookie是服务器在客户端储存的一些信息,用于在后续的请求中传递和存储用户相关的信息或状态,分析请求中的cookies会发现acw_sc__v2,这个字段是阿里系cookie的特点之一
    爬虫响应cookie阿里系案例:某财经_第3张图片爬虫响应cookie阿里系案例:某财经_第4张图片
  2. 切换到Application,清除浏览器的cookie,一定要先清除cookie,清除cookie后,刷新页面,会发现有个无限循环的debugger,在栈中找到_0x355d23,把对应的混肴在控制台输出,会发现是个constructor debugger,_0x355d23所在的文件就是acw_sc__v2生成的文件,在acw_sc__v2代码行打上断点
    爬虫响应cookie阿里系案例:某财经_第5张图片爬虫响应cookie阿里系案例:某财经_第6张图片爬虫响应cookie阿里系案例:某财经_第7张图片
  3. 过constructor debugger,添加constructor_debugger,点击运行,再点击跳过断点,已经进入刚才大的断点,这个debugger也是阿里系cookie的特点之一
    爬虫响应cookie阿里系案例:某财经_第8张图片
  4. 分析setCookie(“acw_sc__v2”, x),会发现 acw_sc__v2 = x,x是reload的参数,从栈中找会发现reload(arg2),传的参数是arg2,继续跟栈,会发现arg2的位置,在该位置打上断点
    爬虫响应cookie阿里系案例:某财经_第9张图片爬虫响应cookie阿里系案例:某财经_第10张图片
  5. 清除cookie,再次刷新页面,遇见无限debugger重复步骤3,过了debugger之后会发现进入刚才之前的断点,分析代码会发现是由_0x23a392[_0x55f3(‘0x1b’, ‘\x7a\x35\x4f\x26’)] 这个函数生成,而_0x23a392 = arg1_0x55f3(‘0x19’, ‘\x50\x67\x35\x34’),鼠标悬浮arg1[_0x55f3(‘0x19’, ‘\x50\x67\x35\x34’)],点击蓝色部分会找到该方法,会发现该方法是 String[‘\x70\x72\x6f\x74\x6f\x74\x79\x70\x65’][_0x55f3(‘0x14’, ‘\x5a\x2a\x44\x4d’)],都是在 l 方法内部,把 l方法、ob混肴方法、ob混肴的数组、arg1拷贝到xueqiu.js,并把l方法最后的timeout删了,替换成return arg2
    爬虫响应cookie阿里系案例:某财经_第11张图片爬虫响应cookie阿里系案例:某财经_第12张图片
  6. 运行xueqiu.js,会发现程序一直卡着,那是因为ob混肴中有格式检测,访问:https://www.jsjiami.com/jiemi.html,把ob混肴的代码压缩下,再执行xueqiu.js会发现报window的错误,开始补环境
    爬虫响应cookie阿里系案例:某财经_第13张图片
  7. 补上window = global,会发现报 _0x55f3 的错误, _0x55f3是一个方法,把该方法和ob混肴的代码,一起拷贝压缩,再复制到xueqiu.js,再次运行xueqiu.js,会发现无报错
    爬虫响应cookie阿里系案例:某财经_第14张图片爬虫响应cookie阿里系案例:某财经_第15张图片爬虫响应cookie阿里系案例:某财经_第16张图片
  8. 修改xueqiu.js,因为arg1是可变的,所以把arg1当作参数传到 l 方法,并打印 l 方法的结果,会发现加密结果生成
    爬虫响应cookie阿里系案例:某财经_第17张图片
  9. 修改xueqiu.py,再次运行会发现虽然请求成功,但是响应cookie中并没有xq_a_token,today请求刚开始触发了两次,第一次触发debugger,拿到acw_tc,并生成acw_sc__v2,然后根据acw_tc、acw_sc__v2又发送一次请求拿到xq_a_token,发送两次请求也是阿里系cookie特点之一
    爬虫响应cookie阿里系案例:某财经_第18张图片爬虫响应cookie阿里系案例:某财经_第19张图片爬虫响应cookie阿里系案例:某财经_第20张图片
  10. 修改xueqiu.py,再次运行xueqiu.py,会发现数据获取成功
    爬虫响应cookie阿里系案例:某财经_第21张图片爬虫响应cookie阿里系案例:某财经_第22张图片爬虫响应cookie阿里系案例:某财经_第23张图片
四、最终代码
  1. xueqiu.js
window = global;

var _0x4818 = ['\x63\x73\x4b\x48\x77\x71\x4d\x49', '\x5a\x73\x4b\x4a\x77\x72\x38\x56\x65\x41\x73\x79', '\x55\x63\x4b\x69\x4e\x38\x4f\x2f\x77\x70\x6c\x77\x4d\x41\x3d\x3d', '\x4a\x52\x38\x43\x54\x67\x3d\x3d', '\x59\x73\x4f\x6e\x62\x53\x45\x51\x77\x37\x6f\x7a\x77\x71\x5a\x4b\x65\x73\x4b\x55\x77\x37\x6b\x77\x58\x38\x4f\x52\x49\x51\x3d\x3d', '\x77\x37\x6f\x56\x53\x38\x4f\x53\x77\x6f\x50\x43\x6c\x33\x6a\x43\x68\x4d\x4b\x68\x77\x36\x48\x44\x6c\x73\x4b\x58\x77\x34\x73\x2f\x59\x73\x4f\x47', '\x66\x77\x56\x6d\x49\x31\x41\x74\x77\x70\x6c\x61\x59\x38\x4f\x74\x77\x35\x63\x4e\x66\x53\x67\x70\x77\x36\x4d\x3d', '\x4f\x63\x4f\x4e\x77\x72\x6a\x43\x71\x73\x4b\x78\x54\x47\x54\x43\x68\x73\x4f\x6a\x45\x57\x45\x38\x50\x63\x4f\x63\x4a\x38\x4b\x36', '\x55\x38\x4b\x35\x4c\x63\x4f\x74\x77\x70\x56\x30\x45\x4d\x4f\x6b\x77\x34\x37\x44\x72\x4d\x4f\x58', '\x48\x4d\x4f\x32\x77\x6f\x48\x43\x69\x4d\x4b\x39\x53\x6c\x58\x43\x6c\x63\x4f\x6f\x43\x31\x6b\x3d', '\x61\x73\x4b\x49\x77\x71\x4d\x44\x64\x67\x4d\x75\x50\x73\x4f\x4b\x42\x4d\x4b\x63\x77\x72\x72\x43\x74\x6b\x4c\x44\x72\x4d\x4b\x42\x77\x36\x34\x64', '\x77\x71\x49\x6d\x4d\x54\x30\x74\x77\x36\x52\x4e\x77\x35\x6b\x3d', '\x44\x4d\x4b\x63\x55\x30\x4a\x6d\x55\x77\x55\x76', '\x56\x6a\x48\x44\x6c\x4d\x4f\x48\x56\x63\x4f\x4e\x58\x33\x66\x44\x69\x63\x4b\x4a\x48\x51\x3d\x3d', '\x77\x71\x68\x42\x48\x38\x4b\x6e\x77\x34\x54\x44\x68\x53\x44\x44\x67\x4d\x4f\x64\x77\x72\x6a\x43\x6e\x63\x4f\x57\x77\x70\x68\x68\x4e\x38\x4b\x43\x47\x63\x4b\x71\x77\x36\x64\x48\x41\x55\x35\x2b\x77\x72\x67\x32\x4a\x63\x4b\x61\x77\x34\x49\x45\x4a\x63\x4f\x63\x77\x72\x52\x4a\x77\x6f\x5a\x30\x77\x71\x46\x39\x59\x67\x41\x56', '\x64\x7a\x64\x32\x77\x35\x62\x44\x6d\x33\x6a\x44\x70\x73\x4b\x33\x77\x70\x59\x3d', '\x77\x34\x50\x44\x67\x63\x4b\x58\x77\x6f\x33\x43\x6b\x63\x4b\x4c\x77\x72\x35\x71\x77\x72\x59\x3d', '\x77\x72\x4a\x4f\x54\x63\x4f\x51\x57\x4d\x4f\x67', '\x77\x71\x54\x44\x76\x63\x4f\x6a\x77\x34\x34\x37\x77\x72\x34\x3d', '\x77\x35\x58\x44\x71\x73\x4b\x68\x4d\x46\x31\x2f', '\x77\x72\x41\x79\x48\x73\x4f\x66\x77\x70\x70\x63', '\x4a\x33\x64\x56\x50\x63\x4f\x78\x4c\x67\x3d\x3d', '\x77\x72\x64\x48\x77\x37\x70\x39\x5a\x77\x3d\x3d', '\x77\x34\x72\x44\x6f\x38\x4b\x6d\x4e\x45\x77\x3d', '\x49\x4d\x4b\x41\x55\x6b\x42\x74', '\x77\x36\x62\x44\x72\x63\x4b\x51\x77\x70\x56\x48\x77\x70\x4e\x51\x77\x71\x55\x3d', '\x64\x38\x4f\x73\x57\x68\x41\x55\x77\x37\x59\x7a\x77\x72\x55\x3d', '\x77\x71\x6e\x43\x6b\x73\x4f\x65\x65\x7a\x72\x44\x68\x77\x3d\x3d', '\x55\x73\x4b\x6e\x49\x4d\x4b\x57\x56\x38\x4b\x2f', '\x77\x34\x7a\x44\x6f\x63\x4b\x38\x4e\x55\x5a\x76', '\x63\x38\x4f\x78\x5a\x68\x41\x4a\x77\x36\x73\x6b\x77\x71\x4a\x6a', '\x50\x63\x4b\x49\x77\x34\x6e\x43\x6b\x6b\x56\x62', '\x4b\x48\x67\x6f\x64\x4d\x4f\x32\x56\x51\x3d\x3d', '\x77\x70\x73\x6d\x77\x71\x76\x44\x6e\x47\x46\x71', '\x77\x71\x4c\x44\x74\x38\x4f\x6b\x77\x34\x63\x3d', '\x77\x37\x77\x31\x77\x34\x50\x43\x70\x73\x4f\x34\x77\x71\x41\x3d', '\x77\x71\x39\x46\x52\x73\x4f\x71\x57\x4d\x4f\x71', '\x62\x79\x42\x68\x77\x37\x72\x44\x6d\x33\x34\x3d', '\x4c\x48\x67\x2b\x53\x38\x4f\x74\x54\x77\x3d\x3d', '\x77\x71\x68\x4f\x77\x37\x31\x35\x64\x73\x4f\x48', '\x55\x38\x4f\x37\x56\x73\x4f\x30\x77\x71\x76\x44\x76\x63\x4b\x75\x4b\x73\x4f\x71\x58\x38\x4b\x72', '\x59\x69\x74\x74\x77\x35\x44\x44\x6e\x57\x6e\x44\x72\x41\x3d\x3d', '\x59\x4d\x4b\x49\x77\x71\x55\x55\x66\x67\x49\x6b', '\x61\x42\x37\x44\x6c\x4d\x4f\x44\x54\x51\x3d\x3d', '\x77\x70\x66\x44\x68\x38\x4f\x72\x77\x36\x6b\x6b', '\x77\x37\x76\x43\x71\x4d\x4f\x72\x59\x38\x4b\x41\x56\x6b\x35\x4f\x77\x70\x6e\x43\x75\x38\x4f\x61\x58\x73\x4b\x5a\x50\x33\x44\x43\x6c\x63\x4b\x79\x77\x36\x48\x44\x72\x51\x3d\x3d', '\x77\x6f\x77\x2b\x77\x36\x76\x44\x6d\x48\x70\x73\x77\x37\x52\x74\x77\x6f\x39\x38\x4c\x43\x37\x43\x69\x47\x37\x43\x6b\x73\x4f\x52\x54\x38\x4b\x6c\x57\x38\x4f\x35\x77\x72\x33\x44\x69\x38\x4f\x54\x48\x73\x4f\x44\x65\x48\x6a\x44\x6d\x63\x4b\x6c\x4a\x73\x4b\x71\x56\x41\x3d\x3d', '\x4e\x77\x56\x2b', '\x77\x37\x48\x44\x72\x63\x4b\x74\x77\x70\x4a\x61\x77\x70\x5a\x62', '\x77\x70\x51\x73\x77\x71\x76\x44\x69\x48\x70\x75\x77\x36\x49\x3d', '\x59\x4d\x4b\x55\x77\x71\x4d\x4a\x5a\x51\x3d\x3d', '\x4b\x48\x31\x56\x4b\x63\x4f\x71\x4b\x73\x4b\x31', '\x66\x51\x35\x73\x46\x55\x6b\x6b\x77\x70\x49\x3d', '\x77\x72\x76\x43\x72\x63\x4f\x42\x52\x38\x4b\x6b', '\x4d\x33\x77\x30\x66\x51\x3d\x3d', '\x77\x36\x78\x58\x77\x71\x50\x44\x76\x4d\x4f\x46\x77\x6f\x35\x64'];
(function(_0x4c97f0,_0x1742fd){var _0x4db1c=function(_0x48181e){while(--_0x48181e){_0x4c97f0['\x70\x75\x73\x68'](_0x4c97f0['\x73\x68\x69\x66\x74']())}};var _0x3cd6c6=function(){var _0xb8360b={'\x64\x61\x74\x61':{'\x6b\x65\x79':'\x63\x6f\x6f\x6b\x69\x65','\x76\x61\x6c\x75\x65':'\x74\x69\x6d\x65\x6f\x75\x74'},'\x73\x65\x74\x43\x6f\x6f\x6b\x69\x65':function(_0x20bf34,_0x3e840e,_0x5693d3,_0x5e8b26){_0x5e8b26=_0x5e8b26||{};var _0xba82f0=_0x3e840e+'\x3d'+_0x5693d3;var _0x5afe31=0x0;for(var _0x5afe31=0x0,_0x178627=_0x20bf34['\x6c\x65\x6e\x67\x74\x68'];_0x5afe31<_0x178627;_0x5afe31++){var _0x41b2ff=_0x20bf34[_0x5afe31];_0xba82f0+='\x3b\x20'+_0x41b2ff;var _0xd79219=_0x20bf34[_0x41b2ff];_0x20bf34['\x70\x75\x73\x68'](_0xd79219);_0x178627=_0x20bf34['\x6c\x65\x6e\x67\x74\x68'];if(_0xd79219!==!![]){_0xba82f0+='\x3d'+_0xd79219}}_0x5e8b26['\x63\x6f\x6f\x6b\x69\x65']=_0xba82f0},'\x72\x65\x6d\x6f\x76\x65\x43\x6f\x6f\x6b\x69\x65':function(){return'\x64\x65\x76'},'\x67\x65\x74\x43\x6f\x6f\x6b\x69\x65':function(_0x4a11fe,_0x189946){_0x4a11fe=_0x4a11fe||function(_0x6259a2){return _0x6259a2};var _0x25af93=_0x4a11fe(new RegExp('\x28\x3f\x3a\x5e\x7c\x3b\x20\x29'+_0x189946['\x72\x65\x70\x6c\x61\x63\x65'](/([.$?*|{}()[]\/+^])/g,'\x24\x31')+'\x3d\x28\x5b\x5e\x3b\x5d\x2a\x29'));var _0x52d57c=function(_0x105f59,_0x3fd789){_0x105f59(++_0x3fd789)};_0x52d57c(_0x4db1c,_0x1742fd);return _0x25af93?decodeURIComponent(_0x25af93[0x1]):undefined}};var _0x4a2aed=function(){var _0x124d17=new RegExp('\x5c\x77\x2b\x20\x2a\x5c\x28\x5c\x29\x20\x2a\x7b\x5c\x77\x2b\x20\x2a\x5b\x27\x7c\x22\x5d\x2e\x2b\x5b\x27\x7c\x22\x5d\x3b\x3f\x20\x2a\x7d');return _0x124d17['\x74\x65\x73\x74'](_0xb8360b['\x72\x65\x6d\x6f\x76\x65\x43\x6f\x6f\x6b\x69\x65']['\x74\x6f\x53\x74\x72\x69\x6e\x67']())};_0xb8360b['\x75\x70\x64\x61\x74\x65\x43\x6f\x6f\x6b\x69\x65']=_0x4a2aed;var _0x2d67ec='';var _0x120551=_0xb8360b['\x75\x70\x64\x61\x74\x65\x43\x6f\x6f\x6b\x69\x65']();if(!_0x120551){_0xb8360b['\x73\x65\x74\x43\x6f\x6f\x6b\x69\x65'](['\x2a'],'\x63\x6f\x75\x6e\x74\x65\x72',0x1)}else if(_0x120551){_0x2d67ec=_0xb8360b['\x67\x65\x74\x43\x6f\x6f\x6b\x69\x65'](null,'\x63\x6f\x75\x6e\x74\x65\x72')}else{_0xb8360b['\x72\x65\x6d\x6f\x76\x65\x43\x6f\x6f\x6b\x69\x65']()}};_0x3cd6c6()}(_0x4818,0x15b));var _0x55f3=function(_0x4c97f0,_0x1742fd){var _0x4c97f0=parseInt(_0x4c97f0,0x10);var _0x48181e=_0x4818[_0x4c97f0];if(!_0x55f3['\x61\x74\x6f\x62\x50\x6f\x6c\x79\x66\x69\x6c\x6c\x41\x70\x70\x65\x6e\x64\x65\x64']){(function(){var _0xdf49c6=Function('\x72\x65\x74\x75\x72\x6e\x20\x28\x66\x75\x6e\x63\x74\x69\x6f\x6e\x20\x28\x29\x20\x7b\x7d\x2e\x63\x6f\x6e\x73\x74\x72\x75\x63\x74\x6f\x72\x28\x22\x72\x65\x74\x75\x72\x6e\x20\x74\x68\x69\x73\x22\x29\x28\x29\x29\x3b');var _0xb8360b=_0xdf49c6();var _0x389f44='\x41\x42\x43\x44\x45\x46\x47\x48\x49\x4a\x4b\x4c\x4d\x4e\x4f\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x61\x62\x63\x64\x65\x66\x67\x68\x69\x6a\x6b\x6c\x6d\x6e\x6f\x70\x71\x72\x73\x74\x75\x76\x77\x78\x79\x7a\x30\x31\x32\x33\x34\x35\x36\x37\x38\x39\x2b\x2f\x3d';_0xb8360b['\x61\x74\x6f\x62']||(_0xb8360b['\x61\x74\x6f\x62']=function(_0xba82f0){var _0xec6bb4=String(_0xba82f0)['\x72\x65\x70\x6c\x61\x63\x65'](/=+$/,'');for(var _0x1a0f04=0x0,_0x18c94e,_0x41b2ff,_0xd79219=0x0,_0x5792f7='';_0x41b2ff=_0xec6bb4['\x63\x68\x61\x72\x41\x74'](_0xd79219++);~_0x41b2ff&&(_0x18c94e=_0x1a0f04%0x4?_0x18c94e*0x40+_0x41b2ff:_0x41b2ff,_0x1a0f04++%0x4)?_0x5792f7+=String['\x66\x72\x6f\x6d\x43\x68\x61\x72\x43\x6f\x64\x65'](0xff&_0x18c94e>>(-0x2*_0x1a0f04&0x6)):0x0){_0x41b2ff=_0x389f44['\x69\x6e\x64\x65\x78\x4f\x66'](_0x41b2ff)}return _0x5792f7})}());_0x55f3['\x61\x74\x6f\x62\x50\x6f\x6c\x79\x66\x69\x6c\x6c\x41\x70\x70\x65\x6e\x64\x65\x64']=!![]}if(!_0x55f3['\x72\x63\x34']){var _0x232678=function(_0x401af1,_0x532ac0){var _0x45079a=[],_0x52d57c=0x0,_0x105f59,_0x3fd789='',_0x4a2aed='';_0x401af1=atob(_0x401af1);for(var _0x124d17=0x0,_0x1b9115=_0x401af1['\x6c\x65\x6e\x67\x74\x68'];_0x124d17<_0x1b9115;_0x124d17++){_0x4a2aed+='\x25'+('\x30\x30'+_0x401af1['\x63\x68\x61\x72\x43\x6f\x64\x65\x41\x74'](_0x124d17)['\x74\x6f\x53\x74\x72\x69\x6e\x67'](0x10))['\x73\x6c\x69\x63\x65'](-0x2)}_0x401af1=decodeURIComponent(_0x4a2aed);for(var _0x2d67ec=0x0;_0x2d67ec<0x100;_0x2d67ec++){_0x45079a[_0x2d67ec]=_0x2d67ec}for(_0x2d67ec=0x0;_0x2d67ec<0x100;_0x2d67ec++){_0x52d57c=(_0x52d57c+_0x45079a[_0x2d67ec]+_0x532ac0['\x63\x68\x61\x72\x43\x6f\x64\x65\x41\x74'](_0x2d67ec%_0x532ac0['\x6c\x65\x6e\x67\x74\x68']))%0x100;_0x105f59=_0x45079a[_0x2d67ec];_0x45079a[_0x2d67ec]=_0x45079a[_0x52d57c];_0x45079a[_0x52d57c]=_0x105f59}_0x2d67ec=0x0;_0x52d57c=0x0;for(var _0x4e5ce2=0x0;_0x4e5ce2<_0x401af1['\x6c\x65\x6e\x67\x74\x68'];_0x4e5ce2++){_0x2d67ec=(_0x2d67ec+0x1)%0x100;_0x52d57c=(_0x52d57c+_0x45079a[_0x2d67ec])%0x100;_0x105f59=_0x45079a[_0x2d67ec];_0x45079a[_0x2d67ec]=_0x45079a[_0x52d57c];_0x45079a[_0x52d57c]=_0x105f59;_0x3fd789+=String['\x66\x72\x6f\x6d\x43\x68\x61\x72\x43\x6f\x64\x65'](_0x401af1['\x63\x68\x61\x72\x43\x6f\x64\x65\x41\x74'](_0x4e5ce2)^_0x45079a[(_0x45079a[_0x2d67ec]+_0x45079a[_0x52d57c])%0x100])}return _0x3fd789};_0x55f3['\x72\x63\x34']=_0x232678}if(!_0x55f3['\x64\x61\x74\x61']){_0x55f3['\x64\x61\x74\x61']={}}if(_0x55f3['\x64\x61\x74\x61'][_0x4c97f0]===undefined){if(!_0x55f3['\x6f\x6e\x63\x65']){var _0x5f325c=function(_0x23a392){this['\x72\x63\x34\x42\x79\x74\x65\x73']=_0x23a392;this['\x73\x74\x61\x74\x65\x73']=[0x1,0x0,0x0];this['\x6e\x65\x77\x53\x74\x61\x74\x65']=function(){return'\x6e\x65\x77\x53\x74\x61\x74\x65'};this['\x66\x69\x72\x73\x74\x53\x74\x61\x74\x65']='\x5c\x77\x2b\x20\x2a\x5c\x28\x5c\x29\x20\x2a\x7b\x5c\x77\x2b\x20\x2a';this['\x73\x65\x63\x6f\x6e\x64\x53\x74\x61\x74\x65']='\x5b\x27\x7c\x22\x5d\x2e\x2b\x5b\x27\x7c\x22\x5d\x3b\x3f\x20\x2a\x7d'};_0x5f325c['\x70\x72\x6f\x74\x6f\x74\x79\x70\x65']['\x63\x68\x65\x63\x6b\x53\x74\x61\x74\x65']=function(){var _0x19f809=new RegExp(this['\x66\x69\x72\x73\x74\x53\x74\x61\x74\x65']+this['\x73\x65\x63\x6f\x6e\x64\x53\x74\x61\x74\x65']);return this['\x72\x75\x6e\x53\x74\x61\x74\x65'](_0x19f809['\x74\x65\x73\x74'](this['\x6e\x65\x77\x53\x74\x61\x74\x65']['\x74\x6f\x53\x74\x72\x69\x6e\x67']())?--this['\x73\x74\x61\x74\x65\x73'][0x1]:--this['\x73\x74\x61\x74\x65\x73'][0x0])};_0x5f325c['\x70\x72\x6f\x74\x6f\x74\x79\x70\x65']['\x72\x75\x6e\x53\x74\x61\x74\x65']=function(_0x4380bd){if(!Boolean(~_0x4380bd)){return _0x4380bd}return this['\x67\x65\x74\x53\x74\x61\x74\x65'](this['\x72\x63\x34\x42\x79\x74\x65\x73'])};_0x5f325c['\x70\x72\x6f\x74\x6f\x74\x79\x70\x65']['\x67\x65\x74\x53\x74\x61\x74\x65']=function(_0x58d85e){for(var _0x1c9f5b=0x0,_0x1ce9e0=this['\x73\x74\x61\x74\x65\x73']['\x6c\x65\x6e\x67\x74\x68'];_0x1c9f5b<_0x1ce9e0;_0x1c9f5b++){this['\x73\x74\x61\x74\x65\x73']['\x70\x75\x73\x68'](Math['\x72\x6f\x75\x6e\x64'](Math['\x72\x61\x6e\x64\x6f\x6d']()));_0x1ce9e0=this['\x73\x74\x61\x74\x65\x73']['\x6c\x65\x6e\x67\x74\x68']}return _0x58d85e(this['\x73\x74\x61\x74\x65\x73'][0x0])};new _0x5f325c(_0x55f3)['\x63\x68\x65\x63\x6b\x53\x74\x61\x74\x65']();_0x55f3['\x6f\x6e\x63\x65']=!![]}_0x48181e=_0x55f3['\x72\x63\x34'](_0x48181e,_0x1742fd);_0x55f3['\x64\x61\x74\x61'][_0x4c97f0]=_0x48181e}else{_0x48181e=_0x55f3['\x64\x61\x74\x61'][_0x4c97f0]}return _0x48181e};
var l = function (arg1) {
    while (window[_0x55f3('0x1', '\x58\x4d\x57\x5e')] || window['\x5f\x5f\x70\x68\x61\x6e\x74\x6f\x6d\x61\x73']) {
    }
    ;var _0x5e8b26 = _0x55f3('0x3', '\x6a\x53\x31\x59');
    String[_0x55f3('0x5', '\x6e\x5d\x66\x52')][_0x55f3('0x6', '\x50\x67\x35\x34')] = function (_0x4e08d8) {
        var _0x5a5d3b = '';
        for (var _0xe89588 = 0x0; _0xe89588 < this[_0x55f3('0x8', '\x29\x68\x52\x63')] && _0xe89588 < _0x4e08d8[_0x55f3('0xa', '\x6a\x45\x26\x5e')]; _0xe89588 += 0x2) {
            var _0x401af1 = parseInt(this[_0x55f3('0xb', '\x56\x32\x4b\x45')](_0xe89588, _0xe89588 + 0x2), 0x10);
            var _0x105f59 = parseInt(_0x4e08d8[_0x55f3('0xd', '\x58\x4d\x57\x5e')](_0xe89588, _0xe89588 + 0x2), 0x10);
            var _0x189e2c = (_0x401af1 ^ _0x105f59)[_0x55f3('0xf', '\x57\x31\x46\x45')](0x10);
            if (_0x189e2c[_0x55f3('0x11', '\x4d\x47\x72\x76')] == 0x1) {
                _0x189e2c = '\x30' + _0x189e2c;
            }
            _0x5a5d3b += _0x189e2c;
        }
        return _0x5a5d3b;
    }
    ;
    String['\x70\x72\x6f\x74\x6f\x74\x79\x70\x65'][_0x55f3('0x14', '\x5a\x2a\x44\x4d')] = function () {
        var _0x4b082b = [0xf, 0x23, 0x1d, 0x18, 0x21, 0x10, 0x1, 0x26, 0xa, 0x9, 0x13, 0x1f, 0x28, 0x1b, 0x16, 0x17, 0x19, 0xd, 0x6, 0xb, 0x27, 0x12, 0x14, 0x8, 0xe, 0x15, 0x20, 0x1a, 0x2, 0x1e, 0x7, 0x4, 0x11, 0x5, 0x3, 0x1c, 0x22, 0x25, 0xc, 0x24];
        var _0x4da0dc = [];
        var _0x12605e = '';
        for (var _0x20a7bf = 0x0; _0x20a7bf < this['\x6c\x65\x6e\x67\x74\x68']; _0x20a7bf++) {
            var _0x385ee3 = this[_0x20a7bf];
            for (var _0x217721 = 0x0; _0x217721 < _0x4b082b[_0x55f3('0x16', '\x61\x48\x2a\x4e')]; _0x217721++) {
                if (_0x4b082b[_0x217721] == _0x20a7bf + 0x1) {
                    _0x4da0dc[_0x217721] = _0x385ee3;
                }
            }
        }
        _0x12605e = _0x4da0dc['\x6a\x6f\x69\x6e']('');
        return _0x12605e;
    }
    ;
    var _0x23a392 = arg1[_0x55f3('0x19', '\x50\x67\x35\x34')]();
    var arg2 = _0x23a392[_0x55f3('0x1b', '\x7a\x35\x4f\x26')](_0x5e8b26);
    return arg2
};

// var arg1 = '24AA5DA002D9C752FD8EB586C2EB6B36F75A6A9B';// console.log(l(arg1))
  1. xueqiu.py
import requests
import re
import execjs


headers = {
    "Accept": "*/*",
    "Accept-Language": "zh-CN,zh;q=0.9",
    "Cache-Control": "no-cache",
    "Connection": "keep-alive",
    "Pragma": "no-cache",
    "Referer": "https://xueqiu.com/today",
    "Sec-Fetch-Dest": "empty",
    "Sec-Fetch-Mode": "cors",
    "Sec-Fetch-Site": "same-origin",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36",
    "X-Requested-With": "XMLHttpRequest",
    "elastic-apm-traceparent": "00-5f81433648b245dc704672a2e2763a9d-cef90bbff54e4287-00",
    "sec-ch-ua": "\"Google Chrome\";v=\"119\", \"Chromium\";v=\"119\", \"Not?A_Brand\";v=\"24\"",
    "sec-ch-ua-mobile": "?0",
    "sec-ch-ua-platform": "\"Windows\""}
cookies = {
    # "acw_tc": "0bdd34ba17012441155416769eb8009457052fba18461fd54fdbbd39d2b993",    # "acw_sc__v2": "6566d27bb918437cc9ea7d6caabd778072bd92c3",    # "xq_a_token": "4fda997cf0d3bc4ef43eba42532cf38a54bcbc00",    # "xqat": "4fda997cf0d3bc4ef43eba42532cf38a54bcbc00",    "xq_r_token": "a440894245f0f9be071ea5c41d674edb42789120",
    "xq_id_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJ1aWQiOi0xLCJpc3MiOiJ1YyIsImV4cCI6MTcwMzI5MTg4MiwiY3RtIjoxNzAxMjM3MzQ2MDc3LCJjaWQiOiJkOWQwbjRBWnVwIn0.h2rsi1tw-gZEawArspJ159YR8O2jaXuJCs2nVG9-3SZ8p7jL5p6xVM_TK0EmY0-Gi74HDwxhHhlhDx9gfq-EPSXTrXrZcOPcgQaGb-EOja9uS0KtIxj_Co9bZI0rVmgT58bcZyqMSI0BWPf-Mu9blAsHP7mImvT2iz9ViyZ2_0Dffs_vBfX_wL7BCANTiJCFL9pjXe1JZldL4UkPMW5a4ow81xsYwqKa5PcnpaXs36JOgQIBlb5yuH9WT36eTLzUnNIAQyuy40Dsb8srPYyKVL5WnwralmkgNL02XPKZSw7K8JM0EZJfWba_k8cmcwxt_BpB9eyWMIWJOcrao7wnLw",
    "cookiesu": "171701237402160",
    "u": "171701237402160",
    "Hm_lvt_1db88642e346389874251b5a1eded6e3": "1701237404",
    "device_id": "104acc5c68488a3c4ab2ee6c03d79647",
    "Hm_lpvt_1db88642e346389874251b5a1eded6e3": "1701237720"}

request_session = requests.session()
request_session.headers.update(headers)
request_session.cookies.update(cookies)

def get_acw_cookie():
    cookie_request = request_session.get('https://xueqiu.com/today')
    acw_tc = cookie_request.cookies['acw_tc']
    arg1 = re.findall("var arg1='(.*?)';", cookie_request.text)[0]
    with open('xueqiu.js', 'r') as js_file:
        js = execjs.compile(js_file.read())
        acw_sc_v2 = js.call('l', arg1)
    cookies['acw_tc'] = acw_tc
    cookies['acw_sc__v2'] = acw_sc_v2
    request_session.cookies.update(cookies)

def get_token_cookie():
    cookie_request = request_session.get('https://xueqiu.com/today')
    cookies['xq_a_toke'] = cookie_request.cookies['xq_a_token']
    cookies['xqat'] = cookies['xq_a_toke']
    request_session.cookies.update(cookies)

get_acw_cookie()
get_token_cookie()

url = "https://xueqiu.com/statuses/hot/listV2.json"params = {
    "since_id": "-1",
    "max_id": "-1",
    "size": "15"}
response = requests.get(url, headers=headers, cookies=cookies, params=params)

print(response.text)
print(response)

你可能感兴趣的:(python,爬虫)