JS逆向——破解百度翻译参数（sign）爬虫 超级详细
一、分析请求参数
打开地址：
https://fanyi.baidu.com/
打开Chrome调试工具，然后随意输入一段文字，查看抓包结果。
- post请求
- Form data 参数
- Form data 参数
如上图所示，Form Date中只有两项是会变化的：
- query：我们要翻译的内容
- sign：本文的BOSS出现了，一串意义不明的浮点数值。
- token是不变的，先给查找出来，全局搜索
- 全局搜索发现 token 的值
- token：13508e550366f3004701d561721e12bd
至此，我们便可以初步构造出表单date了以及URL& headers，代码见下：
import requests
import json
import jsonpath
import execjs
url = "https://fanyi.baidu.com/v2transapi"
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0",
"Cookie": "BAIDUID=C3A44BD2BCAA852D78454F118D907CA2:FG=1; BIDUPSID=C3A44BD2BCAA852D78454F118D907CA2; PSTM=1558237711; delPer=0; H_PS_PSSID=1455_21097_29064_28519_28769_28723_28963_28838_28585_29071; PSINO=7; locale=zh; to_lang_often=%5B%7B%22value%22%3A%22en%22%2C%22text%22%3A%22%u82F1%u8BED%22%7D%2C%7B%22value%22%3A%22zh%22%2C%22text%22%3A%22%u4E2D%u6587%22%7D%5D; REALTIME_TRANS_SWITCH=1; FANYI_WORD_SWITCH=1; HISTORY_SWITCH=1; SOUND_SPD_SWITCH=1; SOUND_PREFER_SWITCH=1; Hm_lvt_64ecd82404c51e03dc91cb9e8c025574=1558237732; Hm_lpvt_64ecd82404c51e03dc91cb9e8c025574=1558237732; from_lang_often=%5B%7B%22value%22%3A%22zh%22%2C%22text%22%3A%22%u4E2D%u6587%22%7D%2C%7B%22value%22%3A%22en%22%2C%22text%22%3A%22%u82F1%u8BED%22%7D%5D; CHINA_PINYIN_SWITCH=0; DOUBLE_LANG_SWITCH=0"
}
data = {
"from": "en",
"to": "zh",
"query": "", # query 即我们要翻译的的内容
"transtype": "translang",
"simple_means_flag": "3",
"sign": "", # sign 是变化的需要我们执行js代码得到
"token": "13508e550366f3004701d561721e12bd" # token没有变化
}
二、分析sign —— JS逆向
-
全局搜索 sign
打断点-跟踪函数
-
可以看出来这是应该ajax请求，p里面的参数就是我们要找的参数。
-
在Sourth中找到刚才全局搜索到的js文件，然后搜索界面、打断点
- 打断点后刷新页面，输入字符后页面出现
我们发现 sign是通过m()函数生成的
- 然后鼠标放在m上面悬停一下会出现一个跳转，点击可以跳转到他的函数那里。
- 跳转过后我们看到m函数是
扣代码–生成sign
复制上图中的JS代码，我们新建一个JS文件，命名为： baidu_translata_js.js
js文件最初的模样O(∩_∩)O
function e(r) {
var o = r.match(/[\uD800-\uDBFF][\uDC00-\uDFFF]/g);
if (null === o) {
var t = r.length;
t > 30 && (r = "" + r.substr(0, 10) + r.substr(Math.floor(t / 2) - 5, 10) + r.substr(-10, 10))
} else {
for (var e = r.split(/[\uD800-\uDBFF][\uDC00-\uDFFF]/), C = 0, h = e.length, f = []; h > C; C++)
"" !== e[C] && f.push.apply(f, a(e[C].split(""))),
C !== h - 1 && f.push(o[C]);
var g = f.length;
g > 30 && (r = f.slice(0, 10).join("") + f.slice(Math.floor(g / 2) - 5, Math.floor(g / 2) + 5).join("") + f.slice(-10).join(""))
}
var u = void 0
, l = "" + String.fromCharCode(103) + String.fromCharCode(116) + String.fromCharCode(107);
u = null !== i ? i : (i = window[l] || "") || "";
// var u = void 0, i = null;
// u = null !== i ? i : (i = "320305.131321201" || "") || "";
for (var d = u.split("."), m = Number(d[0]) || 0, s = Number(d[1]) || 0, S = [], c = 0, v = 0; v < r.length; v++) {
var A = r.charCodeAt(v);
128 > A ? S[c++] = A : (2048 > A ? S[c++] = A >> 6 | 192 : (55296 === (64512 & A) && v + 1 < r.length && 56320 === (64512 & r.charCodeAt(v + 1)) ? (A = 65536 + ((1023 & A) << 10) + (1023 & r.charCodeAt(++v)),
S[c++] = A >> 18 | 240,
S[c++] = A >> 12 & 63 | 128) : S[c++] = A >> 12 | 224,
S[c++] = A >> 6 & 63 | 128),
S[c++] = 63 & A | 128)
}
for (var p = m, F = "" + String.fromCharCode(43) + String.fromCharCode(45) + String.fromCharCode(97) + ("" + String.fromCharCode(94) + String.fromCharCode(43) + String.fromCharCode(54)), D = "" + String.fromCharCode(43) + String.fromCharCode(45) + String.fromCharCode(51) + ("" + String.fromCharCode(94) + String.fromCharCode(43) + String.fromCharCode(98)) + ("" + String.fromCharCode(43) + String.fromCharCode(45) + String.fromCharCode(102)), b = 0; b < S.length; b++)
p += S[b],
p = n(p, F);
return p = n(p, D),
p ^= s,
0 > p && (p = (2147483647 & p) + 2147483648),
p %= 1e6,
p.toString() + "." + (p ^ m)
}然后通过编译器（可以执行js的编译器）执行该文件，也可以通过Python中的 PyExecJS 模块执行 js代码
这里是通过python执行js
import execjs
query = 'Hello spider'
with open('baidu_translate_js.js', 'r', encoding='utf-8') as f:
ctx = execjs.compile(f.read())
sign = ctx.call('e', query)
print(sign)
- 问题来了（i未被定义）
execjs._exceptions.ProgramError: ReferenceError: i is not defined
- 我们继续在浏览器中加断点调试 查看 i 值 是怎么生成的
我们在 6819 行中发现i u = null !== i ? i : (i = window[l] || "") || ""; i是通过浏览器的window生成的，我们不是通过浏览器执行js的话，生成不了这个i，不过经过过我们多次尝试就可以发现这里的i值是不变的，所有很好操作。声明一下 i 就可以了
我们愉快的添加了下面这一行代码：
var i = "320305.131321201"
执行结果
紧接着我们再尝试执行函数，又出现错误提示n未定义。故技重施，找到n，发现是一个函数，进入到函数把整个代码扣下来再复制到js代码最前面。
我们再次运行，已经生成了sign值，
三 至此，我们来写代码吧
1 扣好的JS代码
baidu_translate_js.js
function n(r, o) {
for (var t = 0; t < o.length - 2; t += 3) {
var a = o.charAt(t + 2);
a = a >= "a" ? a.charCodeAt(0) - 87 : Number(a),
a = "+" === o.charAt(t + 1) ? r >>> a : r << a,
r = "+" === o.charAt(t) ? r + a & 4294967295 : r ^ a
}
return r
}
var i = "320305.131321201"
function e(r) {
var o = r.match(/[\uD800-\uDBFF][\uDC00-\uDFFF]/g);
if (null === o) {
var t = r.length;
t > 30 && (r = "" + r.substr(0, 10) + r.substr(Math.floor(t / 2) - 5, 10) + r.substr(-10, 10))
} else {
for (var e = r.split(/[\uD800-\uDBFF][\uDC00-\uDFFF]/), C = 0, h = e.length, f = []; h > C; C++)
"" !== e[C] && f.push.apply(f, a(e[C].split(""))),
C !== h - 1 && f.push(o[C]);
var g = f.length;
g > 30 && (r = f.slice(0, 10).join("") + f.slice(Math.floor(g / 2) - 5, Math.floor(g / 2) + 5).join("") + f.slice(-10).join(""))
}
var u = void 0
, l = "" + String.fromCharCode(103) + String.fromCharCode(116) + String.fromCharCode(107);
u = null !== i ? i : (i = window[l] || "") || "";
for (var d = u.split("."), m = Number(d[0]) || 0, s = Number(d[1]) || 0, S = [], c = 0, v = 0; v < r.length; v++) {
var A = r.charCodeAt(v);
128 > A ? S[c++] = A : (2048 > A ? S[c++] = A >> 6 | 192 : (55296 === (64512 & A) && v + 1 < r.length && 56320 === (64512 & r.charCodeAt(v + 1)) ? (A = 65536 + ((1023 & A) << 10) + (1023 & r.charCodeAt(++v)),
S[c++] = A >> 18 | 240,
S[c++] = A >> 12 & 63 | 128) : S[c++] = A >> 12 | 224,
S[c++] = A >> 6 & 63 | 128),
S[c++] = 63 & A | 128)
}
for (var p = m, F = "" + String.fromCharCode(43) + String.fromCharCode(45) + String.fromCharCode(97) + ("" + String.fromCharCode(94) + String.fromCharCode(43) + String.fromCharCode(54)), D = "" + String.fromCharCode(43) + String.fromCharCode(45) + String.fromCharCode(51) + ("" + String.fromCharCode(94) + String.fromCharCode(43) + String.fromCharCode(98)) + ("" + String.fromCharCode(43) + String.fromCharCode(45) + String.fromCharCode(102)), b = 0; b < S.length; b++)
p += S[b],
p = n(p, F);
return p = n(p, D),
p ^= s,
0 > p && (p = (2147483647 & p) + 2147483648),
p %= 1e6,
p.toString() + "." + (p ^ m)
}
2 执行js的python代码
import execjs
query = 'Hello spider'
with open('baidu_translate_js.js', 'r', encoding='utf-8') as f:
ctx = execjs.compile(f.read())
sign = ctx.call('e', query)
print(sign)
3 完成爬虫，解析数据
数据直接转为json数据，（response.json()）
很好解析
结构清晰，很好获取数据
response[‘trans_result’][‘data’][0][‘dst’]
四、封装优化——面向对象编程
#!/usr/bin/env python
# encoding: utf-8
# @software: PyCharm
# @time: 2019/5/22 9:26
# @author: Paulson●Wier
# @file: baidu_translate.py
# @desc:
import execjs
import requests
class BaiduTranslateJS(object):
def __init__(self, query):
"""
:param query: 待翻译的内容
cookie 值 可能会影响到翻译结果，可以将未登陆情况下百度翻译的cookie填写。
"""
self.query = query
self.url = "https://fanyi.baidu.com/v2transapi"
self.headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0",
"Cookie": "BAIDUID=AFB6E3FB47D3EEA8C525D02E728E0991:FG=1; BIDUPSID=AFB6E3FB47D3EEA8C525D02E728E0991; PSTM=1556177968; REALTIME_TRANS_SWITCH=1; FANYI_WORD_SWITCH=1; HISTORY_SWITCH=1; SOUND_SPD_SWITCH=1; SOUND_PREFER_SWITCH=1; MCITY=-131%3A; Hm_lvt_64ecd82404c51e03dc91cb9e8c025574=1556507678,1557287607,1557714510,1557972796; BDSFRCVID=xr-sJeCCxG3twro9YX2saOEfCZPT14fNd2s33J; H_BDCLCKID_SF=tR3fL-08abrqqbRGKITjhPrM2hKLbMT-027OKKO2b-oobfTyDRbHXULELn6TLT_J5eobot8bthF0HPonHj85j6bQ3J; PSINO=2; delPer=0; H_PS_PSSID=1450_28937_21095_18560_29064_28518_29098_28722_28963_28836_28584_26350; locale=zh; yjs_js_security_passport=5b9f340d92cf7400bd5ba82b49a65bc0520935cc_1558490261_js; Hm_lpvt_64ecd82404c51e03dc91cb9e8c025574=1558493551; to_lang_often=%5B%7B%22value%22%3A%22jp%22%2C%22text%22%3A%22%u65E5%u8BED%22%7D%2C%7B%22value%22%3A%22en%22%2C%22text%22%3A%22%u82F1%u8BED%22%7D%2C%7B%22value%22%3A%22zh%22%2C%22text%22%3A%22%u4E2D%u6587%22%7D%5D; from_lang_often=%5B%7B%22value%22%3A%22jp%22%2C%22text%22%3A%22%u65E5%u8BED%22%7D%2C%7B%22value%22%3A%22zh%22%2C%22text%22%3A%22%u4E2D%u6587%22%7D%2C%7B%22value%22%3A%22en%22%2C%22text%22%3A%22%u82F1%u8BED%22%7D%5D"
}
self.data = {
"from": "en",
"to": "zh",
"query": self.query, # query 即我们要翻译的的内容
"transtype": "translang",
"simple_means_flag": "3",
"sign": "", # sign 是变化的需要我们执行js代码得到
"token": "13508e550366f3004701d561721e12bd" # token没有变化
}
def structure_form(self):
"""
读取js文件
执行js代码得到我们苦求的sign值
构造新的表单
:return:
"""
with open('baidu_translate_js.js', 'r', encoding='utf-8') as f:
ctx = execjs.compile(f.read())
sign = ctx.call('e', self.query)
# print(sign)
# sign成功获取，写入date
self.data['sign'] = sign
def get_response(self):
"""
通过构造的新的表单数据，访问api，获取翻译内容
:return: 翻译结果
"""
self.structure_form()
response = requests.post(self.url, headers=self.headers, data=self.data).json()
# print(response)
r = response['trans_result']['data'][0]['dst']
return r
if __name__ == '__main__':
baidu_translate_spider = BaiduTranslateJS('this is a test')
result = baidu_translate_spider.get_response()
print(result)
print(BaiduTranslateJS('this').get_response())
print(BaiduTranslateJS('is').get_response())
print(BaiduTranslateJS('my').get_response())
print(BaiduTranslateJS('love').get_response())
结果
五、总结
百度翻译的JS逆向过程至此就结束了，分析流程和步骤相当清晰，如果帮助到了你，请点个赞呗。