JS逆向爬虫----响应结果加密①
示例----响应结果加密分析
在浏览器内打开主页链接https://jzsc.mohurd.gov.cn/data/company我们可以访问到以下页面
我们需要获取到图片中标红的数据。
定位需求数据
通常的思路是通过浏览器抓包和关键词搜索方式定位数据接口，我们来尝试一下：
通过f12刷新页面并搜索关键词 91610800064834709T，结果没找到包含关键词的数据。更换一下关键词为 榆林永邦建设工程有限公司
依然没有定位到数据接口！
以上的情况，可能发生的原因：页面内的数据是通过js处理响应结果生成的 即接口的响应结果并不是简单的数据包含在json内而是通过反爬手段将响应结果加密了，并通过前端js处理后展现在网页内。
为了验证这个猜想，我们可以点击其他页，抓包xhr的请求的数据：
我们通过xhr抓包测试，发现点击不同页面，浏览器异步请求接口：https://jzsc.mohurd.gov.cn/APi/webApi/dataservice/query/comp/list?pg=3&pgsz=15&total=450，不同页面仅仅只是查询参数不一致而已。
查看响应
响应为很长的字符串：95780ba0943730051dccb5fe3918f9fe1b6f2130681f99d5620c5497aa480f13…
点击不同页面唯一的xhr请求只有这个，大概可以确定我们需要的数据是通过解密这个响应的字符串获得。
js定位响应结果处理
再次f12刷新页面抓包，搜索关键词：json.parse
定位到3个js文件，我们依次进入js文件，再次搜索关键词json.parse，在包含关键词的位置打上断点，点击下一页，查看是否在断点处停顿，同时伴有关键信息展示出来。
app.248文件断点调试
我们进入来源面板，搜索关键字json.parse,并在有关键字的位置打断点。
断点位置如下：
点击下一页，查看是否有关键信息：
查看到t.data就是接口响应的数据，h(t.data)好像就是解析之后的json，包含需要的数据。
h函数应该就是将加密数据解析为可用json的方法，只要把h函数还原，我们就能获取到信息。
h函数还原
点击h函数，进入h函数定义位置：
h函数的内容如下：
f = d.a.enc.Utf8.parse("jo8j9wGw%6HbxfFn")
, m = d.a.enc.Utf8.parse("0123456789ABCDEF");
function h(t) {
var e = d.a.enc.Hex.parse(t)
, n = d.a.enc.Base64.stringify(e)
, a = d.a.AES.decrypt(n, f, {
iv: m,
mode: d.a.mode.CBC,
padding: d.a.pad.Pkcs7
})
, r = a.toString(d.a.enc.Utf8);
return r.toString()
}
这里有关键词AES,mode: d.a.mode.CBC,padding: d.a.pad.Pkcs7
根据AES加密算法的加密过程，我们知道aes需要密钥key,初始向量,以及加密模式和填充模式。通过观察代码，我们猜测加密的密钥为jo8j9wGw%6HbxfFn，初始向量为0123456789ABCDEF，加密模式为cbc,填充方式为pkcs7。我们可以通过在线aes解密网站验证猜想：
通过测试字符串，同时加入指定的密钥jo8j9wGw%6HbxfFn填充0123456789ABCDEF ，解密后发现为我们需要的包含数据的json
这个h函数，我们可用使用nodejs还原。
nodejs实现解密函数
js代码测试
// 解密方法 nodejs版
const crypto = require('crypto');
const key = Buffer.from("jo8j9wGw%6HbxfFn", 'utf-8');
const iv = Buffer.from("0123456789ABCDEF", 'utf-8');
function decrypt(t) {
const hex = Buffer.from(t, 'hex');
const base64 = hex.toString('base64');
const decipher = crypto.createDecipheriv('aes-128-cbc', key, iv);
decipher.setAutoPadding(true);
let decryptedData = decipher.update(base64, 'base64', 'utf-8');
decryptedData += decipher.final('utf-8');
return decryptedData;
}
// 使用示例
const encryptedText = "95780ba0943730051dccb5fe3918f9fe1b6f2130681f99d5620c5497aa480f13068063ac378e2b22caa5bb9dfd753cdfc5e3e7970c1c42cd2a329175a20ff189d767bbb15783ec2788514321fbd15912c2605cb412f9da2effa938b6e965697dd4eb0c3dd446fa74f442a555e8113669869e0c74d3ea1ebc64d57474386bc4f95148a1ebb3241b420d1e19c674c9299644724a15d1b6add7d11944e08810b676f5e6cd39b45e63f0a7ae0a7219221e3828702fe3fab6a2d6f71d599b15c1d32e2dc217f901d0a3444ffbd2d4bd9652d49e7d152c4b4e8b993046bb33af2e81f01434394d3c56e01297fa8dd8ddedc1ff066c112f8a9717056c220f619bdbb3887cd13ccd2d68a9eda6f53008189b82ccc47b61ea18ed634c0f29973c4de8a2645edef20d3a031bc1e54fb8814b49d3a102d72f42d5596158a00226be2654fe7bcb054dbeb6dfc91c3337e16da02195a589dd4cdd9974bf0a94df313f9e3aa18f4dc3064978d8486c0fbe331e394a1d778f2b57f66b5c180d3f03954aa1894c18f65335e3ab103abdd5621214fb893e6be0471db335df6f27ae0b91c1f8c86f50304daf396658354c6c74bd8587befcb0c488fc0d7adb8eccaa4575b68cc4775236a18f225c07228972adb59cbd6a5c35b874992d36ebbd6cf8292716cc09cc0186409108aaf5bff78aa58975fd225d5b092d74461a2592e0ff5db14224f809f47f16a9e3edffb77e90f652e8542c7d49cd1191da01229ed00391890e84911b0a3664cbc4b7d8f281e9afe4a69b42a74cb2349d5191b6dd9d6f6e19d302dfcfa8b815bdd567a60a3bbae978123f238a6ae150a101a672700f749dc59b473157383a296c9c6009516d52c489080f572d90197343669c10d54508c68d5767e8f6df06b0bdd8cf7a3b16661ac1b4c3446974739c1c83a18b0ea927977cae5dd6b59b369d8b36c45c4188418a8671a57640065089cd74c99fbb5f2f10f7464f7ca452c4222b0e3b3e12d9f482ba9778bf2b8ad699514a4ea0b4761974cd717579e44ca2cbc5b89318adb7e811cd5f129977a0a7d265e8daf625ff7d6989fc73d403b44ab23d46b7401781718d8966fc093bc37c6f8bd9f411e49fda60fdfde81ee6532e7016d13f5dfe32ec406809a8ed4c39b6634d365f9006741efb41bdab6fe36d78637307e39e5a56b7f6ad0d72126ebd4b7e928efcb79b2ff416263d9925fdd658f247c482b5c9b93a5fd2610adc09ca154ac8863d75857994eeea063af680e4c6b45a3c2f2020314e23de479b7b6aaa100c0fd17f59d059b049cacfbf0ef5896db63edf5cf2dd594f12d782c3fbd587cb27c7d76ae3c2bbcf27e557445490a00277c08010c9a0a12225ee5c83c4eb05d33aa96fd19de42068b65c557f40299566a984a3d21006010b5617403b0e7c93f01b0a83fccc93f6b986eea2fe2e94ade222cd1c68bf9cbf9b81a96a0245f436e98d7d0208790812181f2983f6f585bbc4579ef18aabdc6c15c93d53d7316d114f5025ccb45d4381e412ea6dc574ca413834119131283b11706774d43ccb266d29003145005f9df6fff27cef6d915e92d61f7ac28c76a8ec581ccc149f877239f9ac493aff309eeedee370963c9c7eb63b60f994f5efea433a40ba9364894556d71f486eb27e7a6e65ee7e9049c927e9eba7a47ccb6b4f375b53d07e7a5d49d68d466c385cae10aedfbfb62916a73217b60290c038273ce111c154579b8176de01b0df13f29d087039194ea0c243c7bea2f2e4548cbad88fbb61fa924b16da4aee02e474b581f556bc9c2c17f6580103c237de0a8ea935d31f291603c4b1247bb72c3bf4bf4bd64e8a0259413cbc55fd9f3fd129bc8e8de7ab625837b7217546ffd673872c91659d8f3a4ecd661ea162fa24107ea2a678408a72f2cbc03bd0af18e7ae9aa1f8b43553ab79657695767d08c7d43652206d2f3a72c8828024c58f8afcbfda1f42208989facad44af380aac349b1c13d5ce91831e5551d24bf149313d52d6a996a031ec8c0e970c15b7d18fa61e449031a12724dee467565688312e7b8eff4f5bd3ddb9272771557f1e1b2587f56a5a5caa36881fc84395bc816c5eac5d6e82f18c44d51bf5f9230a8804ed7cf87c37078e0fe670ac61e1e6335d4e487ea19fe4130dbaba5f3f10dfeb806825481dad39aac444cb42fd58d854180ad13afc7dbee071629b2286457c7019a499d38f515af629afb377c3a79011f0358e2e11fbe3464191923815090fe578ef24c0387b005803c22e0d5df2b29bf4dd4bd0a810f1fd4f74cd4dc5a781d1010bd9abd19063c894a36860c0990ce0d2e3f46f661338ef9e2ad5128ec4c71f9d07fb933c0ed1ecbd4386bca209153485ad2bcc3ac5f18b48073776a7c5d697f6200caf703d0088139f0241dc9f5020634cf85ddf7019205d727c34dc3bafe80a428df3e15f2d58616c501aebb67c7d770e50f3a5bc76cbf3fd6919069756e7e31b9751bb7bec5fe4df71803f6a1b63a61366b93a31e02f147a40350ca2ccf2a06a3ed47c1d9f6da3dfd4d311103b60eb2ba5175055ababb477b1c3f5ddba461c36654023debbe45acf4e6c850ccd1080f68d5788b5b242ad56dbf2b7c3d525f4eb20231f05e9b35d5594cdd8280279cacbe0ef67dcddd1c523b8ebad06415943c8a0ae0c401c7968cf6ca800e4eca168e337a15afbfd6988aeab30bfb77f76871313ac9fdef9b6e988d83010f68b167223ce59bc8db7845fb2f2837b2bf99b11ef1e9cf9499f91e65ab1aa830998d72a2ee8ee28cca4ad0d1ebfd62e0876e48faf77797d11b0e8c8b14a4b92781ef4c1c6f9d51224aa499762e84c29618a21e38ccdbf9f9eae7e3cae121095a620aa940fe12aec580d2f4a89c59560f562be177f825f63577fb894738f9b41c1fcd5fbbdd5afc56b2b0980c96e026f74cb3574ff555a81f566bf053690d3d0ad096a84ae533570f41ef0f555d9e0fa189a13e73af44600e4bb1345951b64e4a209e17e3fdb717d888faa11135ffa4211b1d99c5473dd665cb6267db9d9d326701a4517f946ba8e0881bfbb67e27e88f19aeb220de4427cd8adb273c0c438a5b2743d7b4d67e580f58801999704d4565c310402c0ce751197005f464cf0984c91b0b7cbb094fff958dc31c3204d750e593fa4022b4b9c2af9076d69467b68af6664f1402fb422b6912f8767aa767ef5fd72a44c2f78060333f7f6b8ec41a50089cd74892b2e5490380e79f6eb6c479c6394da61a8dfd2f9ac1c9b72fa0a0a28905fb0da1c74a7fd550d3ebaa6102b577c697e9dcd102874e2f0ea5d7e558ab46dbc05d1df88ad360e8daf140a9a9484f882ecc1368083c308faae156351138f9f1f26728b0b0e1532baffbaeb97db7a5f3f1b61d523f4dbd10f00a2d1e975eb4bf5d1b2311e9f381e23916cb1ecc5bd9baa9b053b8573d291c93e644575fe97d4f38ead41e4fc43d3290a5321dd25141a9de63ce919fdd234e56c035d59a23223760b70e04ca361eb5e80e211d561cf5e9ab6d2e4fd84783592e294afa5f7dd63b18804b1a26996cac68063ecac4f083a28708bc383ce6a94ec090da1558eea51e5759dc0be637329299809c1a7557bd26c410355fd9a4c5e8e3c0ede1bb22d4e4ea54c0944daa6eda4dc6e973acb46b4e9ce390812644f2de00612a94e384998957454d833a4d321dd479d29c4e548a24f3eb9664054931866f842a166154b276867581202491b8ec281c8590f65bfc3ad80042c0e0cd497a191500299848da6354152e16a93599cf847794d6d93ddb14e1170413c8333a43ec5b2b468c4e669fbfed47ca654ad38041bbbce559d3074b150ce92581b46b4a27fe48d50d5e43b81efd03dbe6e9fc44db6a3ae4132ef8bc1f37600809f1dacddec02877b60d9b3bfd49a01a7e1f671cbf11e04fb4e730fc45a1e2c556db38f0c12237f10055490c3e5a5e1efd692ff999728065fe9729e47ff1a70e8c71befddc2a8a78b07ba01bfe15b07242e8bdfcf2c0c39281fc5eb4b8d44a70fcf95fb9349040146a68a46f35d9fe6ff4bf5806acecc6fcd6e49c8c2eaee2d7059a1ecbe71d16337b6f8735980f039b22526cf38b66e05a141347d98b1a41a5bd229740a87ba236aa59d4b96d272f8a6638d74efd7955fc7764b936e7ec8d3ffc3026427188475522063a4b0b84793ed4294ebd665ac0f7e346ac5692ce35d2748dca1fe4163442eb0e1e17cc9d3ef7ba9b509b40dd2da647ea1c8ec526f4321b87aa31757c7370f6eb69d4a69a7ba02581f47f2b39c5c94af59744a3c0619ee7cfa5e68db748556814e974b343a9daed5542a962770e729f69e2bab37ba41af022bb8f2bb392f9c417ce240a83d8e155069b74588de47f3dd0187a66cbc6b560deddb643521d08c71e9468b3ccef3c564c0a28de00958c4ca13847e05d7473525d3d51dfbc41f6ad221997c3c586295cc41b3d2e25a91df6deef7f8f6ced9e548f92aeae3466d1788fff476ce2227a3f9363076639edee246727840e805a656c7e65f9cb38b19475e74b8b58e02ee35ede5cb6c8423861542b4b71c590c4458afa327060b23fb3f64e4a98d10bb74abbbc74f0f7efd08a0286bc211d6c10afaf5e866f4f86e8ddbb935874f23249444a6620c452ea0cadfff89ca651318485f9a5b92e706e2005df3ee40030132e4289410e2591383f0edd6dccaec2b1d74f301608874ed74d76b966ba0978fe41b53806d323bf2ebd0ed38ae93cf0cf9b0e63d83fab7ea9a9deddb41428a4df70a891c6b4f31ab14c9b64b936bd40f4341fa29ef1816e9b85689326bdcfdd78567dba4b40a87c5ba8b201f93c32bdf6d2c9dab8b0fbc76ecad5bbbaccaff6aa7de44be597cb937699c08f53fe61fd33d8bc8d8f4df58d6e8dd21234c86bb3afe99595616615ff59404f69c6bf984a18b39e845c0e49c33aafdd5c0e34e55d26e8a41b46ceb7fdee11bd387c9f0abc53809b7dcaf6a85c1d8bb36a01412fe36abaceaff78be5abab0969f13facf16bfc3994323060503cf7b00932e449e2f7304dc6931b0c41ed1ebe2ff86e0c00f09d499a6b1902a7bf5c9f650051185c0cd9df8dcc1d92ec35d015531f7075a3609bd6a8a57e4dace91ec7edd89ebf4d5af1d91cdd961feb4b10b1f44c883095b9a741815424584f385e745416656af87272eca7bb3323649ef9e1caf0089c93a2bdcee2f0260caf7f4bd3242cd6a290b166de125a9fee0a2800a122cf8df47b8d6aad6d63ec7ef6233b56340dd6f6dd37e6e96bdd1e303998713de5ee0ee80201d5b3a4a63131997f9eb76efed42353b0967ab437e0ed50ed04cc6fb16fe6342c8d03169fb9667d4f90ba0008c80dd7e71d25a2e50a8fd46bf35faef2a4bbd2072dfd7248b409451ee0e52dccb811407527dcdd9ad5c9ca9e7bb5d022d2518430ad2ebcd0de072c86d85f346bff44cbff0a8467686e9e63729f4a50128826c346b2909a3fac49702e3a4249edec66e927a161d9d8a4d579e7b72711eab250a4d772f7e96e4aecb69286cbeb4991abd13fcdbba10e71a70341ca894b2ea1917e61ce0e7261a2520f688afc0ae34b5c7881eab09e6abbcce1091c4f3e3ddaea5adddc4d566263f60d7cd4bc2debc2dfb3951503c958e4294588c7983471c9eb2315c41dcb56d5754e01011a92cb98f08a2220b22ef5c576d70d9554d446188c792c9c9f0d25b952966d40bdfde530e690090f3992f0ff29a77198547168a11d861458c1a60be30e5a38f52902f743108a5d12d94b2bd7c562d4f44e74c134138a3e5be263ef1bfb3e4621bb60ed8c11cca90a731f56e52813403648de8567a9207a0a986eb2791f7b4caf62e098c85580d7ada64d0606fdc4c60fd0124856c73298746bb062efa7ec60d4505a1b96f3dc9a06b5c138b9402bda137218eb30273ee3366dde186f4aab5bb249cf1ea3c0cce86f62a9715514337e0f048211dd6cd218779cb1db0b23491be51056c555319d3514704c70d33a5475dc75552c2780ccf2654691bf66518dade9891e439acf0b52a85d2f9a1142e37432519dc7aac360eaaf248aee0e56039bc7f1efe64c93bfabfaeb5755654ffdb470887a4c485dd8ce9afd02db3b08f0d312f716066d8f21a292a7e10bbcb718b624ee70f1d490297502d12e24e38e8e74da7f2b17d2758d3fcfa630bbca152766fb3d5409d2e61d4c4b4ec4b18dc600f7bfaee559d35bc3fb9a72422fe8cd55a4d938cf8804155ed0e257302b018c7992a676aaa393113600b6a7bdeb3bd64043f637c01107baa29d3a6309a6e704f009ef90c95b9f19022646bfd1e196412edf83b14b92425cd46b3f835408699dced6c9583590fc123a8fa057c156fafb7885bee6983eb283a0073588ddce9c6511f3aa171a2d91dc04598c1964e86dd99ee3c01dd2a549340a2c65c018eaafe35f9fe3c32c8920aa9897d928b5322926665ec2e1042ce72cf0829fa7092cf09d9d74c31a279e0be703a2ec8127b1362707c37bd1737f2a156b7233f0704af437a61c335a6bd38d802740f38545310fdc9d789b6474fa08a307d74390687cf6ca12c24799c811a679c1d49f0e";
const decryptedText = decrypt(encryptedText);
console.log("解密后的文本:", decryptedText);
js测试结果
以上代码我们只需要将响应结果赋值给decryptedText，即可解密成功。