python爬虫表单总结

动态内容，ajax的数据在XHR里面，刷新可以查看新抓的包里面有没有自己想要的数据。

1. IF-TargetVerb:
   POST
2. IF-TargetContent:
   [{"Lbl":"attachmentWrapper","Src":"div.InFlightAttachment:first","Data":"null","HWA":".","Children":[{"Lbl":"attachmentLink","Src":".","Data":"text:href","Children":[]}]},{"Lbl":"popupMessageContent","Src":"span.InFlightPopup","Data":"html","Children":[]},{"Lbl":"item2","Src":"[id=ZZ_VNDR_AD_WRK_DESCR2000]","Data":"value","Children":[]},{"Lbl":"acceptInvite","Src":"#RESP_INQ_DL0_WK_BID_INV_ACCPT_BTN","Data":"id name","HWA":".","Children":[]},{"Lbl":"if_error_items","Src":"div[id*=RESP_ERR_HTMLAREA]:eq(0)","Data":"null","HWA":".","Children":[]},{"Lbl":"error_items","Src":"div[id*=RESP_ERR_HTMLAREA]","Data":"text","Children":[]},{"Lbl":"eventName","Src":"#AUC_HDR_ZZ_AUC_NAME,#AUC_HDR_AUC_NAME","Data":"text","HWA":".","Children":[]},{"Lbl":"eventIdLabel","Src":"div[id*='divRESP_AUC_H0B_WK_AUC_ID_BUS_UNITlbl'] > span","Data":"text","Children":[]},{"Lbl":"eventId","Src":"#RESP_AUC_H0B_WK_AUC_ID_BUS_UNIT","Data":"text","HWA":".","Children":[]},{"Lbl":"eventFormatLabel","Src":"div[id*='divRESP_AUC_H0B_WK_AUC_FORMAT_BIDBERlbl'] > span","Children":[]},{"Lbl":"format1","Src":"#RESP_AUC_H0B_WK_AUC_FORMAT_BIDBER","Data":"text","Children":[]},{"Lbl":"format2","Src":"#AUC_HDR_AUC_TYPE","Data":"text","Children":[]},{"Lbl":"eventStartDateLabel","Src":"div[id*=divAUC_HDR_AUC_DTTM_STARTlbl] > span","Data":"text","Children":[]},{"Lbl":"eventStartDate","Src":"#AUC_HDR_AUC_DTTM_START","Data":"text","HWA":".","Children":[]},{"Lbl":"dept","Src":"#SP_BU_GL_CLSVW_DESCR,#BUS_UNIT_TBL_FS_DESCR","Data":"text","Children":[]},{"Lbl":"eventVersionLabel","Src":"div[id*=divAUC_HDR_AUC_VERSIONlbl] > span","Data":"text","Children":[]},{"Lbl":"eventVersion","Src":"#AUC_HDR_AUC_VERSION","Data":"text","HWA":".","Children":[]},{"Lbl":"eventEndDateLabel","Src":"div[id*=divAUC_HDR_AUC_DTTM_FINISHlbl] > span","Data":"text","Children":[]},{"Lbl":"eventEndDate","Src":"span[id='AUC_HDR_AUC_DTTM_FINISH']","Data":"text","HWA":".","Children":[]},{"Lbl":"desc","Src":"#AUC_HDR_DESCRLONG","Data":"text","Children":[]},{"Lbl":"prevPageZZ_UNSPSC_CD_VW","Src":"a[id='ZZ_UNSPSC_CD_VW$hup$0']","Data":"id name","HWA":".","Children":[]},{"Lbl":"pagePosZZ_UNSPSC_CD_VW","Src":"div[id*='ZZ_UNSPSC_CD_VWGP'] span[class=PSGRIDCOUNTER]","Data":"text","HWA":".","Children":[]},{"Lbl":"downloadPageZZ_UNSPSC_CD_VW","Src":"a[id^='ZZ_UNSPSC_CD_VW$hexcel']:eq(0)","Data":"id name","HWA":".","Children":[]},{"Lbl":"nextPageZZ_UNSPSC_CD_VW","Src":"a[id='ZZ_UNSPSC_CD_VW$hdown$0']","Data":"id name","HWA":".","Children":[]},{"Lbl":"licenseRow","Src":"tr[id^='trZZ_UNSPSC_CD_VW$0_row']","Data":"id","HWA":".","Children":[{"Lbl":"licensecode","Src":"[id^='ZZ_LICENS_CD_VW_LICENSE_CODE$']","Data":"text","Children":[]},{"Lbl":"licensedesc","Src":"[id^='ZZ_CLS_CD_VW_DESCR254$']","Data":"text","Children":[]}]},{"Lbl":"prevPageZZ_AUC_SA_TBL","Src":"a[id='ZZ_AUC_SA_TBL$hup$0']","Data":"id name","HWA":".","Children":[]},{"Lbl":"pagePosZZ_AUC_SA_TBL","Src":"div[id*='ZZ_AUC_SA_TBLGP'] span[class=PSGRIDCOUNTER]","Data":"text","HWA":".","Children":[]},{"Lbl":"downloadPageZZ_AUC_SA_TBL","Src":"a[id^='ZZ_AUC_SA_TBL$hexcel']:eq(0)","Data":"id name","HWA":".","Children":[]},{"Lbl":"nextPageZZ_AUC_SA_TBL","Src":"a[id='ZZ_AUC_SA_TBL$hdown$0']","Data":"id name","HWA":".","Children":[]},{"Lbl":"serviceRow","Src":"tr[id^='trZZ_AUC_SA_TBL$0_row']","Data":"null","Children":[{"Lbl":"areaId","Src":"[id^='ZZ_AUC_SA_TBL_ZZ_SRVC_AREA_ID$']","Data":"text","Children":[]},{"Lbl":"county","Src":"[id^='ZZ_SA_VW_COUNTY$']","Data":"text","Children":[]}]},{"Lbl":"eventContactLabel","Src":"div[id*=divAUC_HDR_NAME1lbl] > span","Data":"text","Children":[]},{"Lbl":"eventContact","Src":"#AUC_HDR_NAME1","Data":"text","HWA":".","Children":[]},{"Lbl":"eventFormatLabel","Src":"div[id*=divAUC_HDR_EMAILIDlbl] > span","Children":[]},{"Lbl":"emailAddress","Src":"#RESP_INQ_DL0_WK_EMAILID","Data":"id text","Children":[]},{"Lbl":"eventPhoneLabel","Src":"div[id*=divAUC_HDR_PHONElbl] > span","Data":"text","Children":[]},{"Lbl":"eventPhone","Src":"#AUC_HDR_PHONE","Data":"text","HWA":".","Children":[]},{"Lbl":"conferenceRow","Src":"table[id^='ACE_ZZ_BID_CNF_VW$']","Data":"null","Children":[{"Lbl":"mandatory","Src":"[id^='ZZ_BID_CNF_VW_COMMENT1$']","Data":"text","Children":[]},{"Lbl":"date","Src":"[id^='ZZ_BID_CNF_VW_DATE1$']","Data":"text","Children":[]},{"Lbl":"time","Src":"[id^='ZZ_BID_CNF_VW_DUE_DT_TIME$']","Data":"text","Children":[]},{"Lbl":"location","Src":"[id^='ZZ_BID_CNF_VW_DESCR254_1$']","Data":"text","Children":[]},{"Lbl":"comments","Src":"[id^='ZZ_BID_CNF_VW_DESCR254_MIXED$']","Data":"text","Children":[]}]},{"Lbl":"prevPageZZ_UNSPSC_CD_VW2","Src":"a[id='ZZ_UNSPSC_CD_VW2$hup$0']","Data":"id name","HWA":".","Children":[]},{"Lbl":"pagePosZZ_UNSPSC_CD_VW2","Src":"div[id*='ZZ_UNSPSC_CD_VW2GP'] span[class=PSGRIDCOUNTER]","Data":"text","HWA":".","Children":[]},{"Lbl":"downloadPageZZ_UNSPSC_CD_VW2","Src":"a[id^='ZZ_UNSPSC_CD_VW2$hexcel']:eq(0)","Data":"id name","HWA":".","Children":[]},{"Lbl":"nextPageZZ_UNSPSC_CD_VW2","Src":"a[id='ZZ_UNSPSC_CD_VW2$hdown$0']","Data":"id name","HWA":".","Children":[]},{"Lbl":"unspscRow","Src":"tr[id^='trZZ_UNSPSC_CD_VW2$0_row']","Data":"null","Children":[{"Lbl":"category","Src":"[id^='ZZ_CATGRY_CD_VW_CATEGORY_CD$']","Data":"text","Children":[]},{"Lbl":"categoryDescription","Src":"[id^='ZZ_CAT_DESC_VW_DESCR60$']","Data":"text","Children":[]}]},{"Lbl":"postAdButton","Src":"#ZZ_VNDR_AD_WRK_VENDOR_LNK","Data":"id name disabled","HWA":".","Children":[]},{"Lbl":"viewAdButton","Src":"#ZZ_VNDR_AD_WRK_VENDOR_DETAILS_PB","Data":"id name disabled","HWA":".","Children":[]},{"Lbl":"startBidButton","Src":"#RESP_INQ_DL0_WK_AUC_RESP_ADD_PB","Data":"id name disabled","HWA":".","Children":[]},{"Lbl":"windowNum","Src":"form[name^='win']","Data":"name:text","Children":[]},{"Lbl":"hiddenInput","Src":"input[type=hidden]","Data":"id name value","Children":[]}]
3. IF-Template:
   /pages/Events-BS3/event-details.aspx
4. ICType:
   Panel
5. ICElementNum:
   3
6. ICStateNum:
   1
7. ICAction:
   ZZ_UNSPSC_CD_VW2$hexcel$0
8. ICXPos:
   0
9. ICYPos:
   0
10. ResponsetoDiffFrame:
    -1
11. TargetFrameName:
    None
12. FacetPath:
    None
13. ICFocus:
14. ICSaveWarningFilter:
    0
15. ICChanged:
    -1
16. ICAutoSave:
    0
17. ICResubmit:
    0
18. ICSID:
    rc8SOxfZQLGGijgk+9e8A/NVzhxcJTZVwbMXX5w0Mic=
19. ICActionPrompt:
    false
20. ICBcDomData:
21. ICFind:
22. ICAddCount:
23. ICAPPCLSDATA:
24. AUC_LOT_LS$hnewpers$0:
    0|0|0|0|0|0|0#1|0|0|0|0|0|0#2|0|0|0|0|0|0#3|0|0|0|0|0|0#4|0|0|0|0|0|0#
25. AUC_LINEGRD$hnewpers$0:
    0|0|0|1|3|57|0#2|0|0|0|0|44|0#3|0|0|2|1|394|0#4|0|0|0|0|57|0#5|0|0|2|4|87|0#6|0|0|0|0|0|0#7|0|0|0|0|0|0#8|0|0|0|0|0|0#9|0|0|2|2|83|0#11|0|0|0|0|0|0#
26. RESP_DTL_GR1_WK$hnewpers$0:
    0|0|0|0|0|0|0#2|0|0|0|0|0|0#3|0|0|0|0|0|0#4|0|0|0|0|0|0#5|0|0|0|0|0|0#6|0|0|0|0|0|0#7|0|0|0|0|0|0#8|0|0|0|0|0|0#

我今天想用表单formdata里面的数据对请求连接进行请求，结果有返回的数据

1. {Info: {ProposedTemplate: "/pages/Events-BS3/event-details.aspx",…},…}
   1. CaptureResults:{,…}
   2. DebugInfo:{SourceContent: null, SourceHeaders: null, Exception: null}
   3. Info:{ProposedTemplate: "/pages/Events-BS3/event-details.aspx",…}
   4. ProposedTemplate:"/pages/Events-BS3/event-details.aspx"

但是并没有返回我想要的信息，所以表单提交的数据肯定有问题，认真分析各种同类网站以后有些数据是动态的，可以对上个网站的json进行筛选，并添加，再次访问，成功获取数据

self.form['ICSID'] = jsons['CaptureResults']['hiddenInput'][14]['Properties']['value']
self.form['AUC_LOT_LS$hnewpers$0'] = jsons['CaptureResults']['hiddenInput'][20]['Properties']['value']
self.form['AUC_LINEGRD$hnewpers$0'] = jsons['CaptureResults']['hiddenInput'][21]['Properties']['value']
self.form['RESP_DTL_GR1_WK$hnewpers$0'] = jsons['CaptureResults']['hiddenInput'][22]['Properties']['value']

查看cookie是否需要加上，可以在网址栏的左方“查看网站信息”里面进行访问，关闭cookie，如果访问不了，需要加上cookie