网络爬虫_网页登录（苏宁 有验证码）—基于HtmlUnit

输入验证码：

public static String getCode(){
    System.out.println("请输入验证码：");
    Scanner sc = new Scanner(System.in);
    String code = sc.nextLine();
    return code;
}

初始化WebClient方法：

public WebClient getClient(){
    webClient = new WebClient(BrowserVersion.FIREFOX_24);
    webClient.getOptions().setCssEnabled(false);
    webClient.getOptions().setJavaScriptEnabled(true);
    webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
    webClient.getOptions().setThrowExceptionOnScriptError(false);
    webClient.setAjaxController(new NicelyResynchronizingAjaxController());
    webClient.getOptions().setThrowExceptionOnScriptError(false);
    webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
    return webClient;
}

爬取数据代码示例：

/**
 * Create by lingfx on 2017/8/18
 */
public class SuNingLoginTest {

    private Logger logger = Logger.getLogger(SuNingLoginTest.class);
    private WebClient client;
    private String username = "用户名";
    private String kkk = "密码";

    private String filePath = "E://code.jpg";

public void login() throws IOException { client = new WebClientTool().getClient(); HtmlPage htmlPage = client.getPage( "https://mpassport.suning.com/ids/login?service=https%3A%2F%2Fscs.suning.com%2Fsps%2Fauth%3FtargetUrl%3Dhttp%253A%252F%252Fscs.suning.com%252Fsps%252Fmember%252Flogon.do&loginTheme=scs"); // logger.info(htmlPage.asText()); HtmlInput un = htmlPage.getHtmlElementById( "username"); un.setValueAttribute( username); HtmlInput pwd = htmlPage.getHtmlElementById( "pwd"); pwd.setValueAttribute( kkk); HtmlImage codeImg = htmlPage.getHtmlElementById( "vcodeimg1");

	codeImg.saveAs(new File(filePath));
	logger.info("验证码存储位置："+filePath);

String code = ScannerTool. getCode(); HtmlInput codeInput = htmlPage.getHtmlElementById( "validateCode"); codeInput.setValueAttribute(code); ScriptResult res = htmlPage.executeJavaScript( "javascript:logon();"); HtmlPage page2 = (HtmlPage) res.getNewPage(); logger.info(page2.asText()); if(page2.asText().contains( "您好！欢迎回来！")){ logger.info( "登录成功！"); } } public static void main(String[] args) throws IOException { SuNingLoginTest test = new SuNingLoginTest(); test.login(); }}

已测试，无异常。

2017-8-18