selenium爬ajax评论(一)

http://www.santostang.com/2018/07/15/4-3-%E9%80%9A%E8%BF%87selenium-%E6%A8%A1%E6%8B%9F%E6%B5%8F%E8%A7%88%E5%99%A8%E6%8A%93%E5%8F%96/

用selenium 打开网页

报错:
selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable needs to be in PATH.
https://github.com/mozilla/geckodriver/releases
下载最新版的geckodriver，下载是一个zip文件,解压
然后在环境变量的PATH中，加入这个’geckodriver’的地址, 我就放在项目目录下算了

from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary


caps = webdriver.DesiredCapabilities().FIREFOX
caps["marionette"] = True
binary = FirefoxBinary(r'D:\Program Files\Mozilla Firefox\firefox.exe') # 火狐浏览器地址


driver = webdriver.Firefox(firefox_binary=binary, capabilities=caps)
driver.get("http://www.santostang.com/2018/07/04/hello-world/")

Run 后自动打开网页

找到元素

comment = driver.find_element_by_css_selector('div.reply-content')
content = comment.find_element_by_tag_name('p')
print (content.text)

报错
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: div.reply-content

https://blog.csdn.net/apollolkj/article/details/77096547

最常见的原因就是在ifram里面

加一句
driver.switch_to.frame(driver.find_element_by_css_selector("iframe[title='livere']"))
还是不行 , 猜想是来不及加载

import time
time.sleep(3)

得全部

find_elements_by_css_selector 加个s

comments = driver.find_elements_by_css_selector('div.reply-content')
for eachcomment in comments:
    content = eachcomment.find_element_by_tag_name('p')
    print (content.text)

完整代码

from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
import time

caps = webdriver.DesiredCapabilities().FIREFOX
caps["marionette"] = True
binary = FirefoxBinary(r'D:\Program Files\Mozilla Firefox\firefox.exe') # 火狐浏览器地址


driver = webdriver.Firefox(firefox_binary=binary, capabilities=caps)
driver.get("http://www.santostang.com/2018/07/04/hello-world/")  # 打开网页



time.sleep(3)  # 等一下 等页面加载好
driver.switch_to.frame(driver.find_element_by_css_selector("iframe[title='livere']"))
comments = driver.find_elements_by_css_selector('div.reply-content')
for eachcomment in comments:
    content = eachcomment.find_element_by_tag_name('p')
    print (content.text)