selenium+python+browsermob-proxy获取network中请求和响应的方法

1）安装browsermob-proxy：

Browsermob-Proxy是一个开源的Java编写的基于LittleProxy的代理服务。Browsermob-Proxy的具体流程有点类似与Flidder或Charles。即开启一个端口并作为一个标准代理存在，当HTTP客户端（浏览器等）设置了这个代理，则可以抓取所有的请求细节并获取返回内容。这个工具类似于fiddler和charles

安装：

直接到项目的github上下载打好的压缩包即可:https://github.com/lightbody/browsermob-proxy/releases ，支持Linux和Windows。

安装对应的python包命令：

pip install browsermob-proxy

2）准备脚本：

from browsermobproxy import Server
server = Server(r'D:\browserproxy\browsermob-proxy-2.1.4\bin\browsermob-proxy.bat')
server.start()
proxy = server.create_proxy(params={'trustAllServers':'true'})   # 有些时间需要加上这个参数，否则可能页面无法打开
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.chrome.options import Options

# # # todo firefox
# profile = FirefoxProfile()
# option = webdriver.FirefoxOptions()
# # option.add_argument("--headless")   #无界面模式
# # option.add_argument("--disable-gpu")
# option.add_argument('--proxy-server={0}'.format(proxy.proxy))
# driver = webdriver.Firefox(profile, firefox_options=option, executable_path=r'geckodriver-v0.26.0-win64\geckodriver.exe')

#
# profile  = webdriver.FirefoxProfile()
# profile.set_proxy(proxy.selenium_proxy())
# driver = webdriver.Firefox(firefox_profile=profile,executable_path=r'geckodriver-v0.26.0-win64\geckodriver.exe')
#


# # # # todo chrome
chrome_options = Options()
chrome_options.add_argument("--headless")   #无界面模式
# chrome_options.add_argument("--disable-gpu")
chrome_options.add_argument('--proxy-server={0}'.format(proxy.proxy))
chrome_options.add_experimental_option("useAutomationExtension", False)
chrome_options.add_experimental_option("excludeSwitches", ['enable-automation'])
chrome_options.add_argument('--ignore-certificate-errors')
# driver = webdriver.Chrome(chrome_options=chrome_options,executable_path='C:/chromedriver_win32/chromedriver.exe')
driver = webdriver.Chrome(chrome_options=chrome_options, executable_path=r'。。。。。。\Programs\Python\Python37\Scripts\chromedriver.exe')

proxy.new_har("", options={'captureContent': True, 'captureHeaders': True, 'captureBinaryContent': True})  # 最后一个参数很重要，有些结果需要配置这个参数
url='https://www.baidu.com/'  #https://api-test.bimface.com/inside/preview?viewToken=43247a965e2b4eca9eb86fa3c8490baa
driver.get(url)
proxy.wait_for_traffic_to_stop(1, 60)
result = proxy.har
#performance_data=driver.execute_script('return window.performance.getEntries();')
print(result)  # 分析result结果
#print("performance_data",performance_data)

navigationStart = driver.execute_script("return window.performance.timing.navigationStart")

responseEnd = driver.execute_script("return window.performance.timing.responseEnd")
domComplete = driver.execute_script("return window.performance.timing.domComplete")

loadEventEnd = driver.execute_script("return window.performance.timing.loadEventEnd")

backendPerformance = responseEnd - navigationStart

frontendPerformance = loadEventEnd - responseEnd
domperformance = domComplete - responseEnd
FinisheTime = backendPerformance + frontendPerformance


print(backendPerformance)
print(frontendPerformance)
print(FinisheTime)
print(domperformance)

server.stop()
driver.quit()

执行后的结果：（太多了，只截图了一部分）

3）以上的代码主要返回的是红色框中的数据，其实工作中需要用到的是绿色框中的数据，但是没有拿到（。。。。。。）