如何在 Python 上使用 selenium webdriver 和 browsermob 代理捕获网络流量？
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/28169223/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to capture network traffic using selenium webdriver and browsermob proxy on Python?
提问by Jose
I would like to capture network traffic by using Selenium Webdriver on Python. Therefore, I must use a proxy (like BrowserMobProxy)
我想通过在 Python 上使用 Selenium Webdriver 来捕获网络流量。因此，我必须使用代理（如 BrowserMobProxy）
When I use webdriver.Chrome:
当我使用 webdriver.Chrome 时：
from browsermobproxy import Server
server = Server("~/browsermob-proxy")
server.start()
proxy = server.create_proxy()
from selenium import webdriver
co = webdriver.ChromeOptions()
co.add_argument('--proxy-server={host}:{port}'.format(host='localhost', port=proxy.port))
driver = webdriver.Chrome(executable_path = "~/chromedriver", chrome_options=co)
proxy.new_har
driver.get(url)
proxy.har # returns a HAR
for ent in proxy.har['log']['entries']:
print ent['request']['url']
the webpage is loaded properly and all requests are available and accessible in the HAR file. But when I use webdriver.Firefox:
网页已正确加载，并且所有请求都可以在 HAR 文件中访问。但是当我使用 webdriver.Firefox 时：
# The same as above
# ...
from selenium import webdriver
profile = webdriver.FirefoxProfile()
driver = webdriver.Firefox(firefox_profile=profile, proxy = proxy.selenium_proxy())
proxy.new_har
driver.get(url)
proxy.har # returns a HAR
for ent in proxy.har['log']['entries']:
print ent['request']['url']
The webpage cannot be loaded properly and the number of requests in the HAR file is smaller than the number of requests that should be.
网页无法正常加载，HAR 文件中的请求数小于应有的请求数。
Do you have any idea what the problem of proxy settings in the second code? How should I fix it to use webdriver.Firefox properly for my purpose?
你知道第二个代码中代理设置的问题是什么吗？我应该如何修复它以正确使用 webdriver.Firefox 来达到我的目的？
回答by Derek Argueta
Just stumbled across this project https://github.com/derekargueta/selenium-profiler. Spits out all network data for a URL. Shouldn't be hard to hack and integrate into whatever tests you're running.
刚刚偶然发现了这个项目https://github.com/derekargueta/selenium-profiler。吐出 URL 的所有网络数据。应该不难破解并集成到您正在运行的任何测试中。
Original source: https://www.openhub.net/p/selenium-profiler
回答by proprius
For me, following code component works just fine.
对我来说，下面的代码组件工作得很好。
profile = webdriver.FirefoxProfile()
profile.set_proxy(proxy.selenium_proxy())
driver = webdriver.Firefox(firefox_profile=profile)
回答by Mike ASP
I am sharing my solution, this would not write any logs to any file
but you can collect all sort of messages such as Errors,
Warnings, Logs, Info, Debug , CSS, XHR as well as Requests(traffic)
1. We are going to create Firefox profile so that we can enable option of
"Persist Logs" on Firefox (you can try it to enable on your default browser and see
if it launches with "Persist Logs" without creating firefox profile )
2. we need to modify the Firefox initialize code
where this line will do magic : options.AddArgument("--jsconsole");
so complete Selenium Firefox code would be, this will open Browser Console
everytime you execute your automation :
else if (browser.Equals(Constant.Firefox))
{
var profileManager = new FirefoxProfileManager();
FirefoxProfile profile = profileManager.GetProfile("ConsoleLogs");
FirefoxDriverService service = FirefoxDriverService.CreateDefaultService(DrivePath);
service.FirefoxBinaryPath = DrivePath;
profile.SetPreference("security.sandbox.content.level", 5);
profile.SetPreference("dom.webnotifications.enabled", false);
profile.AcceptUntrustedCertificates = true;
FirefoxOptions options = new FirefoxOptions();
options.AddArgument("--jsconsole");
options.AcceptInsecureCertificates = true;
options.Profile = profile;
options.SetPreference("browser.popups.showPopupBlocker", false);
driver = new FirefoxDriver(service.FirefoxBinaryPath, options, TimeSpan.FromSeconds(100));
driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(10);
}
3. Now you can write your logic since you have traffic/ logging window open so don't
go to next execution if test fails. That way Browser Console will keep your errors
messages and help you to troubleshoot further
Browser : Firefox v 61
How can you launch Browser Console for firefox:
1. open firefox (and give any URL )
2. Press Ctrl+Shift+J (or Cmd+Shift+J on a Mac)
Link : https://developer.mozilla.org/en-US/docs/Tools/Browser_Console
