selenium并行
I created my first Selenium Grid about five years ago. The whole setup (see picture above) consists of only two MacBook Pros. The first laptop served as the grid hub as well as the node, and the second laptop as the second grid node. Both nodes were set up to launch and interact with Chrome, Firefox, Android emulators, and iOS simulators for UI testing by installing the software and their dependencies directly onto the operating system. Simple as it sounds, it was, in fact, complicated to set up and worse still to maintain.
大约五年前,我创建了我的第一个Selenium Grid。 整个设置(请参见上图)仅包含两个MacBook Pro。 第一台笔记本电脑充当网格集线器和节点,第二台笔记本电脑充当第二个网格节点。 通过将软件及其依赖项直接安装到操作系统上,两个节点均被设置为启动和与Chrome,Firefox,Android仿真器和iOS仿真器进行交互以进行UI测试。 听起来很简单,但实际上设置起来很复杂,而且维护起来更糟。
In this article, we will explore Selenium Grid and Selenoid/Moon for UI parallel testing on web and mobile apps, their integration with the CI/CD pipeline, the manners that the resources can be arranged to be effective while scalable, the pain points that have been addressed, and those that are still outstanding. UI testing is not limited to just implementations based on the Selenium protocol; there are exciting emerging technologies, like Playwright. However, this is a fairly broad topic and Selenium is still the most widely used UI automation framework, having been around since 2004. This article will be the first in the series dedicated to parallel UI testing. I will be writing about how to implement some of these solutions in the next articles, so stay tuned.
在本文中,我们将探讨Selenium Grid和Selenoid / Moon在Web和移动应用程序上进行UI并行测试,它们与CI / CD管道的集成,在可扩展的同时可以有效安排资源的方式,痛苦点。已经解决,还有那些仍然很出色。 UI测试不仅限于基于Selenium协议的实现; 有令人兴奋的新兴技术,例如Playwright。 但是,这是一个相当广泛的主题,Selenium自2004年以来一直是使用最广泛的UI自动化框架。本文将是致力于并行UI测试的系列文章中的第一篇。 在下一篇文章中,我将撰写有关如何实现其中一些解决方案的文章,敬请关注。
本地并行测试 (Local Parallel Testing)
Before we jump into the server-side arrangement, let’s consider the client-side, because parallel testing still won’t happen until your test framework can kick off the tests in parallel.
在进入服务器端安排之前,让我们考虑一下客户端,因为直到您的测试框架可以开始并行测试,并行测试仍然不会发生。
There are two ways to do this:
有两种方法可以做到这一点:
- The first way is via forking. Instead of spawning multiple threads, new processes are created if the test classes exist. In the world of Java, the straightforward ways to do this are with Maven via the failsafe plugin, and with Gradle via the maxParallelForks option. Let’s say your test framework is using Gradle with maxParallelForks = 2, and you have at least two test runner classes. When you kick off the job in Jenkins to run the tests in Chrome, two Chrome browsers will launch and run the tests simultaneously. 第一种方法是通过分叉。 如果存在测试类,则会创建新的进程,而不是生成多个线程。 在Java的世界中,实现此目的的直接方法是通过Maven通过故障安全插件,通过Gradle通过maxParallelForks选项。 假设您的测试框架使用的maxParallelForks = 2的Gradle,并且您至少有两个测试运行器类。 当您在Jenkins中启动要在Chrome中运行测试的工作时,两个Chrome浏览器将同时启动并运行测试。
- The second way is to split the tests into batches so you can have each batch targeting a different browser on the same or different displays. One way of splitting the tests is by doing it manually by hands, creating Maven profiles targeting each batch, and finally configuring the Jenkins pipeline to launch simultaneously in multiple jobs and combine the test results at the end. 第二种方法是将测试分成多个批次,这样您就可以使每个批次针对相同或不同显示器上的不同浏览器。 拆分测试的一种方法是手动进行测试,创建针对每个批次的Maven配置文件,最后配置Jenkins管道以在多个作业中同时启动并最终合并测试结果。
You could use one of the two methods discussed, or you could use both of them together. So if you have two batch jobs in addition to the fork count of two, the total number of browsers running simultaneously will be four.
您可以使用所讨论的两种方法之一,也可以一起使用。 因此,如果除了两个叉计数之外还有两个批处理作业,则同时运行的浏览器总数将为四个。
You don’t need the Selenium Server package to run tests on browsers locally. However, you need Appium Server to run tests on Android and iOS, both locally and remotely.
您不需要Selenium Server软件包即可在本地浏览器上运行测试。 但是,您需要Appium Server才能在Android和iOS上本地和远程运行测试。
Prior to Appium 1.7, the only way to launch multiple virtual and real mobile targets on the same host is by installing multiple Appium Servers, starting them up on different ports, and pairing one target to one Appium Server port. Check out the first diagram in this article, where Node B has three Appium Servers configured on ports 4723, 4724, and 4725 for one simulator and two emulators.
在Appium 1.7之前,在同一主机上启动多个虚拟和实际移动目标的唯一方法是安装多个Appium Server,在不同端口上启动它们,并将一个目标与一个Appium Server端口配对。 请查看本文中的第一张图,其中节点B在端口4723、4724和4725上为一个模拟器和两个模拟器配置了三个Appium服务器。
With Appium 1.7 and above, one Appium Server can launch multiple targets by connecting multiple Android devices to one Appium Server on different systemPorts, then specify this capability in your client code to connect to the intended target. For iOS devices, the driver capability is called wdaLocalPort. Note that you are only able to launch more than one iOS simulators on the same host from Xcode 9 onwards.
使用Appium 1.7及更高版本,一台Appium Server可以通过将多个Android设备连接到不同systemPorts上的一台Appium Server来启动多个目标,然后在客户端代码中指定此功能以连接到预期的目标。 对于iOS设备,驱动程序功能称为wdaLocalPort。 请注意,从Xcode 9开始,您只能在同一主机上启动多个iOS模拟器。
This is the simplest form of parallel testing as it can all happen in one single host. But we are looking for more hosts and more variety of OS/browser combinations to run simultaneously. Furthermore, you cannot be sure that a single host can reliably run multiple targets at the same time. In fact, changing the focus on multiple targets may actually slow things down.
这是并行测试的最简单形式,因为它都可以在一个主机中进行。 但是,我们正在寻找更多的主机和更多种类的OS /浏览器组合来同时运行。 此外,您不能确定单个主机可以可靠地同时运行多个目标。 实际上,改变对多个目标的关注实际上可能会使事情变慢。
VM / Real上的Selenium Grid (Selenium Grid on VM/Real)
Enter Selenium Grid, one of the components in the Selenium suite that is made up of a hub and connecting nodes. The hub acts as the proxy that accepts requests from the WebDriver client and routes the JSON commands to the remote drivers on the nodes to run the tests.
输入Selenium Grid,Selenium套件中的一个组件,它由一个集线器和连接节点组成。 集线器充当代理,接受来自WebDriver客户端的请求,并将JSON命令路由到节点上的远程驱动程序以运行测试。
The assignments of the hub and nodes happen during the start-up of the Selenium Server package. By default without any custom configurations, starting the node allows for concurrent use of 5 Firefox, 5 Chrome, and 1 Internet Explorer. You are also able to configure your Appium Server to subscribe to the grid hub, thus allowing for remote execution of tests on Android emulators, iOS simulators, and real devices.
集线器和节点的分配在Selenium Server程序包启动期间进行。 默认情况下,没有任何自定义配置,启动节点允许同时使用5个Firefox,5个Chrome和1个Internet Explorer。 您还可以将Appium Server配置为订阅网格中心,从而允许在Android模拟器,iOS模拟器和真实设备上远程执行测试。
With a distributed grid, you reap the following benefits:
使用分布式网格,您可以获得以下好处:
- Run your tests against different browsers and operating systems at the same time. 同时针对不同的浏览器和操作系统运行测试。
- In the event that a node goes down, the hub will forward the tests to the other nodes so no last-minute clamoring to procure another machine, install the needed packages, configure the node capabilities, subscribe to the hub, and finally run your tests. 万一某个节点发生故障,集线器会将测试转发到其他节点,这样就不会在最后一刻大声疾呼来购买另一台机器,安装所需的软件包,配置节点功能,订阅集线器并最终运行测试。
As I was saying at the beginning of the article, I started out with Selenium Grid by manually installing the necessary software directly onto the operating system, which can either be a host OS or a guest OS on a hypervisor. Here’s what’s wrong with this setup:
就像我在文章开头所说的那样,我首先通过将必需的软件直接手动安装到操作系统上来开始使用Selenium Grid,该操作系统可以是虚拟机管理程序上的主机OS或客户机OS。 这是此设置的问题:
- The most glaring weakness is the grid hub. It’s a single point of failure. If it goes down, there’s no other way of reaching all the nodes. And no, you can’t put more than one hub behind a load balancer for fault-tolerance, because Selenium Server is stateful and the instances do not share the sessions with each other. 最明显的缺点是网格中心。 这是单点故障。 如果出现故障,则没有其他方法可以到达所有节点。 不,您不能在负载平衡器后面放置多个集线器以实现容错功能,因为Selenium Server是有状态的,并且实例之间不会共享会话。
The solution does not scale. The grid hub has often been cited as an inefficient Java implementation that can only support a few nodes. Throwing more power into the mix to achieve performance through vertical scaling is not going to help and does not justify the cost. This article suggests using a VM with only 2 VCPUs and 4Gb of RAM as the hub.
解决方案无法扩展。 网格中心通常被认为是一种仅支持几个节点的低效Java实现。 通过垂直扩展将更多的精力投入到组合中以实现性能,这将无济于事,也无法证明成本。 本文建议使用仅具有2个VCPU和4Gb RAM的VM作为集线器。
- You cannot add additional nodes easily. You need to manually repeat the installation and configuration tasks on each new node that you want to add to the grid. 您无法轻松添加其他节点。 您需要在要添加到网格的每个新节点上手动重复安装和配置任务。
- Frustration at version synchronization. For example, the latest web driver may not work together with your browser version, especially if you want to test an older browser version. Or, you might find that you have to upgrade the Appium Server to test a new Android OS. 版本同步时感到沮丧。 例如,最新的Web驱动程序可能无法与您的浏览器版本一起使用,特别是如果您想测试旧版的浏览器。 或者,您可能会发现必须升级Appium Server才能测试新的Android OS。
- Wasted resources. The hub and all its nodes are up and running all the time — unless you manually stop and start them up one-by-one when you need to run some tests. 浪费资源。 集线器及其所有节点始终处于运行状态,除非需要进行某些测试时手动逐个停止并启动它们。
- Stale resources. It is also a known fact that the grid ends up in a poor state after a while so a manual restart is required every now and then. 资源过时。 众所周知的事实是,网格会在一段时间后处于不良状态,因此有时需要手动重新启动。
- How do you install and test different versions of the same browser on the same operating system? The answer is, you don’t. An update on Windows OS removes the older versions of IE and Edge. 如何在同一操作系统上安装和测试同一浏览器的不同版本? 答案是,你不会。 Windows OS上的更新会删除IE和Edge的旧版本。
- The grid hub does not have a way to auto-recover Appium Server (for mobile targets) if it crashes. 如果崩溃,网格中心无法自动恢复Appium Server(用于移动目标)。
- Because the VM is up all the time, browsers are subject to memory running out and process leaks. You will eventually get a slow web browsing experience or a browser that takes a long time to start. 由于VM一直处于启动状态,因此浏览器受内存不足和进程泄漏的影响。 最终,您将获得缓慢的Web浏览体验或启动时间较长的浏览器。
- Virtual machines like VirtualBox and VMWare are introduced to lower cost and increase efficiency by having multiple operating systems yet isolated environments on the same machine. VMs, however, being a full copy of an operating system means that they can take up a lot of system resources. 引入了VirtualBox和VMWare之类的虚拟机,以通过在同一台计算机上安装多个操作系统但隔离的环境来降低成本并提高效率。 但是,VM是操作系统的完整副本,意味着它们会占用大量系统资源。
For anyone interested, further explanation of the mechanics of this solution can be found on my Github page.
如果您有兴趣,可以在我的Github页面上找到有关此解决方案机制的进一步说明。
In the upcoming release of Selenium, Selenium Grid 4 will be a complete rewrite. In addition to the traditional hub-node mode, the new grid will support a standalone mode that starts everything up locally, and the fully distributed mode that exposes the original internals of the hub by offering each component (the router, session map, and distributor) as a process of its own. A distributed mode will allow you to debug and troubleshoot better when something goes wrong. It will also enable you to better utilise the features of modern technologies like Docker, Kubernetes, and the cloud.
在即将发布的Selenium中, Selenium Grid 4将被完全重写。 除了传统的集线器节点模式之外,新的网格还将支持独立模式(该模式在本地启动所有操作)和完全分布式模式,该模式通过提供每个组件(路由器,会话映射和分发服务器)公开集线器的原始内部结构。 )作为自己的过程。 当出现问题时,分布式模式将使您能够更好地进行调试和故障排除。 它还将使您能够更好地利用Docker,Kubernetes和云等现代技术的功能。
Selenium-Docker (Selenium-Docker)
With the advent of containerization as a way to standardise the packaging of applications and dependencies to run across different computing environments, I soon moved on to using Selenium-Docker, container images that the people at Selenium so graciously provide us with.
随着容器化的出现,它成为标准化应用程序和依赖项的包装以在不同的计算环境中运行的一种方式,我很快转而使用Selenium-Docker,Selenium的工作人员如此慷慨地为我们提供了容器镜像。
The advantages of using Selenium-Docker are obvious:
使用Selenium-Docker的优势显而易见:
- It is now easier to scale up the number of node containers, and starting and stopping them via docker-compose. 现在,可以更轻松地扩展节点容器的数量,并通过docker-compose启动和停止它们。
- You can avoid the pain of setting up and maintaining the grid. Just specify the browser version you want to use and all the required dependencies are ready in the image. You also do not have to grapple with version synchronization. 您可以避免设置和维护网格的麻烦。 只需指定您要使用的浏览器版本,图像中所有必需的依赖项便已准备就绪。 您也不必进行版本同步。
- More stable testing since the grid is not left running for a long period of time where there is a higher chance for something to go down. 测试更加稳定,因为网格不会长时间处于运行状态,因此发生故障的可能性更高。
- No wasted resources. The test resources are created on-demand and destroyed when testing completes. 没有浪费的资源。 测试资源按需创建,并在测试完成时销毁。
- You can choose to focus on running the tests on only one target per container since it is easy to spin up a new one. In fact, this is preferred because running tests on multiple targets in the same node is not reliable. 您可以选择集中精力在每个容器的一个目标上运行测试,因为很容易启动一个新的目标。 实际上,这是首选,因为在同一节点中的多个目标上运行测试并不可靠。
- An ephemeral test architecture means you can potentially get rid of the staging or QA testing phase. Developers or testers are likely more keen to test the feature branch if they can quickly spin up a container app and a container Selenium Grid in the CI. Each branch is accepted and committed when the tests pass, therefore there is no need to have another test cycle that tests the develop or main branch. This also encourages early feedback in the development cycle. 临时测试体系结构意味着您可以摆脱暂存或QA测试阶段。 如果开发人员或测试人员可以快速启动CI中的容器应用程序和容器Selenium Grid,则他们可能更愿意测试功能分支。 测试通过时,每个分支都会被接受并提交,因此,无需再有另一个测试周期来测试开发或主分支。 这也鼓励了开发周期中的早期反馈。
The outstanding issues that cannot be resolved by Selenium-Docker in the hub-node mode are:
Selenium-Docker在中心节点模式下无法解决的突出问题是:
- The fact remains that the hub can only handle a limited number of nodes, rendering the ability to add more nodes easily to the grid useless. 事实仍然是,集线器只能处理有限数量的节点,从而使向网格轻松添加更多节点的功能变得毫无用处。
- The hub is still a SPOF. 集线器仍然是SPOF。
- You can optionally use container orchestration tools like Kubernetes for auto-scaling, auto-recovery, and monitoring of the Selenium containers but it’s pointless to use something as powerful as that when the grid hub is not designed to be fault-tolerant and unable to handle many nodes. 您可以选择使用Kubernetes之类的容器编排工具来自动缩放,自动恢复和监视Selenium容器,但是当网格集线器不被设计为容错且无法处理时,使用功能强大的工具毫无意义。许多节点。
Selenium化物 (Selenoid)
Selenoid was born to address the weaknesses of the grid hub. Selenoid solves the problem by basically getting rid of Selenium Server, whose sole responsibility is to launch and proxy all Selenium sessions to the web drivers. Previously, Selenium Server was doing more than just that but with the appearance of geckodriver for Firefox in addition to chromedriver for Chrome and safaridriver for Safari, things have become much simpler.
Selenoid的诞生是为了解决网格中心的弱点。 Selenoid通过基本上摆脱Selenium Server来解决该问题,Selenium Server的唯一职责是启动所有Selenium会话并将其代理到Web驱动程序。 以前,Selenium Server不仅可以做更多的事情,而且除了用于Chrome的chromedriver和用于Safari的safaridriver之外,还出现了Firefox的geckodriver和safaridriver的出现。
These attributes make Selenoid a superior solution:
这些属性使Selenoid成为出色的解决方案:
- It is a lightweight daemon written in golang, in contrast to Selenium Server’s bulkiness. 与Selenium Server的庞大性相比,它是用golang编写的轻量级守护程序。
- Selenoid’s Configuration Manager makes it really simple to install Selenoid and the containers. All it takes is just one command to install and start up a daemon that launches the containers based on the desired capabilities, then forwards the test requests to the Webdriver in the containers. Selenoid的配置管理器使安装Selenoid和容器非常简单。 只需一个命令即可安装和启动守护程序,该守护程序将根据所需功能启动容器,然后将测试请求转发到容器中的Webdriver。
- It spins up a new browser container for each test, providing a fresh browser each time in an isolated and stable environment. You no longer have to worry about browser cache and setting affecting another test. 它为每次测试启动一个新的浏览器容器,从而在隔离且稳定的环境中每次提供一个新的浏览器。 您不再需要担心浏览器缓存和设置会影响另一个测试。
- Each Selenoid, by default, runs five parallel sessions. This can be configured based on your hardware capability. The recommended setting is the number of CPU cores * 1.5 - 2. When used with Ggr, a load balancer also by the same people, to distribute load across multiple Selenoid instances, you can create a Selenoid cluster that truly scales. 每个Selenoid默认情况下都运行五个并行会话。 可以根据您的硬件功能进行配置。 推荐的设置是CPU内核数* 1.5-2。当与同样由同一个人使用的负载均衡器Ggr一起使用以在多个Selenoid实例之间分配负载时,您可以创建真正可扩展的Selenoid群集。
- The Selenoid UI is offered alongside Selenoid where you can view live browser screens, session logs, and browser consumption for troubleshooting purposes. Selenoid UI与Selenoid一起提供,您可以在其中查看实时浏览器屏幕,会话日志和浏览器使用情况以进行故障排除。
- Available up-to-date Docker images for Firefox, Chrome, Opera, and Android. 适用于Firefox,Chrome,Opera和Android的最新Docker映像。
Next comes the question — what can you not do with Selenoid?
接下来是问题-Selenium油能做什么?
- There is no value in using Selenoid with container orchestration tools. Selenoid launches browser containers using Docker API therefore all the browsers will be started only on one Kubernetes node. 将Selenoid与容器编排工具一起使用没有任何价值。 Selenoid使用Docker API启动浏览器容器,因此所有浏览器将仅在一个Kubernetes节点上启动。
- Selenoid is only meant for cross-browser testing, not for mobile apps. Selenoid仅用于跨浏览器测试,不适用于移动应用程序。
月亮 (Moon)
Moon is a commercial product by the same people at Aerokube that works similarly to Selenoid but intended to work with Kubernetes and OpenShift. You can use it for free by running up to four parallel browser pods.
Moon是Aerokube的同一个人的商业产品,其工作方式与Selenoid类似,但旨在与Kubernetes和OpenShift一起使用 。 您最多可以运行四个并行的浏览器容器来免费使用它。
The features that make Moon even better than Selenoid:
使Moon甚至比Selenoid更好的功能:
- Instead of launching the containers via Docker API, Moon starts and stops browser pods using the Kubernetes API. Moon没有使用Docker API启动容器,而是使用Kubernetes API启动和停止了浏览器容器。
- Moon is completely stateless. This allows replicas of Moon across data centres. Browser sessions are kept alive when one or more replicas go down. 月亮是完全无状态的。 这样就可以跨数据中心复制Moon。 当一个或多个副本出现故障时,浏览器会话将保持活动状态。
- Auto-scaling based on the load. 根据负载自动缩放。
Moon is available in the DigitalOcean and Google Cloud marketplace. With just a few clicks, you can have a Kubernetes cluster installed with Moon. To use another cloud provider, you can create a Kubernetes cluster then apply the Moon ready to use manifests.
在DigitalOcean和Google Cloud市场上都可以找到Moon。 只需单击几下,便可以在Moon上安装Kubernetes集群。 要使用其他云提供商,您可以创建一个Kubernetes集群,然后应用Moon随时可以使用清单。
Similar to Selenoid, Moon does not provide supports for mobile targets.
与Selenoid相似,Moon不提供对移动目标的支持。
云解决方案 (Cloud Solutions)
Cloud testing platform providers like Sauce Labs, BrowserStack, and Perfecto Mobile enable users to run web and mobile tests in the cloud. Most people who turned to these providers do so because:
Sauce Labs,BrowserStack和Perfecto Mobile等云测试平台提供商使用户可以在云中运行Web和移动测试。 求助于这些提供程序的大多数人这样做是因为:
- They support almost every operating system under the sun, including Safari, IE, and Edge. 它们支持阳光下几乎所有的操作系统,包括Safari,IE和Edge。
- They take care of everything — like device upgrades, maintenance, connectivity, and charging — so you can avoid situations where you are asked to confirm software upgrades during a test run (on a real device). 他们会处理一切(例如设备升级,维护,连接和充电),因此您可以避免在测试运行期间(在真实设备上)要求您确认软件升级的情况。
- Integration with popular CIs. 与流行的CI集成。
- They are usually very quick to offer beta versions of upcoming new releases. 他们通常很快就会提供即将推出的新版本的Beta版本。
- Sauce Labs offers open-source projects access to a wealth of browsers, real and virtual devices for free (up to five concurrent sessions for testing in parallel). Sauce Labs使开源项目可以免费访问大量浏览器,真实和虚拟设备(最多五个并发会话以并行测试)。
- High availability by Sauce Labs by configuring a pool of Sauce Connect tunnels. 通过配置Sauce Connect隧道池,由Sauce Labs实现高可用性。
They sound like they are the answers to almost all the problems in our previous solutions. So why wouldn’t you want to sign up with one of them?
他们听起来像是我们先前解决方案中几乎所有问题的答案。 那么,您为什么不愿意与其中之一注册?
- First of all, they are not cheap (if you are not open source) and that is expected because the real and virtual OSes and devices of your choice are reserved for you and only you until your tests complete. 首先,它们并不便宜(如果您不是开源的话),这是可以预料的,因为您选择的真实和虚拟OS和设备是为您保留的,只有您保留,直到测试完成。
- Sauce Labs spins up a new VM for every test. VMs are slower than containers that have less overhead. The delays can be significant when you are running multiple tests. Sauce Labs每次测试都会启动一个新的VM。 虚拟机比开销较小的容器慢。 当您运行多个测试时,延迟可能会很严重。
- It can be rather complicated to set up and maintain if your application is behind your corporate’s firewall or there is an existing internal network proxy. 如果您的应用程序位于公司的防火墙后面或存在现有的内部网络代理,则设置和维护可能会非常复杂。
BrowserStack publishes the advantages of Selenium Cloud over Selenium Grid here. How often do you find yourselves requiring access to the phone’s camera for automated UI testing?
BrowserStack在此处发布了Selenium Cloud优于Selenium Grid的优势。 您多久发现自己需要访问手机的摄像头以进行自动UI测试?
- They do take most of your maintenance pain away but not all. When something goes wrong, you still need to spend time analyzing the logs to figure out if it is your problem or their problem before hitting the red button to alert them that something has gone wrong at their end. 它们确实消除了您的大部分维护难题,但并非全部。 当出现问题时,您仍然需要花费时间分析日志以找出是您的问题还是问题,然后单击红色按钮以警告他们末端出现了问题。
非Selenium解决方案 (Non-Selenium Solutions)
It’s worth briefly talking about parallel testing support in other non-Selenium implementations. The popular ones are Playwright by Microsoft, Puppeteer by Google, and Cypress.
值得简要讨论一下其他非Selenium实现中的并行测试支持。 最受欢迎的是Microsoft的Playwright ,Google的Puppeteer和Cypress 。
Playwright is sort of a successor of Puppeteer, as the main developers of Playwright were working on Puppeteer before jumping ship. Playwright has supports for Chromium (Google Chrome and the new Microsoft Edge), Firefox, and WebKit. Meanwhile, Cypress supports the first two browsers, and Puppeteer currently only supports Chromium.
Playwright是Puppeteer的继任者,因为Playwright的主要开发人员在跳船之前就在Puppeteer上工作。 Playwright支持Chromium(Google Chrome和新的Microsoft Edge),Firefox和WebKit。 同时,赛普拉斯支持前两种浏览器,而Puppeteer当前仅支持Chromium。
Playwright is the main contender to Selenium that covers a wide variety of browsers, with version 1.0 released just a few months ago. In Playwright, JSON objects are sent over WebSocket that is kept alive throughout, as opposed to Selenium’s approach that sends an HTTP request for every test command.
Playwright是Selenium的主要竞争者,它涵盖了各种浏览器,而几个月前才发布了1.0版。 在Playwright中,JSON对象是通过始终保持活动状态的WebSocket发送的,这与Selenium的为每个测试命令发送HTTP请求的方法相反。
One of the features in Playwright that makes it a game-changer is a new concept called browser context. A browser context can host multiple web views and contextual behaviour, emulate mobile viewpoints, and mock geolocation. This enables multi-page scenarios across multiple device configurations such as iPhone, iPad, and desktop in parallel. All this is made possible with just a single instance of a browser.
Playwright的一项使游戏规则改变的功能是一个称为浏览器上下文的新概念。 浏览器上下文可以承载多个Web视图和上下文行为,模拟移动视点以及模拟地理位置。 这样可以跨多个设备配置(如iPhone,iPad和台式机)并行执行多页方案。 仅通过浏览器的一个实例,所有这些都可以实现。
Cypress is another popular cross-browser automation implementation with parallel testing support in a licensed version that is worth looking into.
赛普拉斯是另一种流行的跨浏览器自动化实现,在值得许可的许可版本中具有并行测试支持。
结论 (Conclusion)
So there you have it. The six different ways to run UI automated tests in parallel. To recap, these are:
所以你有它。 并行运行UI自动化测试的六种不同方式。 回顾一下,这些是:
- Local runs with the forking and batching methods on the client-side, and installing Appium Server for simultaneous testing on multiple mobile targets. Local在客户端使用分叉和批处理方法运行,并安装Appium Server以在多个移动目标上同时进行测试。
- Remote runs via Selenium Grid on VMs in the hub and node mode. 远程通过Selenium Grid在集线器和节点模式下在VM上运行。
- Remote runs via selenium-docker. We have established that there is little point in running dockerised Selenium Grid on container orchestration tools prior to Selenium Grid 4. 通过selenium-docker远程运行。 我们已经确定,在Selenium Grid 4之前的容器编排工具上运行dockerized Selenium Grid毫无意义。
- Remote runs via Selenoid, working with Docker API directly. 远程通过Selenoid运行,直接与Docker API一起使用。
- Remote runs via Moon, working with Kubernetes and OpenShift. 通过Kubernetes和OpenShift可以通过Moon运行远程运行。
- Remote runs via Cloud test providers such as Sauce Labs and BrowserStack. 通过云测试提供商(例如Sauce Labs和BrowserStack)进行远程运行。
Three additional ways that will be made possible by the upcoming release of Selenium Grid 4 are:
即将发布的Selenium Grid 4将使三种其他方式成为可能:
- Remote runs via Selenium Grid in a standalone mode. 远程通过Selenium Grid以独立模式运行。
- Remote runs via Selenium Grid in a fully distributed mode (with or without Kubernetes). 通过Selenium Grid以完全分布式模式(带有或不带有Kubernetes)进行远程运行。
- Local runs via Selenium IDE. 通过Selenium IDE在本地运行。
Choosing one over another depends on the scale and maturity of your project, available budget, resources, skills, and time. We have not considered Zalenium, an open-source project that provides extra functions on top of a flexible and disposable Selenium Grid, as a viable option because it is no longer supported as of March 2020.
选择一个项目取决于项目的规模和成熟度,可用预算,资源,技能和时间。 我们尚未考虑将Zalenium这个可行的选项作为一个开源项目,该项目在灵活和一次性的Selenium Grid之上提供额外的功能,因为Zalenium截止2020年3月不再受支持。
We also discussed the pros and cons of each solution. The verdict is that Selenoid/Moon (Solution 4 and 5) is very promising for browser testing based on the Selenium and Playwright APIs.
我们还讨论了每种解决方案的利弊。 结论是Selenoid / Moon(解决方案4和5)对于基于Selenium和Playwright API的浏览器测试非常有前途。
For large scale testing on mobile devices, most people would turn to the Cloud test providers (Solution 6). I would take it with a grain of salt, based on my previous experience with one of them. Having said that, there are simply no other viable options unless your organization has a team dedicated to managing an in-house virtual/real device farm and that is a massive undertaking. There is also no Selenoid/Moon equivalent for mobile app testing. so you are stuck with Selenium Grid (Solution 2 and 3) or multiple targets on a single machine with Appium (Solution 1).
对于在移动设备上进行大规模测试,大多数人会求助于云测试提供商(解决方案6)。 根据我以前对其中一个的经验,我会加一点盐。 话虽如此,除非您的组织有专门负责管理内部虚拟/真实设备场的团队,否则根本没有其他可行的选择,这是一项艰巨的任务。 也没有等效的Selenoid / Moon用于移动应用程序测试。 因此,您只能在一台装有Appium的计算机上使用Selenium Grid(解决方案2和3)或多个目标(解决方案1)。
Perhaps a better way is to look at your UI tests again and ask yourselves these questions to decrease the number of UI tests to run: Should they be covered as lower-level tests instead? Are they the right tests for the code changes?
也许更好的方法是再次查看您的UI测试,并问自己以下问题,以减少要运行的UI测试的数量:是否应该将它们作为低级测试覆盖? 他们是对代码更改的正确测试吗?
翻译自: https://medium.com/better-programming/6-ways-to-do-automated-ui-testing-in-parallel-with-selenium-132e47403c4f
selenium并行