I’ve been investing time in researching different end to end testing frameworks. I have been looking for an automation framework that is reliable, fast, cross-browser, and easy to develop. 

Playwright’s relatively new testing framework caught my attention. Playwright is a Microsoft owned and developed framework for end to end testing.

While taking time to learn and understand this framework, I thought I would write a series of posts related to it. 

The first post in the series is an introductory post that will cover what Playwright can do and some cool features that I like. 

I’ll cover adding Playwright to .NET applications that run locally using the WebApplicationFactory and running Playwright against deployed applications in future posts.

What Is Playwright?

The first public version of Playwright was released on January 31st 2020. On May 7th 2020, version 1.0 was released. While early adopters could start using Playwright from January, many features were missing. Version 1.0 contains key features allowing for easier adoption. 

One of the significant changes with v1.0 was cross-browser support. While still being able to test against Chromium, v1.0 now allows us to test against Firefox and WebKit.

Here is a fun fact: Puppeteer’s creators are also the creators of Playwright. One of the significant disadvantages of Puppeteer was that it only supported running tests against Chromium-based browsers.

“We are the same team that originally built Puppeteer at Google, but has since then moved on. Puppeteer proved that there is a lot of interest in the new generation of ever-green, capable, and reliable automation drivers. With Playwright, we’d like to take it one step further and offer the same functionality for all the popular rendering engines. We’d like to see Playwright vendor-neutral and shared governed.”

https://github.com/microsoft/playwright/pull/711/files

Playwright allows us to run tests against the following browsers

• Chromium,
• WebKit 
• Firefox

Playwright supports the following platforms

• Windows, 
• Linux
• macOS

You can use the following languages to write your Playwright scripts

• .NET
• Typescript
• Javascript
• Python
• Java

Develop in the Language of Your Choice

Selecting the language you develop in allows you to build your test suite using a language that your team is comfortable with. To me, this is quite important.  

You might have already experienced the team trying to develop in a language that they are unfamiliar with. When you are not familiar with the language, it is easy to end up with a solution that doesn’t follow good practices. Ultimately you end up with a codebase that is hard to maintain.

I also believe that keeping your tests as close to the code base as possible is a good thing. It helps with visibility. You often have unit and integration tests in your codebase, while the end to end tests live somewhere outside your repo. 

Having the end to end tests outside of your solution can make it harder to identify what each of the test levels is testing. It’s easy to add unnecessary tests to your end to end solution or miss adding tests thinking they were covered in another test level.

Having all tests in the same code base now allows you to view what tests are in each level. If you are like me, I try to move away from end to end tests as much as possible, I only want end to end tests that are valuable. They are often slower and more brittle than unit and integration tests.

Dealing With Flaky Tests

I’ve used several different end to end testing frameworks. Selenium, Test Cafe, Cyprus, to name a few, and we constantly battle with flaky tests.

When tests are flaky, you lose trust in the results. You end up re-running these tests by default without paying much attention to the failure. As a result, the confidence in the tests drop, and no one trusts the tests. Not having faith in the tests is not a good position to be in. 

Every time a test flakes, a developer needs to investigate whether the failure is true or false. If the “flake rate” of your tests is constant, every new test you add represents more time the engineering team will have to spend in the future to maintain that test. – Scott White

https://dev.to/scahhht/why-end-to-end-tests-are-flaky-and-how-to-fix-them-2mio

Playwright offers several features to help combat these flaky tests.

Auto Wait

Most testing frameworks require you to add waits between actions, such as thread.sleep(). Adding delays is necessary as the testing framework attempts to action an element before it’s ready. An example of this is a button. The test loads the page and immediately tries to click the button. However, the page hasn’t fully loaded when the button click is attempted. As a result, the test thinks it has clicked the button and waits for the next element to be loaded. This is a common cause of test flakiness. 

Adding these waits can take time as you run the tests repeatedly to identify where you need to add extra delays. Playwright has auto-waits built in by default. 

Further information on how to implement auto waits in Playwright .NET can be seen on the Playwright .NET documenation – Auto Waits

Tracing

Trying to identify why a test has failed can be time-consuming. Playwright offers the ability to add tracing to your tests. Tracing allows you to easily see each step and process that has taken place during the test run. Adding tracing can reduce the time it takes to work out what has gone wrong. 

Further information on how to implement tracing in Playwright .NET can be seen on the Playwright .NET documentation – Tracing

Recording Videos

Having the ability to watch a video of the test run is helpful to help quickly identify where the failure has occurred. The test videos are saved to a folder you have specified.

Recording and saving videos can use up a lot of disk space. If space is a concern, you can capture videos only on a test failure.

Further information on how to implement videos in Playwright .NET can be seen on the Playwright .NET documenation – Videos

Screenshots

An expected feature of any end to end testing framework is the ability to take screenshots on failures. As you might have guessed, Playwright also offers this. So with screenshots and the other features, tracing and recording videos, Playwright is a perfect solution to identify failing tests.

Further information on how to implement screenshots in Playwright .NET can be seen on the Playwright .NET documenation – Screenshots

Authentication

Another feature that I like with Playwright is the ability to save the authentication after logging in the first time. 

Saving the authentication allows you to log in once, then save that authentication and re-use it in your following tests. This removes the need to log in each time. Your web apps must use cookie-based or token-based authentication for this to work.

Re-using the authentication is useful when you either don’t need to test the login process or you only need to test it once. Not having to log in each time speeds up the test run time.

Further information on how to implement reusing authentication state in Playwright .NET can be seen on the Playwright .NET documenation – Authentication

Page Object Model 

The page object model is a common pattern that is used to help abstract your code. For example, abstracting certain methods helps show the required information to the user while the rest of the information is hidden. The abstractions help separate and simplify the interactions for each web page. 

If you have multiple pages, i.e. Login, Movie and Contact page, you would create classes for each page. You would then capture all element selectors used to interact with the page in this class. This helps with reusability and maintainability. 

To find out more information on the Page Object Model pattern in Playwright .NET, you can be view this in the Playwright documentation.

Playwright .NET documenation – Page Object Model

References:

Playwright compared to other end to end automation frameworks

Testim have done some evaluations around key areas of end to end testing frameworks. They have produced a rating for these components. You can find the detailed list here 

https://www.testim.io/blog/puppeteer-selenium-playwright-cypress-how-to-choose/

It will depend on what areas are most important to you, but Playwright comes out pretty top for most evaluations.  

Applitools Top Questions on Playwright

Applitools has an excellent section on the top questions likely to be asked about Playwright. That can be viewed here

Your 80 Questions about Playwright, Answered