In the fast-paced world of software development, speed is paramount. Being the first to market can make all the difference, allowing you to reap the rewards of your hard work. Software testing is no exception, and end-to-end (E2E) testing is known as the slowest type of testing. In this blog post, we’ll explore the reasons behind the slowness of E2E tests and discuss strategies to enhance their performance.
Reasons for Slow E2E Tests
1. Too Many Layers
Typical web applications consist of a minimum of three layers: frontend, backend, and database. Testing each layer adds complexity and time to the testing process.
2. Testing Purpose
E2E tests cover user flows, including business logic, which often involves multiple actions before results can be asserted. This complexity increases testing time.
3. Data Setup and Cleanup
E2E testing frequently requires extensive data setup and cleanup procedures, further slowing down the testing process.
4. Poorly Written Tests
Inexperienced software developers and testers tend to write inefficient tests that duplicate functions and handle data setup via the UI layer, leading to slowness.
5. Flaky Tests
Flaky tests, which can be a result of poor test design, add uncertainty and delay to the testing process. Take a look at my blog post on how to fix flaky e2e tests.
The Importance of Speed in E2E Testing
You might wonder why speed matters if tests are working as expected. Here are two crucial reasons:
1. Feedback Loop
Imagine you complete your work at the end of the day or week and want to merge your changes. If testing takes too long, you either have to wait or postpone the merge until the next day. This increases the risk of stale branches and slows down development.
2. CI Cost
If you are using a cloud CI solution, you are likely paying based on the time you use cloud resources. Slow-running tests can significantly increase CI costs.
Strategies for Improving E2E Test Performance
Now that we understand why E2E tests are slow and why speed matters, let’s explore strategies to enhance test performance.
1. Parallel Test Execution
Running tests in parallel can significantly speed up testing, but it requires careful consideration to avoid race conditions and assess whether the setup time justifies the benefits. If your application is robust enough you can just balance the load and equally distribute the tests. In some cases, to avoid conflicts you would need to define how to distribute the tests.
2. Preserve Session
Preserving user sessions across test runs can save time by eliminating the need to log in for every test suite. Reusing stored sessions can streamline the testing process. Here is an example of how to use preserve sessions in Cypress
3. Single Responsibility Testing
Avoid redundant actions such as repeatedly logging in through the UI for every test. Use API endpoints to perform tasks more efficiently, like obtaining tokens or navigating directly to specific pages. Let’s look at the example of an online store and testing of a product page. Inexperienced engineers would go through a search page and look for the product. Unless testing search is the goal of your tests this step is unnecessary and doesn’t serve any purpose, instead, the engineer should open the product page directly and do the testing.
4. Data Seeding or Setup through API
Whenever possible, use data seeding or set up data using API requests instead of UI forms. This approach reduces the time needed to prepare test data.
5. Define Levels of Testing by Criticality
Define testing levels based on criticality. Execute Level 1 tests, covering essential flows, whenever someone tries to merge changes to the master branch. Less critical tests can run during nightly regression tests as Level 2 tests, and so on.
6. Disable Video Reports
Modern test automation frameworks allow you to record videos of running tests, although it is a nice feature, it might take up lots of testing time. Consider disabling video compression if it’s not essential for your testing process. In many cases, nobody watches test videos when tests fail, making this feature unnecessary and time-consuming.
7. Disable requests to a 3rd party applications.
Modern applications make lots of requests to third-party tools that help to collect analytical data. As an example Google Analytics, google maps, and so on, unless you testing integrations with it, consider disabling requests to third-party tools. Here is an example of how to do it with Cypress.
In the race to deliver software faster, optimizing the speed of E2E tests is essential. By understanding the reasons behind their slowness and implementing the strategies outlined above, you can significantly improve testing efficiency, reduce feedback loop delays, and minimize CI costs. Speeding up E2E testing ultimately contributes to a more agile and competitive development process.
