Lately, we more and more hear that not every UI test needs to be automated. I tend to disagree with this statement. Even though there are cases where automating something will take longer than checking it manually I will still shoot for automation. Here are my reasons for that.
Number one and most important reason is developing a habit of automation. It’s just human nature to avoid difficult tasks, and if you have reason to excuse yourself from doing it (like for example make a statement “not everything has to be automated”), most likely that is exactly what’s going to happen.
Reason number two I think people who advocate for manual testing either never done late-night deployments or they drink lots of energetics to stay awake and focused at midnight. It is just way easier to get something done right once and then have confidence it is not going to be missed during late-night deployment because I got tired or not feeling well enough. At the end of the day, we all humans and automation are here to help us be more productive.
Number three is test coverage and proof of validation. Let’s say you have four environments to test against and the common practice for those who do testing manually is to reduce the number of tests once you get to upper environments. What do I mean by that? Let’s say you have 50 stories in your release candidate, typically you would test against all 50 of them on development and testing environments, moving up to pre-staging and staging environments (in my current work there is 3 of them) the upper you move the less you test, narrowing down to core functionalities in this release. The reason for doing it that way is to reduce testing time and sign off on release with a good confidence level.
But let’s get back to automation, in 2020 I aiming for release automation, meaning this automation will be used only once (or actually few times, every time we deploy RC to a new environment) during this specific release. Most of the tests from this test suite not going to be included in the regression suite. Of cause, some tests will be migrated and will be running in regression, and having them in release automation suite makes it way either. Why not include it right away to regression? Just to avoid rollbacks of automation code in case RC didn’t pass through all stages and would fail by the end of the release cycle.
There are a few different types of UI automation:
- Release Test Automation Suite – it is automated tests that are covering single release, to make it fast and easy, record it using Selenium IDE. The test suite never gets to CI/CD pipeline and runs locally. Of cause there are some limitations which some times create a problem, use JavaScript tools like Tampermonkey to help with that. Few examples of where to use Tampermonkey is Multi-Factor Authentication (MFA) and custom waits (where built-in Selenium IDE Waits is not enough)
- Standard regression tests typically are done through Selenium WebDriver (or Cypress) covering core functionalities to make sure customers get everything working as expected despite changes in the upcoming release.
- Visual Testing Regression – machine learning approach where you look for any visual changes and your framework makes the decision of either accept the change or raise an error.
Of cause, there is another type of testing need to be done in order for a release to pass through the cycle and it can include security, performance/load testing, API testing, backend, microservices integration, unit testing, etc. But it is not the scope of this post and we will leave for the next posts. As for now hope this post was useful and if not feel free post comment with your thoughts on this topic.
