Evaluating Test Automation Tools – Cost Should Be Only a Minor Factor
How are you selecting your tools for test automation?
One of the first steps in software test automation is to evaluate test automation tools. Many organizations key considerations in evaluating test automation tools is cost. We found a table that lists out prices and specific browser support. Since most people read left to right, this may lead you to believe that cost is one of the most important criteria in evaluating test automation tools right?
Not us. We believe that one of the most critical criteria is the ability to recognize objects. The table below is an excerpt of one evaluation we did for a desktop and web-based software application. That’s right, ‘for a desktop and web-based software application’. Evaluating and selecting the testing automation tool that is best for you depends on the software that is under test.
Let’s talk more about evaluating test automation tools for Object Recognition for testing web applications. You’ll need to evaluate which browsers are supported by each tool one by one. We normally check the top 3 popular browsers – Firefox, IE (Edge), and Chrome. Each uses a different rendering engine – Gecko, Trident, and Webkit. Because of this, automation tools need separate methods while working with them.
The table below only gives a very basic and a quick glance of the capability for Object Recognition for desktop and web, but you may need to include mobile test automation as well. For the full evaluation, we are more specific and expand into more details that are more contextual and targeted toward the software under test.
Evaluating Test Automation Tools
Additionally, tools often support some versions and not others. As seen in the table above, we know that Silktest supports recording with IE, but it doesn’t support recording with Firefox at all; and for another tool – Selenium 2.x(Webdriver), it only supports up to Firefox 46 using selenium-server-standalone-2.xx.x.jar. For Firefox 47 or above, you have to use Selenium 3.x with geckodriver.
Another thing you must consider during the evaluation is what is the language or technologies your company is using to develop your application. For example, Flex and Silverlight are very popular for rich web applications. If your company is using one of them, you must be very careful when evaluating their specific capabilities to recognize some Silverlight controls directly.
This means that TestComplete doesn’t support all versions of Silverlight. This same situation may happen with other automation tools as well. Of course, with some extra work like installing a higher version of Silverlight and deploying files from the server side, you can conquer this obstacle. But as a reminder, it’s important to pay attention to all types of UI elements during object recognition.
Of course evaluating and choosing a tool is just one of many steps in a software test automation project. Developing a maintainable software test automation framework that is suitable with the architecture of the software is another of several critical steps.
Leave A Comment