In my most recent talk at the mobile dev and test conference in San Diego, I got a chance to sit with Srilu Pinjala from PNSQC. We discussed the topics at the upcoming conference, and the new “Concept Papers”, that is, papers that can be presented outside the normal technical track. We also discussed some of the areas that are hot in the software world, namely mobile, devops, agile and user experience among others, and what kinds of topics would be cool to have at the conference. One topic that sticks in my mind regarding mobile is implementing mobile test automation.

We often work with clients that want to implement a software test automation effort and their immediate objective is “I want to achieve X% test automation”, where X is often 80 or even 90 percent. Now, as mobile testing tools begin to mature and most organizations have mobile apps, the same mindset is emerging. Unfortunately we have the same problems and issues in implementing mobile test automation as we do and did with web-apps. That is, requirements change, and things move fast, especially with agile where we are doing new builds every day or even every hour via continuous integration. So what to do?

  • Firstly have realistic goals for automation and start small. Don’t set yourself up for failure. Start with a few key functions or set a small goal, say 10%, then move up from there. Keep in mind that automation is not a magic bullet, but rather just another tool in your QA arsenal.
  • Secondly, use a tool that you can grow with. Pick one that works on both iOS and Android because although you may only support one platform now, chances are, you’ll need to support both. Also think about the extensibility of the tool and if you’ll be able to create a framework that can easily grow and expand with your application.
  • Thirdly, think carefully about the features and functions of your application that you want to automate. In doing this, we set up a three dimensional matrix with X = Feature importance, Y = Feature frequency of use, and Z = Feature prone to change.

Perhaps your criteria matrix for implementing mobile test automation will be different, but the main point is automate where is counts and where you can be relatively sure that the application and hence the automation will not change too much.