Mobile applications are more and more composite applications. Data is gathered from many different sources and presented to the client. However, most mobile application testing does not take into account all these different sources of information. But it is imperative to test your mobile service as the user will experience it.
We’ve been doing mobile application testing for several years now, especially with smartphones which are now getting more and more powerful and therefore having more complex applications.
Mobile Application Testing – Key Elements
Based on our experience, some of the key aspects of developing a mobile application testing plan include:
1. Functional testing with targeted device: Whether testing a website with mobile service or an app on a mobile device, over 50% stop here* and declare testing a success. But this is a mistake.
2. Testing with a representative set of devices:
The Android world is becoming more fragmented as time goes on. There are now 4000 different unique devices with endless (not really, but it seems that way) permutations of Android. So how do you write Apps in the Android market to cover as many devices as possible without testing on every device? You test with a representative subset of devices. The problem isn’t as bad for Apple, but you still have to test on multiple platforms.
3. Using Service Virtualization to test parts of the application that aren’t accessible: DB or mainframe services of your company provide vital data for your mobile service, but the main IT department won’t let you access them to test. You must then write an intelligent stub that returns comparable data. And then there may be other parts of the application that may not be finished yet. You can’t wait for it to be completed. You must test your part now. The answer is the same as in the DB: Service Virtualization.
4. Using Service Virtualization to test 3rd party services: The 3rd party may charge for access or does not want you to performance test against their service. The solution is the same as above.
5. Performance stress and load testing: Your application will work fine for one user, but what if 100, 1000, or 10,000 users access at the same time? There are good tools which allow you to test for this.
6. Performance testing with different networks: You don’t know ahead of time what network you will be connected to or what the quality of the network will be. It is very hard to repeat network conditions for testing because everything is constantly changing. And then there are global differences in network speeds and reliability. Network emulators will allow you to test all the different kinds of networks with different levels of quality.
Customers don’t care where the problem is. You will get blamed for poor performance even if the fault is the network, a 3rd party data provider, the corporate DB, or the mobile device. And worse of all, what if your mobile service is widely successful and your infrastructure can’t handle the load?
When you do mobile application testing, make sure you consider all of these aspects as the end-user experiences it for a successful deployment of your mobile service.
* Mobile Testing Report 2013, https://xbosoft.com/contact/research/mobile-testing-report-2013